Industrial robot industrial product defect image data augmentation method

Abstract

The application relates to an industrial robot industrial product defect image data augmentation method, wherein after corresponding industrial product close-up images are obtained from cached industrial product long-range images during feature recognition, the industrial product long-range images are reserved, the missing part is judged through part of the clear features obtained from the industrial product close-up images, feature reconstruction information is extracted from the industrial product long-range images through extraction of long-range key information, the extracted feature reconstruction information is input into a feature augmentation model to obtain a reconstructed feature image, then image splicing is carried out, and then learning is carried out through a deep learning algorithm, so that the processing speed and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to automation equipment, more particularly, to an industrial robot industrial product defect image data augmentation method. BACKGROUND

[0002] At present, industrial robots have been widely applied in visual detection technology. On the one hand, image recognition technology is constantly innovating, and on the other hand, the cost of hardware computing power is constantly decreasing, so that visual recognition is no longer a missing link in the industrial intelligent production line. Visual detection plays a crucial role in positioning, sorting, identification and other functional links, and one of the links that is more commonly used in visual robots is the industrial quality inspection link. Since industrial products are diverse, the defects of industrial products are generally identified by humans. Human identification itself is uncertain and prone to errors. At the same time, human identification is based on experience and cognition, and it is difficult to analyze the situations and links that affect the yield in the industrial production line. Therefore, robot visual technology has been widely applied in the quality inspection link.

[0003] When the robot visual technology identifies the surface defects of industrial products, it generally realizes the collection of defect images through the three steps of feature recognition, scaling and adjusting the focal length. However, there are two problems. The first problem is that the scaling of the collection area is performed according to the feature size, but actual defects may be larger, resulting in that the collected image after scaling may only present part of the defects. The second problem is that the consistent focal length may cause the edge to be unclear. These two situations will cause the collected defect image to be unclear, making it difficult to analyze the causes. SUMMARY

[0004] Therefore, the present application aims to provide an industrial robot industrial product defect image data augmentation method.

[0005] In order to solve the above technical problems, the technical scheme of the present application is as follows:

[0006] An industrial robot industrial product defect image data augmentation method, characterized in that:

[0007] Step S1, real-time acquisition of an industrial product long-range image, and identification of defect features of the industrial product long-range image through a pre-set defect feature neural network;

[0008] Step S2, magnifying the industrial product long-range image by a magnification ratio with the position of the defect feature in the long-range image as the center to obtain an industrial product close-range image through feature recognition, and automatically focusing and acquiring a focus parameter;

[0009] Step S3, judging loss information of the industrial product close-up image according to defect features by a preset loss judging strategy, and taking the defect features as indexes to obtain feature reconstruction conditions from a preset feature reconstruction database;

[0010] Step S4, obtaining feature reconstruction information from the industrial product long shot image by the feature reconstruction conditions;

[0011] Step S5, taking the feature reconstruction information into a preset feature augmentation model to obtain a reconstructed feature image;

[0012] Step S6, splicing the reconstructed feature image and the industrial product close-up image to obtain a defect augmented image;

[0013] Step S7, comparing the obtained defect augmented image and the industrial product long shot image to calculate an image similarity value, when the image similarity value is greater than an upper threshold, performing a positive feedback adjustment strategy, and when the image similarity value is less than a lower threshold, performing a negative feedback adjustment strategy.

[0014] Further, the loss judging strategy comprises:

[0015] Step S3-1, calculating a maximum convolution value of each pixel point in the industrial product close-up image according to the defect features;

[0016] Step S3-2, determining an upper limit matching value and a lower limit matching value according to a distribution of the maximum convolution values;

[0017] Step S3-3, marking a pixel point with a maximum convolution value higher than the upper limit matching value as a defect pixel point, marking a pixel point with a maximum convolution value between the upper limit matching value and the lower limit matching value as a reconstruction pixel point, marking a pixel point with a maximum convolution value lower than the lower limit matching value as a redundant pixel point, and dividing a defect pixel point with a distribution mode satisfying a center distribution condition into a center image area;

[0018] Step S3-4, determining a reconstruction pixel point with a distance less than a preset redundancy distance value from the center image area as an adjacent data set, dividing a reconstruction pixel point with a distribution mode satisfying a loss distribution condition in the adjacent data set into a loss image area, and determining a pixel point located at an edge of the industrial product close-up image in the adjacent data set as a loss edge;

[0019] Step S3-5, generating the loss information according to positions of the loss image area and the loss edge.

[0020] Further, the step S4 comprises:

[0021] Step S4-1, determining a starting coordinate in the industrial product long shot image by the loss information;

[0022] Step S4-2, determining a reconstruction contour from the industrial product prospective image by recognizing a contour identification reference in the feature reconstruction condition;

[0023] Step S4-3, determining a plurality of reconstruction features in the reconstruction contour by a reconstruction reference in the feature reconstruction condition, and generating the feature reconstruction information according to the reconstruction features.

[0024] Further, the feature augmentation model comprises a reconstruction neural network, each node of the reconstruction neural network corresponds to a convolution kernel data set, the convolution kernel data set comprises a plurality of deconvolution kernels, each deconvolution kernel corresponds to a convolution priority value, and the nodes of the reconstruction neural network are associated through a progressive index vector.

[0025] Further, the step S5 comprises

[0026] Step S5-1, generating a plurality of reconstruction paths according to the reconstruction features so that the reconstruction paths cover the reconstruction contour;

[0027] Step S5-2, determining a reconstruction image point with the smallest distance from the defect image point as a starting point of the reconstruction path, determining a starting node from the reconstruction neural network according to the corresponding reconstruction feature of the reconstruction path, and determining a matching relationship;

[0028] Step S5-3, in the matching relationship, calculating a convolution matching value of each deconvolution kernel and the image point in sequence of the convolution priority value through a preset convolution fitting algorithm until the convolution matching value reaches a reconstruction sub-condition;

[0029] Step S5-4, calculating an index value of each next-level node through a preset path index algorithm, the index value is positively correlated with a module length of the progressive index vector, determining a next-level node according to the index value and determining a matching relationship with a next image point in the reconstruction path, and re-executing step S5-3 until the matching of all image points in the reconstruction path in the reconstruction neural network is completed;

[0030] Step S5-5, completing the deconvolution of all reconstruction paths to generate the reconstruction feature image.

[0031] Further, the convolution fitting algorithm is: wherein A is the convolution matching value, b is a preset weight purpose parameter, is a maximum convolution value of the current image point after deconvolution calculated through the defect feature, is an upper limit matching value determined in the loss judgment strategy, U is the number of image points in the deconvolution kernel, u is the number of coincident image points of the current image point and the previous image point in the deconvolution algorithm; c is the sequence value of the current image point in the reconstruction sequence; is a maximum convolution value of the previous image point recalculated through the defect feature; The maximum convolution value of the last image point obtained in the last calculation.

[0032] Further, the path index algorithm is: Wherein, B is the index value corresponding to the node, , , Respectively, the preset index weight parameter, there , The module length of the index vector, The trajectory angle of the index path at the current image point, The matching angle of the feature data set in the reconstructed neural network, and z is the information element corresponding to the node.

[0033] Further, the proportion generation algorithm in the step S5-4 is: Wherein, The index proportion value, m is the number of subordinate nodes, The nth subordinate node, The mth subordinate node;

[0034] The step S5-4 further includes configuring an index value range for each subordinate node according to the index proportion value, generating a random number, and determining the corresponding subordinate node to establish the matching relationship according to the random number falling into the index data range.

[0035] Further, the pheromone decay strategy further includes configuring pheromone for each node in the reconstructed neural network, when any node establishes a matching relationship, the corresponding pheromone increases by a preset reinforcement value, and the pheromone of other nodes in the same level decreases by a preset decay value.

[0036] Further, the positive feedback adjustment strategy is to increase the module length of the corresponding progressive index vector in the reconstructed neural network by a positive adjustment value, and the positive adjustment value is proportional to the difference between the image similarity value and the upper threshold value; the negative feedback adjustment strategy is to reduce the module length of the corresponding progressive index vector in the reconstructed neural network by a negative adjustment value, and the negative adjustment value is proportional to the difference between the image similarity value and the lower threshold value.

[0037] The technical effects of the present application mainly reflect the following aspects: after the corresponding industrial product close-up image is obtained through the cached industrial product long-range image in feature recognition, the industrial product long-range image is retained, the missing part is judged through the part clear feature obtained through the industrial product close-up image, the feature reconstruction information is extracted from the industrial product long-range image through extracting the long-range key information, the extracted feature reconstruction information is input into the feature augmentation model to obtain the reconstructed feature image, then image stitching is performed, and then deep learning algorithm is used for learning, so that the processing speed and accuracy are improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A flow chart of an industrial robot industrial product defect image data augmentation method of the present application;

[0039] Figure 2 A step S3 flow principle diagram of the method of the present application;

[0040] Figure 3 A step S4 flow principle diagram of the method of the present application;

[0041] Figure 4 A step S5 flow principle diagram of the method of the present application. DETAILED DESCRIPTION

[0042] The specific embodiments of the present application are further described in detail below with reference to the accompanying drawings, so that the technical solutions of the present application are easier to understand and master.

[0043] An industrial robot industrial product defect image data augmentation method,

[0044] Step S1, real-time acquisition of industrial product perspective image, and identification of defect features of the industrial product perspective image through a pre-set defect feature neural network; the visual robot recognition defect feature neural network has been applied more in industrial product defect recognition, and the present application is to identify the logic of the defect features, and with the optimization of the accuracy of the defect image data augmentation method, the defect feature recognition condition can be correspondingly reduced; the defect features here are individual nodes in the neural network, and it is not said that the presence of a defect feature in the industrial product perspective image means that there is a defect in the industrial product perspective image; the defect feature of the industrial product perspective image meets the condition of triggering the defect in the neural network, and this image area here is judged as a defect, which can be scratch, uneven coloring, etc., and will not be described here.

[0045] Step S2, the position of the defect feature in the close-up image of the industrial product is enlarged by a magnification ratio to obtain an industrial product close-up image for feature recognition, and automatic focusing and focusing parameters are obtained; the center point can be determined by the position and distribution of the defect feature, and the industrial product close-up image can be obtained by enlarging the industrial product far view image by a fixed magnification ratio. The purpose here is to obtain consistent size of the industrial product close-up image, facilitate storage and data processing, and also to focus according to the position of the defect feature, so that the imaging of the points with more defect feature distribution in the corresponding industrial product close-up image is clear; automatic focusing and automatic zooming are hardware technologies that visual robots currently have, and will not be described here. However, it needs to be explained that although this design can obtain clear close-up images, it may miss some large or unclear parts, and if the previous technology is used, it may need to adjust the magnification ratio and focal length to obtain several new images for image stitching. However, this way, the industrial vision robot needs more time, because it needs to perform missing operation first, and then find the surrounding defect part at different magnification ratios through the defect feature neural network after obtaining the result, and then focus again. This process requires the visual angle of the visual robot to be on the industrial product at all times, and more importantly, if the industrial product is also in a moving state during this process, the light and shadow and imaging effect will be different, resulting in errors in the recognition result. Such time waste is not in line with the design requirements of the flow line inspection. Therefore, the far view image and the close-up image are directly spliced based on the feature recognition moment in the present application, so as to increase the data saturation and data clarity in the close-up image as much as possible, and complete the defect image data augmentation:

[0046] Step S3, judging the loss information of the industrial product close-up image according to the defect feature through a preset loss judgment strategy. First, the close-up image obtained is judged for missing information according to the loss judgment strategy

[0047] Step S3-1, calculating the maximum convolution value of each pixel point in the industrial product close-up image according to the defect feature; since the close-up image has higher clarity than the far view image, the matching degree of each pixel point and the defect feature is calculated by a convolution algorithm first. Here, the defect feature neural network is also used, and the maximum convolution value of each pixel point is obtained by convolution operation of each pixel point through the convolution kernel in the defect feature neural network. The maximum convolution value indicates the matching degree of the pixel point and the sample in the defect image. Theoretically, the maximum convolution value tends to 1, indicating that the pixel point and the sample in the defect image have high matching degree.

[0048] Step S3-2, determining the upper limit matching value and the lower limit matching value according to the distribution of the maximum convolution value; after the maximum convolution value of each image is calculated, the following situations exist, 1, the actual defect image point due to unclear imaging, but the maximum convolution value after convolution is low; 2, the actual normal image point due to unclear imaging, which leads to the maximum convolution value after convolution; 3, the actual defect image point, which is located at the edge of the near image, which leads to the maximum convolution value during convolution; 4, the error caused by imaging. In view of the above several situations, the image point type is divided into three categories by the upper limit matching value and the lower limit matching value, and the upper limit matching value and the lower limit matching value are preferably generated according to the fixed magnification ratio and the dynamically generated focusing parameter. In theory, the larger the focusing change, the higher the upper limit matching value, and the lower the lower limit matching value, so that the imaging error can be eliminated.

[0049] Step S3-3, marking the image point whose maximum convolution value is higher than the upper limit matching value as a defect image point, marking the image point whose maximum convolution value is between the upper limit matching value and the lower limit matching value as a reconstruction image point, marking the image point whose maximum convolution value is lower than the lower limit matching value as a redundant image point, and dividing the defect image point whose distribution mode meets the center distribution condition into a center image area; according to the above situation, the image points in the near image are marked into three categories, and the distribution law of the image points is used, for example, the defect image point, the reconstruction image point and the redundant image point are separately distributed except for the image point at the edge position, and the misjudgment information can be filtered out, and the distribution position can be determined according to the distribution of the image points. Since the position corresponding to the defect image point should be the clearest, the area of the defect image point can be determined by the distribution of the defect image point, and the center image area is formed;

[0050] Step S3-4, determining the reconstruction image point with a distance less than a predetermined redundant distance value from the center image area as an adjacent data set, dividing the reconstruction image point in the adjacent data set whose distribution mode meets the loss distribution condition into a loss image area, and determining the image point located at the edge of the near image of the industrial product in the adjacent data set as a loss edge; after the center image area is formed, the relationship of the reconstruction image point can be determined by the distance from the center image area, for example, the high-density area of the reconstruction image point far away has little relationship with the defect image point, and should be identified as different defects or errors, which can be filtered out here. The loss image area is determined by selecting the area where the reconstruction image points are dense and close to the defect image points, that is, the area caused by unclear imaging, and the position located at the edge of the near image is determined as the loss edge.

[0051] Step S3-5, generating the loss information according to the position of the loss region and the loss edge. The loss information can be generated according to the above position, and then the step S3 further includes obtaining the feature reconstruction condition from the preset feature reconstruction database as an index according to the defect feature. The feature reconstruction database is preset, which reflects that based on the known defect feature, the classification is judged under the loss condition, and the information needed to be obtained from the distant view image is determined, for example, the set of defect features is judged as 1 type of wear, and then the corresponding feature reconstruction database can derive the contour recognition reference and the reconstruction reference needed to be extracted for 1 type of wear in reconstruction. The contour recognition reference reflects the basis for extracting the contour from the distant view image, such as the color difference on both sides of the contour, the color of the contour line itself, and the like. The reconstruction reference is to extract internal features, such as finding several parallel or nearly parallel scratch lines in the interior, so that part of the information of the loss region, such as the shape of the edge contour and the internal clues, can be found from the distant view image. For example, the contour shape is an ellipse, and there are three scratch lines in the interior, and the positions of the three scratch lines are determined. However, due to the problem of clarity, the clarity cannot be directly obtained as the same as that in the central region of the close-up image. However, the image can be reconstructed through the above information. Specifically:

[0052] Step S4, obtaining feature reconstruction information from the industrial product distant view image through the feature reconstruction condition. After obtaining the reconstruction condition, the step S4 includes:

[0053] Step S4-1, determining the starting coordinate in the industrial product distant view image through the loss information. According to the position relationship and the magnification ratio of the industrial product close-up image and the distant view image, the corresponding starting coordinate can be determined. If it is a loss edge, the position of the reconstruction image point is taken as the starting coordinate, and if it is a loss region, the coordinate closest to the central region of the region is taken as the starting coordinate.

[0054] Step S4-2, determining the reconstruction contour in the industrial product distant view image through the contour recognition reference in the feature reconstruction condition. The starting coordinate is taken as the basis, the reconstruction contour in the distant view image is recognized through the contour recognition reference, and the basic shape of the contour is obtained.

[0055] Step S4-3, determining a plurality of reconstruction features in the reconstruction contour through the reconstruction reference in the feature reconstruction condition, and generating the feature reconstruction information according to the reconstruction features, so as to obtain the contour. After obtaining the contour, the feature reconstruction information needed in the internal region where the contour is located is found according to the reconstruction reference. The feature reconstruction information is a set of coordinates of the path of the found reconstruction features, for example, three scratches, and each scratch corresponds to a reconstruction path.

[0056] Step S5, the feature reconstruction information is brought into the preset feature augmentation model to obtain a reconstructed feature image; the key of the application is to construct a feature augmentation model for obtaining a clear image, the feature reconstruction information is brought into the feature augmentation model to obtain a reconstructed feature image, the feature augmentation model includes a reconstruction neural network, each node in the reconstruction neural network corresponds to a convolution kernel data set, the convolution kernel data set includes a plurality of deconvolution kernels, each deconvolution kernel corresponds to a convolution priority value, and the nodes of the reconstruction neural network are associated through a progressive index vector. Since the reconstruction path of the restored image can be found from the feature reconstruction model, the reconstruction path sequentially restores the image points in the corresponding reconstruction contour from the starting point to the ending point. Since one image point in the long-range image corresponds to a plurality of image points in the near-range image, the reconstruction path needs to be determined in the near-range image first. The reconstruction path sequentially passes through the image points to complete the deconvolution calculation of the image point corresponding features, so that a clear image can be obtained. The reconstruction strategy is pre-constructed by the reconstruction neural network according to the sample training mode, that is, after the starting node is determined, it can be associated to the next node. For example, for a scratch, the next image point in the reconstruction path is either to increase the depth and continue to extend, or to gradually reduce the depth, so that the sudden disconnection does not occur. Therefore, according to this rule, the possible situations can be pre-configured in the database by different nodes of the neural network, and then the image reconstruction is completed by the neural network matching mode. The difference is that the original neural network has a single node and a single convolution kernel, and the neural network of the application has a single node and a plurality of deconvolution kernels. The deconvolution kernels of the same node may be the same, but the priority is different to form the uniqueness of the node elements. Since the purpose of the application is to obtain a clear image, if the deep learning mode is used, the node exploration needs to be continuously completed, so that all nodes need to be traversed to obtain the optimal matching result. However, since the image does not exist by itself, the optimal matching result may not reflect the true result, but increase the amount of calculation. By configuring the convolution data set and the convolution priority value, the optimal matching is completed in one node through the logic of priority convolution, and the next node is directly entered, reducing the amount of calculation while maintaining the learnability of the system. Furthermore,

[0057] The step S5 includes:

[0058] Step S5-1, a plurality of reconstruction paths are generated according to the reconstruction features so that the reconstruction paths cover the reconstruction contour; the reconstruction paths can have intersections, and in addition to the paths determined by the reconstruction features, the reconstruction paths are also formed in other spare areas. The existing reconstruction paths can be used as a basis until the entire reconstruction contour is covered.

[0059] Step S5-2, determine the reconstructed image point with the smallest distance from the defective image point as the starting point of the reconstruction path, determine the starting node from the reconstruction neural network according to the reconstruction feature corresponding to the reconstruction path and determine the matching relationship; the purpose of determining the starting point is to have a part of reference when deconvolution, because the reliability of the image point at the position close to the center image area of the close-range image is higher, so the deconvolution starting point can be based on the missing image point to perform deconvolution operation when deconvolution, so that the existing image point can be used as a basis for judgment, and the corresponding image point with high reliability can be formed in the reconstruction process, for example, the edge position of the missing image in the close-range image is used as the operation basis, and is sequentially brought into the reconstruction neural network, so that the image formed is equivalent to “growth” from the edge of the close-range image, and the principle of determining the matching relationship in the reconstruction neural network is as follows: the reconstruction feature analysis is “scratch 1”, and the corresponding mark is configured in the reconstruction neural network corresponding to each starting node in advance, so that the node with the matching relationship can be found through the reconstruction feature as an index,

[0060] Step S5-3, in the matching relationship, the convolution matching value of each deconvolution kernel and the image point is calculated through the preset convolution fitting algorithm in the order of the convolution priority value, until the convolution matching value reaches the reconstruction sub-condition; the convolution fitting algorithm is: wherein A is the convolution matching value, b is a preset weight purpose parameter, is the maximum convolution value of the current image point after deconvolution calculated by the defect feature, is the upper limit matching value determined in the loss judgment strategy, U is the number of image points in the deconvolution kernel, u is the number of overlapping image points of the current image point and the previous image point in the deconvolution algorithm; c is the sequence value of the current image point in the reconstruction sequence; is the maximum convolution value of the previous image point recalculated by the defect feature; is the maximum convolution value of the previous image point obtained by the last calculation. By setting in this way, the optimal deconvolution kernel is found from the convolution kernel data set to complete the convolution operation. The maximum convolution value obtained after deconvolution calculation reflects the matching degree with the defective image, the closer to the specified upper limit matching value, the higher the deconvolution, and the closer to the real image. On the other hand, if the position is closer to the center image area, the maximum convolution value of the difference indicates that the reliability of the real image is higher, the proximity is reflected by the sequence value, and on the other hand, the higher the convolution overlap with the previous image point, the higher the reliability, so the optimal deconvolution result can be determined through the above algorithm.

[0061] Step S5-4, calculate the index value of each lower node by a preset path index algorithm, the index value is positively correlated with the module length of the progressive index vector, determine the lower node according to the index value and determine the matching relationship with the next image point in the reconstructed path, and re execute step S5-3 until the matching of all image points in the reconstructed path in the reconstructed neural network is completed; after completing the deconvolution once each time, the corresponding matching node of the next image point under the coincidence path is determined in the reconstructed neural network, and the basis for determining the matching node is to calculate the index value, and the path index algorithm is: Wherein, B is the index value corresponding to the node, 、 、 , , , , , ,

[0062] , , , , , ,

[0063] The index value is based on three logical factors, one is the module length of the index vector, which is preconfigured and can be learned according to the result, the second is the relative matching angle, which is preconfigured in each node, each node corresponding to the upper node will have a matching angle range, that is, the preferred matching relationship of the node, and the reconstructed path will also have a matching angle, for example, the distribution of 3*3 image points will produce a path angle every 45 degree image point position, and 5*5 image points will produce a path angle every 22.5 degree image point position, and the lower node corresponding to the upper node will also have a reliable matching angle, which is determined by the sample number of the deconvolution kernel preset in the node, and one of the division basis of the convolution data set is to divide the deconvolution kernel by angle, so that each node corresponding to the upper node will have a preferred angle, the difference between the trajectory angle corresponding to the index path and the matching angle corresponding to the reconstructed neural network is calculated, if the matching is higher, it means that the lower node is more suitable as the preferred node, on the other hand, the pheromone is used as the basis for determining the lower node. After obtaining the index value, the optimal lower node is not determined directly by the size of the index value, but is determined according to the random number logic.

[0062] The step S5-4 further includes a proportion generation algorithm, and the proportion generation algorithm is: , , , , ,

[0063] ,The step S5-4 further comprises configuring an index value range for each lower node according to the index ratio value, and generating a random number, and determining the corresponding lower node to establish the matching relationship according to the random number falling into the index data range.

[0064] The pheromone decay strategy is further included, and the pheromone is configured for each node in the reconstructed neural network, and when the matching relationship is established by any node, the corresponding pheromone is increased by a preset reinforcement value, and the pheromone of other nodes in the same level is reduced by a preset decay value. The reconstructed neural network has learning property through the pheromone, and the advantage node is more easily identified, so that the augmented image is generated faster.

[0065] Step S5-5, the deconvolution of all reconstructed paths is completed to generate the reconstructed feature image.

[0066] Step S6, the reconstructed feature image and the industrial product close-range image are spliced to obtain a defect augmented image; since the position of the missing part is known, the defect augmented image is completed through image superposition or splicing, and after splicing, the center of the defect image point distribution center of the entire defect augmented image is determined as the center, and part of the redundant image points are deleted to form a region.

[0067] Step S7, the obtained defect augmented image and the industrial product long-range image are compared to calculate an image similarity value, the image point matching is completed through the inflation algorithm or the corrosion algorithm, and then the corresponding color values are compared to calculate the similarity value, when the image similarity value is greater than an upper threshold value, a positive feedback adjustment strategy is executed, and when the image similarity value is less than a lower threshold value, a negative feedback adjustment strategy is executed. The positive feedback adjustment strategy is to increase the length of the corresponding progressive index vector in the reconstructed neural network by a positive adjustment value, and the positive adjustment value is proportional to the difference between the image similarity value and the upper threshold value; the negative feedback adjustment strategy is to reduce the length of the corresponding progressive index vector in the reconstructed neural network by a negative adjustment value, and the negative adjustment value is proportional to the difference between the image similarity value and the lower threshold value. In this way, the depth learning of the model can be realized by adjusting the length of the index vector, and the accuracy optimization is completed.

[0068] Of course, the above is only a typical example of the present application, in addition to this, the present application can have other various specific embodiments, and any technical solution formed by equivalent substitution or equivalent transformation falls within the scope of the present application.

Claims

1. A method for augmenting image data of industrial product defects using an industrial robot, characterized in that: Step S1: Acquire distant images of industrial products in real time, and identify defect features of the distant images of industrial products through a preset defect feature neural network. Step S2: Using the location of the defect feature in the distant image as the center, magnify the distant image of the industrial product by magnification to perform feature recognition and obtain a close-up image of the industrial product. At the same time, automatically focus and obtain the focusing parameters. Step S3: Based on the defect features, determine the loss information of the close-up image of the industrial product using a preset loss judgment strategy, and use the defect features as an index to obtain feature reconstruction conditions from the preset feature reconstruction database. Step S4: Obtain feature reconstruction information from the distant view image of industrial products using the aforementioned feature reconstruction conditions; Step S5: Input the feature reconstruction information into the preset feature augmentation model to obtain the reconstructed feature image; Step S6: Stitch the reconstructed feature image and the close-up image of the industrial product to obtain a defect augmentation image; Step S7: Compare the obtained defect augmented image with the industrial product distant view image to calculate the image similarity value. When the image similarity value is greater than the upper limit threshold, execute the positive feedback adjustment strategy. When the image similarity value is less than the lower limit threshold, execute the negative feedback adjustment strategy.

2. The method for augmenting industrial robot product defect image data as described in claim 1, characterized in that: The loss detection strategy includes: Step S3-1: Calculate the maximum convolution value of each pixel in the close-up image of the industrial product based on the defect features; Step S3-2: Determine the upper limit matching value and the lower limit matching value based on the distribution of the maximum convolution value; Step S3-3: Mark the image points whose maximum convolution value is higher than the upper limit matching value as defect image points, mark the image points whose maximum convolution value is between the upper limit matching value and the lower limit matching value as reconstructed image points, mark the image points whose maximum convolution value is lower than the lower limit matching value as redundant image points, and divide the defect image points whose distribution mode meets the center distribution condition into the center map area. Step S3-4: Determine the reconstructed image points whose distance from the center image area is less than the preset redundancy distance value as the adjacent dataset. The reconstructed image points in the adjacent dataset whose distribution pattern meets the loss distribution conditions are divided into loss image areas. The image points in the adjacent dataset located at the edge of the close-up image of industrial products are determined as loss edges. Step S3-5: Generate the loss information based on the location of the loss map area and the loss edge.

3. The method for augmenting industrial robot product defect image data as described in claim 2, characterized in that: Step S4 includes: Step S4-1: Determine the starting coordinates in the distant view image of the industrial product using the loss information; Step S4-2: Determine the reconstructed contour from the distant view image of industrial products using the contour recognition benchmark in the feature reconstruction conditions; Step S4-3: Determine several reconstruction features in the reconstruction contour using the reconstruction benchmark in the feature reconstruction conditions, and generate the feature reconstruction information based on the reconstruction features.

4. The method for augmenting industrial robot product defect image data as described in claim 3, characterized in that: The feature augmentation model includes a reconstructed neural network, in which each node corresponds to a convolutional kernel dataset, the convolutional kernel dataset includes several deconvolutional kernels, each deconvolutional kernel corresponds to a convolution priority value, and the nodes of the reconstructed neural network are associated with each other through a progressive index vector.

5. The method for augmenting industrial robot product defect image data as described in claim 4, characterized in that: Step S5 includes: Step S5-1: Generate several reconstruction paths based on the reconstruction features so that the reconstruction paths cover the reconstruction outline; Step S5-2: Determine the reconstructed image point with the smallest distance to the defect image point as the starting point of the reconstruction path, and determine the starting node and matching relationship from the reconstruction neural network according to the reconstruction features corresponding to the reconstruction path; Step S5-3: In the matching relationship, the convolution matching value between each deconvolution kernel and the image point is calculated in order of convolution priority value through a preset convolution fitting algorithm until the convolution matching value reaches the reconstruction sub-condition. Step S5-4: Calculate the index value of each lower-level node using a preset path indexing algorithm. The index value is positively correlated with the magnitude of the progressive index vector. Determine the lower-level node based on the index value and establish a matching relationship with the next image point in the reconstruction path. Then, repeat step S5-3 until the matching of all image points in the reconstruction path is completed in the reconstruction neural network. Step S5-5: Perform deconvolution on all reconstruction paths to generate the reconstructed feature image.

6. The method for augmenting industrial robot product defect image data as described in claim 5, characterized in that: The convolution fitting algorithm is as follows: Where A is the convolution matching value, and b is the preset weight target parameter. This represents the maximum convolution value calculated by deconvolving the current image points using defect features. U is the upper limit matching value determined in the loss judgment strategy, U is the number of images in the deconvolution kernel, u is the number of overlapping images between the current image and the previous image in the deconvolution algorithm; c is the sequence value of the current image in the reconstructed sequence; This is the maximum convolution value of the previous image point, recalculated using the defect features; This is the maximum convolution value of the previous pixel obtained from the last calculation.

7. The method for augmenting industrial robot product defect image data as described in claim 5, characterized in that: The path indexing algorithm is as follows: ， Where B is the index value corresponding to this node. , , These are the preset index weight parameters, respectively. , The magnitude of the index vector. The angle of the index path trajectory at the current image point. To reconstruct the matching angle of the feature dataset in the neural network, z represents the pheromone of the corresponding node.

8. The method for augmenting industrial robot product defect image data as described in claim 6, characterized in that: Step S5-4 further includes a ratio generation algorithm, which is as follows: ,in, This represents the index ratio, where m is the number of child nodes. For the nth subordinate node, It is the m-th child node; Step S5-4 further includes configuring the index value range for each lower-level node according to the index ratio value, generating a random number, and determining the corresponding lower-level node to establish the matching relationship based on the random number falling within the index data range.

9. The method for augmenting industrial robot product defect image data as described in claim 7, characterized in that: It also includes a pheromone attenuation strategy, which configures pheromones for each node in the reconstructed neural network. When any node establishes a matching relationship, its corresponding pheromone is increased by a preset reinforcement value, and the pheromones of other nodes at the same level are decreased by a preset attenuation value.

10. The method for augmenting industrial product defect image data using an industrial robot as described in claim 5, characterized in that: The positive feedback adjustment strategy involves increasing the magnitude of the corresponding progressive index vector in the reconstructed neural network by a positive adjustment value, which is proportional to the difference between the image similarity value and the upper threshold. The negative feedback adjustment strategy involves decreasing the magnitude of the corresponding progressive index vector in the reconstructed neural network by a negative adjustment value, which is proportional to the difference between the image similarity value and the lower threshold.

Images