A vision-guided method and system for robotic arms used in industrial assembly.

Abstract

This invention relates to the field of image data processing technology, and more particularly to a visual guidance method and system for a robotic arm used in industrial assembly. The method includes the following steps: acquiring image sequences of multiple independent light sources illuminating a workpiece to be matched from different directions; constructing grayscale response curves for each pixel, and calculating specular reflection purity and peak dominance factor based on these curves, then fusing them into a specular confidence score insensitive to highlights; simultaneously obtaining the optimal illumination index representing the geometric orientation; and matching the specular confidence score, the optimal illumination index, and a preset template to obtain the target pose of the workpiece to guide the robotic arm. This invention, by constructing stable geometric reflection features, can overcome specular reflection interference from highly reflective workpieces such as metals, achieving highly robust visual positioning and guidance.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a vision guidance method and system for robotic arms used in industrial assembly. Background Technology

[0002] In the automated assembly of high-end electrical equipment such as vacuum circuit breakers, robotic arms need to accurately position and grasp metal workpieces such as flanges and contacts. In actual production, the position and orientation of workpieces often exhibit unavoidable random deviations during conveying or loading.

[0003] Traditional solutions rely on pre-set fixed motion trajectories or rigid tooling fixtures, lacking the ability to perceive the actual position and posture of the workpiece, making it difficult to adapt to changes in incoming materials and failing to balance high precision and production flexibility. Therefore, machine vision-based guidance technology has become a key means to achieve automated assembly. Among these, template matching techniques based on two-dimensional grayscale images, such as the normalized cross-correlation algorithm, are applied to the visual positioning of workpieces due to their mature algorithms and computational efficiency. This method determines the spatial position and orientation of the target workpiece by searching for the region in the image acquired by the camera that is most similar to a pre-set template in pixel grayscale distribution.

[0004] However, this method has limitations when processing metal workpieces. Due to the strong specular reflection properties of metal surfaces, under industrial directional lighting, high-brightness areas with unstable positions and shapes are easily generated, causing drastic changes in the image grayscale distribution. This light sensitivity disrupts the constant grayscale premise upon which template matching depends, significantly reducing the correlation between the template and the real-time image, resulting in positioning deviations or even matching failures. Especially when there are slight fluctuations in the incoming material's posture or changes in ambient lighting, the system's positioning accuracy and stability decrease, leading to frequent mispositioning, production line downtime, and other malfunctions, affecting the reliability and operational efficiency of automated assembly processes. Summary of the Invention

[0005] To address the technical problem that the aforementioned visual guidance method based on two-dimensional grayscale template matching cannot overcome the high-gloss interference and drastic grayscale changes caused by specular reflection on the surface of metal workpieces, leading to positioning failures and making it difficult to meet the requirements of high-precision assembly, this invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a vision-guided method for a robotic arm in industrial assembly, the method comprising the steps of:

[0007] Images of the workpiece to be matched are acquired by sequentially illuminating it from multiple independent light sources from different directions. These images are then arranged in the order of acquisition to obtain an image sequence of the workpiece. For each pixel in the image sequence, its grayscale value in each frame is used to construct a grayscale response curve. Based on the grayscale response curve, the specular purity and peak dominance factor of each pixel are calculated. The specular purity characterizes the peak sharpness of the grayscale response curve, and the peak dominance factor characterizes the proportion of peak energy in the sum of all non-peak energy. The images of the pixels are then fused. The specular reflection purity and peak dominance factor of a point and its neighborhood are used to obtain the specular confidence of each pixel. Each pixel in the image is traversed, and the light source index with the largest gray value is obtained, which is used as the optimal illumination index of that pixel. The optimal illumination index and specular confidence of a preset workpiece template are obtained. For each pose in the image of the workpiece to be matched, the single-point matching contribution value of all pixels that meet the preset matching conditions is accumulated to obtain the matching score of the pose. The pose that gives the highest matching score is obtained and recorded as the target pose. The robotic arm is then guided to perform assembly actions based on the target pose.

[0008] This invention first acquires multi-source image sequences and constructs a grayscale response curve for each pixel. Second, it extracts two stable features from the grayscale response curves: specular reflection purity, representing peak sharpness, and peak dominance factor, representing peak energy proportion. Then, by fusing these two features and combining them with spatial neighborhood information, a specular confidence score is generated. Simultaneously, by finding the light source with the largest grayscale value for each pixel, the optimal illumination index representing the geometric orientation is obtained. Finally, in the matching stage, feature points with consistent optimal illumination indices are weighted and accumulated for specular confidence scores. This strategy, based on geometric index matching and signal quality weighting, overcomes the specular interference problem of traditional grayscale matching under specular reflection of metal workpieces, enabling stable and high-precision workpiece positioning and visual guidance from images filled with variable specular highlights.

[0009] Preferably, obtaining the specular reflection purity of each pixel includes: obtaining the maximum gray value, the second maximum gray value, and the minimum gray value of the pixel in the image sequence; recording the difference between the maximum gray value and the second maximum gray value as a first difference; recording the difference between the maximum gray value and the minimum gray value as a second difference; and the specular reflection purity is the ratio of the first difference to the second difference.

[0010] This invention calculates the purity of specular reflection by using the difference between the maximum and second-maximum grayscale values ​​as the numerator and the difference between the maximum and minimum grayscale values ​​as the denominator. The numerator effectively measures the sharpness of the peak of the grayscale response curve, while the denominator represents the complete grayscale dynamic range of the pixel. Dividing the two can accurately distinguish between real specular reflection points with prominent peaks and strong directionality and diffuse reflection points with gentle curves and dispersed responses.

[0011] Preferably, the calculation of the peak dominance factor includes: calculating the square of the maximum gray value of the gray-scale response curve, denoted as the peak energy; calculating the sum of the squares of all gray values ​​in the gray-scale response curve, denoted as the total signal energy; subtracting the peak energy from the total signal energy to obtain the sum of all non-peak energies; the peak dominance factor is the ratio of the peak energy to the sum of all non-peak energies.

[0012] This invention calculates the peak dominance factor by calculating the ratio of the square of the maximum gray value to the sum of all non-peak energies. This non-linearly amplifies the difference between signal and noise, solving the technical problem that the purity of specular reflection alone cannot distinguish between real strong reflection peaks and weak noise pseudo-peaks.

[0013] Preferably, the step of fusing the specular purity and peak dominance factor of each pixel and its neighborhood to obtain the specular confidence of each pixel includes: multiplying the specular purity of the pixel by the peak dominance factor, and recording it as a single-point quality assessment; calculating the average value of the single-point quality assessments of each pixel within the neighborhood window of the pixel, and recording it as a spatial consistency assessment; the specular confidence is the product of the single-point quality assessment and the spatial consistency assessment of the pixel.

[0014] This invention addresses the limitations of single-point evaluation by introducing the principle of spatial consistency. This fusion strategy ensures that only pixels with high quality and generally high quality in their neighborhoods can obtain high confidence, effectively filtering out isolated noise points that lack spatial continuity.

[0015] Preferably, the step of accumulating the results of all pixels that meet the preset matching conditions to obtain the pose matching score includes: determining the consistency between the optimal illumination index of the pixel and the optimal illumination index of the preset template; when consistent, calculating the product of the specular confidence of the pixel and the specular confidence of the corresponding position in the preset template, and recording it as the single-point matching contribution value; when inconsistent, recording the single-point matching contribution value as zero; accumulating the single-point matching contribution values ​​of each pixel in the image of the workpiece to be matched to obtain the total matching contribution value, and normalizing it to obtain the matching score.

[0016] This invention achieves matching of workpiece geometric features by determining whether the optimal illumination index is consistent. Under the premise of successful geometric matching, the specular confidence of the pixel to be matched is further multiplied with the specular confidence of the preset workpiece template, and this is used as the single-point matching contribution value. This makes feature points with high signal quality and strong credibility have higher weights on the matching results, while the influence of feature points with poor signal quality is also suppressed.

[0017] Preferably, the normalization to obtain the matching score includes: taking the total matching contribution as the numerator; for the current pose of the workpiece to be matched, summing the squares of the mirror confidence scores of all pixels in the preset workpiece template corresponding to the pose to obtain the denominator; and recording the ratio of the numerator to the denominator as the matching score.

[0018] Preferably, the consistency between the optimal illumination index of the pixel and the optimal illumination index of the preset template is determined by the Kronecker delta function.

[0019] Preferably, the plurality of independent light sources includes four light sources, which are respectively arranged at the upper, lower, left and right sides of the camera field of view corresponding to the image of the workpiece to be matched.

[0020] Preferably, the guided robotic arm performs assembly actions based on the target pose, including: converting the target pose from the camera coordinate system to the robotic arm coordinate system; controlling the robotic arm to move to the target pose in the robotic arm coordinate system and performing grasping or placing operations.

[0021] In a second aspect, the present invention provides a robotic arm vision guidance system for industrial assembly, the robotic arm vision guidance system for industrial assembly including a memory and a processor, the memory storing computer program instructions, which, when executed by the processor, implement a robotic arm vision guidance method for industrial assembly according to the first aspect of the present invention.

[0022] By adopting the above technical solution, a computer program for a robotic arm vision guidance method for industrial assembly according to the first aspect of the present invention is generated and stored in a memory so that it can be loaded and executed by a processor, thereby creating a terminal device based on the memory and the processor for convenient use.

[0023] The beneficial effects of this invention are as follows: This invention acquires image sequences using independent light sources from multiple directions and constructs a grayscale response curve for each pixel. Instead of relying on grayscale values, this invention extracts two stable features: an optimal illumination index representing geometric orientation and a specular confidence score representing signal quality. The specular confidence score is a highly reliable indicator that integrates specular reflection purity, peak dominance factor, and spatial consistency. During the matching stage, this invention uses the optimal illumination index for geometric alignment and the specular confidence score for weighted scoring. This effectively and stably locates the workpiece target pose with high precision from images filled with unstable specular interference, solving the positioning failure problem caused by specular interference, providing reliable visual guidance for robotic arms, and meeting the requirements of high-precision assembly. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a vision-guided robotic arm method for industrial assembly, provided as an embodiment of the present invention;

[0025] Figure 2 This is a structural block diagram of a robotic arm vision guidance system for industrial assembly, provided as an embodiment of the present invention. Detailed Implementation

[0026] The first aspect of this invention provides a vision-guided method for a robotic arm used in industrial assembly, such as... Figure 1 As shown, the method includes steps S100-S400:

[0027] Step S100: Collect images of the workpiece to be matched from multiple independent light sources illuminating it individually from different directions, and arrange them in the order of acquisition to obtain an image sequence of the workpiece to be matched.

[0028] It should be noted that, in order to overcome the limitation of a single image having only one dimension of information, this step controls multiple independent light sources to illuminate the workpiece to be matched from different directions and simultaneously acquires images in order to construct a multidimensional dataset containing rich surface reflection information.

[0029] Specifically, an industrial camera is fixed at the end of the robotic arm or above the assembly line, ensuring that the camera's field of view completely covers the workpiece to be gripped and matched. A set of independent light sources is arranged around the workpiece. In one feasible implementation, taking into account the specular reflection characteristics of the metallic workpiece, a preferred method is to use... Each LED light source is positioned on a ring-shaped bracket directly above the camera's field of view, corresponding to the top, bottom, left, and right sides of the bracket; these are denoted as light sources. ,in This quantity and layout can cover the main reflection directions of the workpiece surface to be matched, while taking into account both acquisition efficiency and data integrity.

[0030] When performing a vision-guided positioning task, each individual light source is illuminated sequentially and individually by the light source controller. Simultaneously, ensure that other light sources are off, and trigger the industrial camera to capture an image of the workpiece to be matched the instant the independent light source is turned on. Repeat this process. Next, a set containing Image sequence of the workpiece to be matched Furthermore, it should be noted that each individual light source is illuminated sequentially, and one image frame is captured. Therefore, the total number of image frames is equal to the total number of individual light sources. This will be used consistently throughout the following text. This indicates the quantity.

[0031] At this point, the image sequence of the workpiece to be matched has been obtained.

[0032] Step S200: For each pixel in the image sequence, construct a grayscale response curve for its grayscale value in each frame image; based on the grayscale response curve, calculate the specular purity and peak dominance factor of each pixel.

[0033] It should be noted that for any pixel in an image sequence, the change in its grayscale value under different illuminations constitutes a pixel photometric response curve, which is the trajectory of grayscale value change as the light source direction switches. The characteristic differences of this curve are key to distinguishing between diffuse reflection and specular reflection: the grayscale value of diffuse reflection pixels mainly reflects the surface texture of the workpiece to be matched, is relatively stable under different illumination directions, and the pixel photometric response curve is flat without obvious peaks, while lacking precise information related to the geometric structure of the workpiece to be matched; on the other hand, the response mode of specular reflection pixels has a one-to-one quantitative relationship with the orientation of the geometric normal of the workpiece surface, containing high-precision geometric positioning information, and its pixel photometric response curve has obvious directional uniqueness, producing strong reflection only for specific light sources, and significantly weakening the response to other light sources. Based on this characteristic, this step first extracts the pixel photometric response curve of each pixel from the image sequence, and then distinguishes between the two types of reflection by evaluating the peak characteristics of the curves, finally selecting pixels with a high proportion of specular reflection.

[0034] Specifically, firstly, the pixel photometric response curve of each pixel is extracted, including: firstly determining the correspondence between pixel position and illumination direction. Since each frame of the image is individually acquired by a light source from a specific direction, an image frame-light source direction mapping table is first established. For example, that is , , , The top, bottom, left, and right light sources are sequentially matched to ensure that the mapping relationship is completely consistent with the order of light source arrangement, avoiding orientation matching errors. Simultaneously, the industrial camera position is fixed, and the pixel coordinates of each frame can be directly aligned, ensuring that subsequent reads are of pixels at the same physical location. Then, the grayscale values ​​of the same pixel are extracted and integrated for any pixel in the image sequence. Read its in sequence The grayscale value in the image is obtained by comparing it with the grayscale value in the image. Each light source direction corresponds to Each grayscale data point; finally, with the light source direction as the horizontal axis and the grayscale value as the vertical axis, [the data is processed]. Plot the grayscale data corresponding to the light source direction, connect the points in order of light source direction, and obtain the pixel. The pixel luminous response curve.

[0035] Then, the degree to which a pixel is dominated by specular reflection is evaluated by the ratio of the peak sharpness to the grayscale dynamic range of the pixel's photometric response curve. For any pixel in the image... Its specular reflection purity satisfies the following relationship:

[0036] ;

[0037] in, It is a pixel. Specular reflection purity; , They are pixels The maximum and second-maximum gray values ​​in the image sequence; It is a pixel. The minimum gray value in an image sequence; It is a preset first tiny value used to prevent the denominator from being zero, and can be set to 0.001.

[0038] In this relation, It is used to measure the difference between the peak and the second-highest peak, reflecting the pixel. The sharpness of the peak of the pixel's photometric response curve; if a pixel is predominantly specular, producing strong reflections only from specific light sources, Significantly greater than At this point, the molecule value is relatively large, indicating that the curve peak is prominent; if the pixel is dominated by diffuse reflection, the difference in grayscale values ​​under different light sources is small. When the numerator value approaches 0, it indicates that the curve has no obvious peak. The denominator... Representing pixels The complete grayscale dynamic range under all illuminations serves to normalize molecules. When pixels... When specular reflection is dominant, Significantly greater than ,at this time A value close to 1 indicates that its reflective properties are dominated by specular reflection; conversely, if it is a diffuse reflection point, its grayscale values ​​under multiple light sources will be similar. and The difference is very small, at this time Approaching 0.

[0039] It should be noted that while specular reflection purity can characterize the sharpness of the peak in a pixel response curve, in scenarios where the noise signal intensity in the image is close to or exceeds the effective signal intensity, this indicator may not be able to effectively distinguish between strong peaks with concentrated energy caused by genuine specular reflection and spurious peaks with occasional amplitude jumps caused by weak stray light. Although spurious peaks have amplitude jumps, their energy does not form a significant advantage and is fundamentally different from the high energy concentration of genuine peaks. Therefore, this step evaluates the dominance of reflection peak energy in the total signal energy from the perspective of signal energy, thereby effectively identifying high-quality genuine reflection signals.

[0040] Specifically, for any pixel in the image Based on its Gray values ​​and maximum gray values ​​in a frame image Calculate the peak dominance factor, which satisfies the following relationship:

[0041] ;

[0042] in, It is a pixel. The peak dominant factor; It is a pixel. exist The maximum grayscale value in the frame image; It is a pixel. In the Gray values ​​in a frame image; It is the total number of image frames; It is a preset second tiny value used to prevent the denominator from being zero, and can be set to 0.001.

[0043] In this relationship, the molecule Representing pixels The energy of the peak in the response sequence; the energy in the denominator This represents the total signal energy, and the difference between it and the peak energy represents the pixel energy. The total energy of all non-peak signals; This calculation measures the ratio of peak energy to the total energy of all non-peak signals. Since grayscale values ​​in industrial images approximate light intensity, and signal energy is proportional to the square of light intensity, this metric non-linearly amplifies the difference between peak and background noise. In true, high-quality specular reflection, the peak energy will be greater than the sum of the non-peak energies, leading to… The peak energy is relatively high; however, the peak energy of spurious peaks caused by noise is close to the sum of the peak and non-peak energies, leading to... Approaching 0.

[0044] Thus, the specular purity and peak dominance factor of each pixel in the image sequence were obtained.

[0045] Step S300: Fuse the specular reflection purity and peak dominance factor of each pixel and its neighborhood to obtain the specular confidence of each pixel; traverse each pixel in the image and obtain the light source index with the largest gray value, and use it as the best illumination index of the pixel.

[0046] It should be noted that specular purity and peak dominance factor were used to evaluate the signal from two dimensions: peak sharpness and energy dominance, respectively. However, in the physical world, true specular reflection should not only be of high quality at a single point but also exhibit spatial continuity. The surface geometry of the metal workpiece to be matched has continuity, and its true specular reflection area must be spatially distributed in patches, while isolated noise or defect points do not possess this characteristic. The principle of spatial consistency aligns with this physical law, accurately distinguishing continuous true highlight areas from isolated interference points, effectively compensating for the limitations of single-point indicators. Based on this, this step utilizes the generally high specular purity and peak dominance factor of the center pixel and neighboring pixels in continuous true highlight areas to improve the overall confidence of the region through multi-dimensional feature fusion, making the true effective area more prominent.

[0047] Based on the above logic, any pixel in the image The mirror confidence level satisfies the following relationship:

[0048] ;

[0049] in, It is a pixel. Mirror confidence; It is a pixel. Specular reflection purity; It is a pixel. The peak dominant factor; It is a pixel. The set of pixels within the neighborhood window, neighborhood window Can be set to or The dimensions are the same, but the specific size is adjusted according to the surface detail and precision requirements of the workpiece to be matched. It is a pixel. The total number of pixels within the neighborhood window; It is a pixel. The pixels within the neighborhood window; It is a pixel. Pixels within the neighborhood window Specular reflection purity; It is a pixel. Pixels within the neighborhood window The peak dominant factor.

[0050] In this relation, Pixel The single-point quality assessment ensures that only pixels with both high specular purity and high peak dominance factor can obtain a high single-point base score. Used to calculate pixels The average quality of each pixel within the neighborhood window is the average value of the single point. Since the real specular reflection area is continuous in space, the reflection characteristics of its neighboring pixels should be consistent with the center pixel. Therefore, this average value is an evaluation of spatial consistency, which can ensure that the surrounding area of ​​high-scoring pixels also has reliable reflection characteristics, and avoid isolated noise points or defect points being misjudged as valid features. Only pixels with high individual quality and equally excellent overall neighborhood quality can obtain high confidence scores. This can accurately filter out physically continuous real specular reflection areas, while reducing interference from isolated noise points or local defects, providing a highly reliable set of pixels for subsequent feature extraction.

[0051] Based on the specular confidence level, the optimal lighting index for each pixel is further obtained. It should be noted that, according to the law of reflection, specular reflection only produces highlights when the light source direction, viewing direction, and surface normal satisfy a specific geometric relationship. Therefore, the light source that makes the pixel brightest corresponds to the geometric orientation of the pixel's surface normal.

[0052] Based on this characteristic, obtaining the optimal illumination index for each pixel includes: traversing each pixel in the image, obtaining the light source index with the largest grayscale value, and using it as the optimal illumination index for that pixel. .

[0053] At this point, the specular confidence and optimal lighting index of each pixel have been obtained.

[0054] Step S400: Obtain the optimal illumination index and specular confidence of the preset workpiece template; for each pose in the image of the workpiece to be matched, accumulate the single-point matching contribution value of all pixels that meet the preset matching conditions to obtain the matching score of the pose, obtain the pose that makes the matching score the highest, record it as the target pose, and guide the robotic arm to perform assembly actions based on the target pose.

[0055] It should be noted that, in order to improve the robustness of matching, this step further integrates the geometric consistency judgment with the mirror confidence for weighted matching, so as to ensure that high-quality feature points dominate the matching. Based on this, the present invention weights and integrates the geometric consistency of the optimal illumination index with the signal confidence of the mirror confidence, so that the effective features with high confidence have a higher weight in the matching, and the influence of the interference features with low confidence is weakened, ultimately achieving highly robust matching and localization.

[0056] Specifically, first, a standard reference workpiece template is obtained, and the specular confidence and optimal lighting index of all its pixels are generated; then, by sliding the reference workpiece template, different translation positions are covered. and rotation angle Calculate the confidence-weighted optimal illumination score for each translation position. The weighted matching score satisfies the following relationship:

[0057] ;

[0058] in, It is the workpiece template in position. Weighted matching score at the time; These are the coordinates of any pixel in the image of the workpiece to be matched; It is the set of coordinates of all pixels in the image of the workpiece to be matched; It is a pixel. Mirror confidence; It is through rotation The subsequent reference workpiece template, at the pixel point Search coordinates at location Mirror confidence level at the location; It is a pixel. Optimal lighting index; It is through rotation The subsequent reference workpiece template, at the pixel point Search coordinates at location Optimal lighting index for the location; It is the Kronecker delta function, when And when none of them are zero, ,otherwise ; It is a preset third micro value used to prevent the denominator from being 0, and can be set to 0.001.

[0059] In this relationship, the numerator is the reference workpiece template in pose. Below, the weighted sum of the contributions of all high-quality and geometrically matched points, in the molecule As a criterion for geometric matching, the pixel is judged. and Are the optimal illumination indices consistent and non-zero? In molecules... As a confidence weight, this confidence weight is only included in the matching score when the geometric matching condition is met. In the denominator... This represents the total confidence energy of the reference workpiece template itself after rotation. It is used to eliminate the influence of template size or its overall confidence on the matching score, ensuring that the scores between different templates are comparable.

[0060] Finally, in all possible poses Search on it to find The pose that reaches the global maximum value is the accurate position and orientation of the workpiece to be matched in the camera coordinate system. Through the preset camera and robot arm calibration parameters, this coordinate is converted into coordinates in the robot arm coordinate system, and the robot arm is guided to perform subsequent gripping or assembly actions.

[0061] The second aspect of this embodiment provides a vision guidance system for robotic arms used in industrial assembly, such as... Figure 2 As shown, the robotic arm vision guidance system for industrial assembly includes a memory and a processor. The memory stores computer program instructions, which, when executed by the processor, implement a robotic arm vision guidance method for industrial assembly according to the first aspect of the present invention.

[0062] The vision guidance system for robotic arms used in industrial assembly also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the setup and functions of which are known in the art and will not be described in detail here.

[0063] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (DRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (DRAM), high-bandwidth memory, hybrid memory cube, etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device.

[0064] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A vision-guided method for a robotic arm used in industrial assembly, characterized in that, Including the following steps: Images of the workpiece to be matched are acquired by acquiring multiple independent light sources that illuminate the workpiece from different directions in sequence, and then arranged in the order of acquisition to obtain an image sequence of the workpiece to be matched; For each pixel in the image sequence, its grayscale value in each frame image is used to construct a grayscale response curve; based on the grayscale response curve, the specular purity and peak dominance factor of each pixel are calculated, wherein the specular purity characterizes the peak sharpness of the grayscale response curve, and the peak dominance factor characterizes the proportion of the peak energy of the grayscale response curve in the sum of all non-peak energies. By fusing the specular reflection purity and peak dominance factor of each pixel and its neighborhood, the specular confidence of each pixel is obtained; by traversing each pixel in the image, the light source index with the largest gray value is obtained and used as the best illumination index of the pixel. Obtain the optimal illumination index and specular confidence of the preset workpiece template; for each pose in the image of the workpiece to be matched, accumulate the single-point matching contribution value of all pixels that meet the preset matching conditions to obtain the matching score of the pose, obtain the pose that makes the matching score the highest, record it as the target pose, and guide the robotic arm to perform assembly actions based on the target pose.

2. The vision-guided robotic arm method for industrial assembly according to claim 1, characterized in that, The acquisition of the specular reflection purity of each pixel includes: Obtain the maximum, second-maximum, and minimum gray values ​​of a pixel in an image sequence; The difference between the maximum grayscale value and the second maximum grayscale value is denoted as the first difference. The difference between the maximum grayscale value and the minimum grayscale value is denoted as the second difference. The purity of the mirror reflection is the ratio of the first difference to the second difference.

3. The vision-guided robotic arm method for industrial assembly according to claim 1, characterized in that, The calculation of the peak dominance factor includes: Calculate the square of the maximum gray value of the gray-scale response curve, and denote it as the peak energy; Calculate the sum of squares of all gray values ​​in the gray-scale response curve, and denote it as the total signal energy; Subtract the peak energy from the total energy of the signal to obtain the sum of all non-peak energies; The peak dominance factor is the ratio of peak energy to the sum of all non-peak energies.

4. The vision-guided robotic arm method for industrial assembly according to claim 1, characterized in that, The specular confidence of each pixel is obtained by fusing the specular purity and peak dominance factor of each pixel and its neighborhood, including: The product of the specular reflection purity of the pixel and the peak dominance factor is denoted as the single-point quality assessment. Calculate the average value of the single-point quality assessment of each pixel within the neighborhood window of the pixel, and denot it as the spatial consistency assessment; The mirror confidence level is the product of the single-point quality assessment and the spatial consistency assessment of a pixel.

5. The vision-guided robotic arm method for industrial assembly according to claim 1, characterized in that, The result of summing all pixels that meet the preset matching conditions is used to obtain the matching score of the pose, including: Determine the consistency between the optimal illumination index of the pixel and the optimal illumination index of the preset template; When they match, the product of the mirror confidence of the pixel and the mirror confidence of the corresponding position in the preset template is calculated and recorded as the single-point matching contribution value; when they do not match, the single-point matching contribution value is recorded as zero. The single-point matching contribution value of each pixel in the image of the workpiece to be matched is accumulated to obtain the total matching contribution value, which is then normalized to obtain the matching score.

6. The vision-guided robotic arm method for industrial assembly according to claim 5, characterized in that, The normalization process to obtain the matching score includes: Use the total value of the matching contribution as the numerator; For the current pose of the workpiece to be matched, sum the squares of the specular confidence scores of all pixels in the preset workpiece template corresponding to the pose to obtain the denominator; The ratio of the numerator to the denominator is recorded as the matching score.

7. The vision-guided robotic arm method for industrial assembly according to claim 5, characterized in that, The consistency between the optimal illumination index of the pixel and the optimal illumination index of the preset template is determined by the Kronecker delta function.

8. The vision-guided robotic arm method for industrial assembly according to claim 1, characterized in that, The multiple independent light sources include four light sources, which are respectively positioned at the top, bottom, left, and right sides of the camera's field of view when acquiring images of the workpiece to be matched.

9. The vision-guided robotic arm method for industrial assembly according to claim 1, characterized in that, The guided robotic arm performs assembly actions based on the target pose, including: Transform the target pose from the camera coordinate system to the robotic arm coordinate system; Control the robotic arm to move to the target pose in the robotic arm coordinate system and perform grasping or placing operations.

10. A vision guidance system for a robotic arm used in industrial assembly, characterized in that, The robotic arm vision guidance system for industrial assembly includes a processor and a memory, the memory storing computer program instructions, which, when executed by the processor, implement a robotic arm vision guidance method for industrial assembly according to any one of claims 1-9.

Images