Monocrystalline silicon robust grabbing detection method

Abstract

The invention discloses a robust grabbing detection method for monocrystalline silicon. The robust grabbing detection method comprises the following steps: S1, acquiring an RGB-D image containing a monocrystalline silicon target to be grabbed; s2, performing weighted fusion on the RGB image and the depth image to generate a fused image; s3, inputting the RGB-D image into a pre-trained end-to-end grabbing detection network, and outputting an initial grabbing posture and an initial quality image; s4, performing double-layer geometric structure feature extraction based on the fused image; s5, executing a grabbing process, and judging whether an improvement condition is met or not; s6, if the improvement condition is met, a final improved grabbing posture is output; the method has the advantages that visual noise interference caused by high light reflection of the material surface is effectively overcome, and edge grabbing deviation of the deep learning model is forcibly corrected by using geometric center information, so that grabbing precision, stability and safety of automatic sorting of monocrystalline silicon are remarkably improved.

Description

A robust gripping detection method for single-crystal silicon Technical Field

[0001] This invention relates to the field of vision control technology, and in particular to a robust grasping and detection method for single-crystal silicon. Background Technology

[0002] In the production and processing of monocrystalline silicon, automated sorting and loading / unloading are crucial for improving production efficiency. Monocrystalline silicon materials (such as monocrystalline silicon rods, square rods, and edge materials) typically have regular geometric shapes. However, the smooth surface and high reflectivity of monocrystalline silicon can easily lead to missing depth data or noise in depth data acquired by depth cameras (RGB-D). Existing deep learning grasping and detection networks (such as GG-CNN) tend to learn local features. With limited training data, the network is prone to misidentifying the highly reflective edges of monocrystalline silicon as optimal grasping points. However, monocrystalline silicon is brittle and hard, and edge grasping can easily lead to edge chipping or slippage. The ideal grasping position should be located on the geometric center axis of the object. Existing end-to-end deep learning grasping and detection networks (such as GG-CNN) tend to learn local features. With limited training data, the network is prone to misidentifying the highly reflective edges of monocrystalline silicon as optimal grasping points. However, monocrystalline silicon is brittle and hard, and edge gripping can easily lead to edge chipping or slippage. The ideal gripping position should be located on the geometric center axis of the object. Although traditional image processing can find the geometric center, it is not robust in complex backgrounds or occlusion situations. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a robust grasping detection method for monocrystalline silicon. By introducing a two-layer geometric feature extraction and verification mechanism, the method uses the geometric skeleton information of monocrystalline silicon itself to correct the prediction results of the deep learning network, and forces the grasping point to return to the geometric center of the object, thereby improving the grasping success rate.

[0004] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0005] A robust grasping and detection method for monocrystalline silicon includes the following steps: S1, acquiring an RGB-D image containing the monocrystalline silicon target to be grasped, wherein the RGB-D image is composed of an RGB image and a depth image; S2, performing weighted fusion on the RGB image and the depth image to generate a fused image containing monocrystalline silicon texture and depth information; S3, inputting the RGB-D image into a pre-trained end-to-end grasping detection network and outputting an initial grasping posture. and initial quality image ,in To capture the center coordinates, In order to capture the angle, S4. Based on the fused image, perform dual-layer geometric structure feature extraction, and extract the first layer skeleton image respectively. First layer candidate images and the second-layer skeleton image Second layer candidate images S5. Execute the capture process, comparing the extracted candidate images with the initial quality image. Perform calculations to determine if the improvement conditions are met; S6. If the improvement conditions are met, use the selected skeleton image to improve the initial quality image. Weighted corrections are performed to obtain an improved quality image. And output the final improved grasping posture. ,in, To improve the crawling quality score, the original network output is: This represents the probability or confidence level of a successful capture. It combines geometric structural features (such as skeleton extraction and central axis information) to the original score The new score, after correction and improvement, is calculated by fusing the original quality image with the geometric feature image. It more robustly reflects the feasibility of the grasp, especially by focusing high-scoring regions on the geometric central axis of the object.

[0006] Specifically, in step S2, a fused image is generated. The specific formula is:

[0007] ;

[0008] in, To convert an RGB image into a single-channel grayscale image, A depth image normalized to 0-256 grayscale levels; and These are the fusion ratio coefficients, where , .

[0009] Specifically, in step S4, the first layer skeleton image is extracted. and the first layer candidate image The process includes: using edge detection methods to extract edges from the fused images. Obtain the profile of single-crystal silicon and set a low threshold. The high threshold is twice the low threshold; a filling operation is performed inside the effective edge of the closed loop. Obtain the first mask image :

[0010] ;

[0011] right Perform distance transformation Obtain distance image The first layer of candidate images is obtained by setting a distance threshold to extract foreground pixels. ,right The first layer skeleton image is obtained by refining. The distance threshold is a distance image. The maximum value multiplied by the scaling factor ,in .

[0012] Specifically, in step S4, the second-layer skeleton image is extracted. Second layer candidate images The process includes: based on the first layer of candidate images Calculate the adaptive threshold The formula is:

[0013] ;

[0014] in and Candidate images The maximum and minimum values ​​in the image; based on the first mask image. Perform threshold determination; if the pixel grayscale value is greater than... If the background image is not retained, it is retained; otherwise, it is suppressed and used as background to generate a second mask image. ;right Distance transformation and skeleton extraction are performed to obtain the second-layer candidate image. Second layer skeleton image The distance threshold scaling factor used at this time .

[0015] Specifically, the improved crawling process described in step S5 is as follows:

[0016] First-layer candidate images are preferred. Compared with the initial quality image Perform logical AND operation ;

[0017] like If the improvement conditions are met, then the first-layer skeleton image is used. Make corrections;

[0018] like If the improvement criteria are not met, then the second-layer candidate image is used. Compared with the initial quality image Perform logical AND operation ;

[0019] like If the improvement conditions are met, then use the second-layer skeleton image. Make corrections;

[0020] If none of these conditions are met, then maintain the initial quality image. constant.

[0021] Specifically, the improvement condition is defined as: when the candidate image is compared with the initial quality image... The pixels corresponding to the foreground pixels in the new image obtained after performing a logical AND operation contain the initial quality image. Extract the top three pixels by score.

[0022] Specifically, in step S6, the skeleton image is used to refine the initial quality image. The formula for weighted adjustment is:

[0023] ;

[0024] in, For the selected skeleton image ( or ), for The inverted image, This represents a logical AND operation while preserving the original image pixel values. For the initial quality image, This represents the union operation; It is a fixed scaling factor with a value of 1.2.

[0025] Specifically, the distance transformation Specifically, a 5×5 mask kernel is used to calculate the Euclidean distance from the foreground pixel to the nearest background pixel within the mask.

[0026] A robust single-crystal silicon grasping system incorporating geometric structural features includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method.

[0027] The beneficial effects of this invention are:

[0028] This invention effectively overcomes visual noise interference caused by high reflectivity of material surfaces and uses geometric center information to forcibly correct edge grasping deviation of deep learning models, thereby significantly improving the grasping accuracy, stability and safety of automated sorting of monocrystalline silicon. Attached Figure Description

[0029] Figure 1 is a schematic diagram of the process structure of the present invention. Detailed Implementation

[0030] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0031] Example 1

[0032] The method flow of this embodiment mainly includes the following steps:

[0033] S1. Image Acquisition: An RGB-D camera (such as an Intel RealSense D435) is fixedly installed above the monocrystalline silicon sorting workstation. The camera captures images of the monocrystalline silicon material on the conveyor belt, obtaining RGB color images and depth images.

[0034] S2. Image Fusion: To simultaneously utilize the texture edge information and height geometric information of single-crystal silicon, a fused image needs to be generated. .

[0035] fused images Generation: Convert a three-channel RGB image into a single-channel grayscale image. The depth image is normalized and mapped to a monochrome image with 0-256 gray levels. This step removes the influence of absolute distance, retaining only relative morphology; weighted calculation: according to the formula To perform fusion; in this embodiment, set , Parameter basis: Single-crystal silicon has relatively weak surface texture ( Low weighting, but with significant thickness characteristics ( (High weighting). This ratio preserves the geometric outline of the object to the greatest extent, while using texture information to help distinguish overlapping materials.

[0036] S3, Deep Network Prediction: Input the raw RGB-D image into a pre-trained end-to-end grasping and detection network (e.g., an architecture based on a fully convolutional neural network, FCN); Input: The image is pixel-level, and the output is a prediction map containing four channels, representing the captured quality image. (Pixel values ​​0-1 represent capture success rate), capture angle image Capture angle image and capture width image The results obtained at this stage The image may be due to reflections at the edges of monocrystalline silicon, causing high-resolution regions to be incorrectly concentrated at the edges of objects.

[0037] S4. Dual-layer geometric structure feature extraction: Extract the geometric central axis (skeleton) of monocrystalline silicon using machine vision algorithms, and divide it into a first layer and a second layer.

[0038] First layer: Rigorous feature extraction is performed on morphologically intact single-crystal silicon, and the Canny operator is used to fuse the image. Perform edge extraction and set a low threshold. High threshold Parameter basis: Single-crystal silicon has sharp edges, and a lower threshold ensures the continuity of the contour; a filling operation is performed inside the detected closed edges, denoted as... The first mask image is obtained. ;right Perform distance transformation ;use The mask kernel slides across the image, calculates the Euclidean distance from the foreground pixel to the nearest background pixel, and generates a distance image. In the calculation, a larger pixel value indicates that it is closer to the geometric center of the single-crystal silicon; The maximum value in Set distance threshold ,in ,extract The region is used as the first layer of candidate images ,right The center line is extracted using a thinning algorithm to obtain the first-layer skeleton image. .

[0039] The second layer: For monocrystalline silicon with damaged surface shape, statistics... Calculate the threshold based on the grayscale values ​​of all pixels in the dataset. ,in They are respectively The maximum and minimum gray values ​​in the data; traversal If the pixel grayscale value is greater than Otherwise, set it to 0 (background), thus generating the second mask image. It can effectively filter out the weak parts at the edges of monocrystalline silicon; for Perform distance transformation and use a scaling factor Extracting the second-layer candidate image Second layer skeleton image .

[0040] S5. Grasp Improvement Judgment: This step determines whether to trust geometric features and use them to correct the network results.

[0041] First layer candidate images Compared with the quality of the network output image Perform a logical AND operation, that is This operation extracts the capture score located within the geometric center region; and statistically analyzes the initial quality image. Extract the positions of the top three pixels by score from the entire image; the improved condition is defined as: The result image's foreground pixels are checked to see if they contain the top three ranked pixels. If the network's high-confidence capture points do indeed fall near the geometric center region, it indicates that the geometric features and semantic features are consistent, and reinforcement can be performed. If the conditions are met, select. As a corrective skeleton; if it is not satisfied, then calculate ,like If the conditions are met, select. As a corrected skeleton, if neither of these conditions is met (an extremely rare case), it indicates that geometric feature extraction has failed, and the original skeleton should be preserved. constant.

[0042] S6. Quality Image Correction: Assuming the skeleton was selected in step S5. For the initial quality image The formula has been revised as follows:

[0043] ;

[0044] in, In order to capture quality enhancement factors, This indicates extracting the pixels corresponding to the skeleton $L_1$. value, Indicates extraction of areas outside the skeleton ( (Invert) corresponding Value, corrected The highest score is forcibly pulled back to the geometric center axis of monocrystalline silicon, avoiding edge grabbing.

[0045] S7. In the improved quality image Search for the global maximum value point in the middle and use it as the best crawling point. Read the angle corresponding to that point. and width The image coordinates are converted into physical coordinates in the robot arm's base coordinate system, and the robot arm's end effector is controlled to move to these coordinates, adjusting its posture. Opening width , and perform the grabbing action.

[0046] This embodiment provides a hardware system for implementing the above method, the system comprising:

[0047] The memory stores the computer program and the parameters of the capture and detection network model; the processor can be a CPU or a GPU (such as the NVIDIA Jetson series). When the processor executes the program in the memory, it performs steps S1 to S7 above. The processor receives RGB-D camera data and executes the fusion formula. The processor runs the Canny algorithm (threshold 20 / 40) and distance transform (5x5 cores) to extract the skeleton. The processor then performs logical judgments and weighted correction formulas. The processor outputs control commands to the robotic arm controller.

[0048] Robust grasping detection incorporating geometric features is a method used in robotic grasping tasks to improve the accuracy and robustness of grasping detection by utilizing the geometric structure information of objects. In industrial sorting scenarios, robust grasping detection incorporating geometric features can be applied to automated sorting systems to accurately grasp objects of different shapes, sizes, and materials. It can improve the efficiency and accuracy of sorting systems while reducing the cost and error rate of manual operations. Furthermore, this method can also be applied to intelligent warehousing and logistics systems to automate the storage, picking, and distribution of objects, improving the efficiency and accuracy of logistics management.

[0049] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A robust grasping and detection method for single-crystal silicon, characterized in that: Includes the following steps: S1. Acquire an RGB-D image containing the monocrystalline silicon target to be grasped, the RGB-D image consisting of an RGB image and a depth image; S2. Perform weighted fusion on the RGB image and the depth image to generate a fused image containing monocrystalline silicon texture and depth information; S3. Input the RGB-D image into a pre-trained end-to-end grasping detection network and output the initial grasping posture. and initial quality image ,in To capture the center coordinates, In order to capture the angle, S4. Based on the fused image, perform dual-layer geometric structure feature extraction, and extract the first layer skeleton image respectively. First layer candidate images and the second-layer skeleton image Second layer candidate images ； S5. Execute the capture process, comparing the extracted candidate images with the initial quality image. Perform calculations to determine if the improvement conditions are met; S6. If the improvement conditions are met, use the selected skeleton image to improve the initial quality image. Weighted corrections are performed to obtain an improved quality image. And output the final improved grasping posture. ,in, The improved grasping quality score. Step S7: Control the robotic arm to grasp the monocrystalline silicon according to the grasping posture.

2. The robust grasping and detection method for single-crystal silicon according to claim 1, characterized in that, In step S2, a fused image is generated. The specific formula is: ;in, To convert an RGB image into a single-channel grayscale image, A depth image normalized to 0-256 grayscale levels; and These are the fusion ratio coefficients, where ， 。 3. The robust grasping and detection method for single-crystal silicon according to claim 2, characterized in that, In step S4, the first layer skeleton image is extracted. and the first layer candidate image The process includes: using edge detection methods to extract edges from the fused images. Obtain the profile of single-crystal silicon and set a low threshold. The high threshold is twice the low threshold; a filling operation is performed inside the effective edge of the closed loop. Obtain the first mask image ： ;right Perform distance transformation Obtain distance image And set a distance threshold to extract foreground pixels to obtain the first layer of candidate images. ,right The first layer skeleton image is obtained by refining. The distance threshold is a distance image. The maximum value multiplied by the scaling factor ,in 。 4. The robust grasping and detection method for single-crystal silicon according to claim 3, characterized in that, In step S4, the second skeleton image is extracted. Second layer candidate image The process includes: based on the first layer of candidate images Calculate the adaptive threshold The formula is: ;in and Candidate images The maximum and minimum values ​​in the image; based on the first mask image. Perform threshold determination; if the pixel grayscale value is greater than... If the background image is not retained, it is retained; otherwise, it is suppressed and used as background to generate a second mask image. ;right Distance transformation and skeleton extraction are performed to obtain the second-layer candidate image. Second layer skeleton image The distance threshold scaling factor used at this time 。 5. The robust grasping and detection method for single-crystal silicon according to claim 1, characterized in that, The improved capture process described in step S5 specifically involves prioritizing the use of first-layer candidate images. Compared with the initial quality image Perform logical AND operation ； like If the improvement conditions are met, then the first-layer skeleton image is used. Make corrections; if If the improvement criteria are not met, then the second-layer candidate image is used. Compared with the initial quality image Perform logical AND operation ； like If the improvement conditions are met, then use the second-layer skeleton image. Make corrections; if none of these conditions are met, retain the initial quality image. constant.

6. The robust grasping and detection method for single-crystal silicon according to claim 5, characterized in that, The improvement condition is defined as: in the candidate image and the initial quality image The pixels corresponding to the foreground pixels in the new image obtained after performing a logical AND operation contain the initial quality image. Extract the top three pixels by score.

7. The robust grasping and detection method for single-crystal silicon according to claim 5, characterized in that, In step S6, the skeleton image is used to improve the initial quality image. The formula for weighted adjustment is: ;in, For the selected skeleton image ( or ）， for The inverted image, This represents a logical AND operation while preserving the original image pixel values. For the initial quality image, This represents the union operation; It is a fixed scaling factor with a value of 1.

2.

8. The robust grasping and detection method for single-crystal silicon according to claim 3, characterized in that, The distance transformation Specifically, a 5×5 mask kernel is used to calculate the Euclidean distance from the foreground pixel to the nearest background pixel within the mask.

9. A robust single-crystal silicon grasping system incorporating geometric structural features, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

Images