Material sorting robot visual identification method and device based on deep learning

Abstract

The invention relates to the technical field of visual recognition, in particular to a material sorting robot visual recognition method and device based on deep learning, and the method comprises the following steps: S100, obtaining material image information, and importing an image into a deep learning model obtained through training; s200, the outline and pin information of the material and the position information of the material in the material box are obtained through calculation; s300, detailed information of the materials is calculated, and the detailed information is transmitted to the sorting robot; and S400, the sorting robot adjusts the grabbing posture according to the detailed information and the position information, and after the materials are grabbed according to the posture information, the sorting robot conveys the materials to the installation position of the circuit board according to the installation posture. According to the invention, the feeding system can directly select proper components from the material box and convey the components to the chip mounting position of the chip mounter.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and in particular to a material sorting robot visual recognition method and device based on deep learning. Background Art

[0003] Material sorting refers to the process of classifying and screening materials of different types and properties according to certain standards or requirements during industrial production.

[0004] With the continuous development of industry, the number of electrical equipment used by people in production and life is also increasing. The circuit boards in some electrical equipment are manufactured by SMT machines, which are mainly used to automatically mount electronic components on printed circuit boards. During the SMT process, the raw materials are first taken out and placed on the rack. After the feeding system takes the components from the rack, it is transported to the bottom of the SMT machine through a conveyor belt and welded. Existing electrical equipment has complex functions, and the types and quantities of components used in circuit boards are large. If they are placed on the corresponding racks according to the component types, the racks in the SMT machine will occupy a lot of space, which will cause the SMT machine to be bulky and require many parts, which is not conducive to installation and movement.

[0005] Therefore, those skilled in the art are committed to developing a material sorting robot visual recognition method and device based on deep learning, which is conducive to the feeding system to directly select suitable components in the material box and transport them to the placement machine. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a material sorting robot visual recognition method and device based on deep learning, which is beneficial for the feeding system to directly select suitable components in the material box and transport them to the placement of the placement machine. The technical solution of the present invention to solve the above technical problems is as follows: A material sorting robot visual recognition method based on deep learning, characterized in that it includes the following steps: S100. Obtain images of electronic components to be sorted in the material box and coding information on the surface of the components through the sorting robot camera, and import the images into the deep learning model obtained through training; S200. The image obtains the material contour, pin information, and location information of the material in the material box through an edge detection algorithm; S300. Calculate the type of the current material through the contour, and pass the material type and coding information to the calculator. The calculator compares the material type and coding information with the input data in the training database, calculates the detailed information of the material, and passes the detailed information to the sorting robot; S400. The sorting robot adjusts the grasping posture according to the detailed information and position information. After grasping the material according to the posture information, the sorting robot transports it to the circuit board installation position according to the installation posture.

[0008] The beneficial effects of adopting the above scheme are: using the camera of the sorting robot to obtain the image of the electronic components to be sorted in the material box and the inkjet information on the surface of the components, importing the image into the deep learning model obtained through training, and using the edge detection algorithm to obtain the material's contour, pin information and the location information of the material in the material box. Through the deep learning model and edge detection algorithm, the system can more accurately identify and locate components, thereby improving the accuracy and efficiency of sorting; The sorting robot adjusts the grasping posture according to the detailed information and location information. After grasping the material according to the posture information, the sorting robot transports it to the circuit board installation position according to the installation posture. The material sorting system integrating deep learning and machine vision technology improves the intelligence and automation level of the entire production line. Since different components, such as resistors, capacitors, transistors, diodes, magnetic coils, chips, etc., have different shapes and pins, and have different parameters under the same shape, when the camera obtains the image in the material box, it first preliminarily calculates the type of material based on the material's outline and pin information, and then compares the information input into the database with the inkjet coding on the component (the components have different shapes, and it is difficult to obtain complete inkjet coding information by taking images, and most of the time only partial inkjet coding information can be obtained), so as to determine the type, parameters and other information of the material, and locate the component in the material box. When this component is needed, the manipulator on the sorting robot adjusts the grasping posture or adopts different grasping methods (such as suction cups, precision tweezers, etc.) to grasp the component and adjust it to the posture to be installed, and then transport it to the circuit board patch position for installation, thereby reducing manual contact with dangerous or precise components and reducing safety risks in the workplace.

[0009] Based on the above technical solution, the present invention can also be improved as follows.

[0010] Further, step S200 includes the following steps: S210. After the image obtains the outline of the material through the edge detection algorithm, the outline of the material that is accurately identified is marked as a recognizable material, and the outline of the material that cannot be accurately identified is marked as an unrecognizable material, and the position of the recognizable non-material and unrecognizable material is continuously located; S220. During the continuous positioning process, the shaking device shakes the material box so that the posture of the material in the material box changes, and the robot camera obtains the image of the marked unrecognizable material in the material box, and transmits the image information to the trained deep learning model to obtain the contour, pin information and position information of the material for the second time; S230. Repeat steps S210 to S220 three to five times, mark the materials that cannot be identified as unused materials, and transmit the contour information of all identifiable materials to the calculator.

[0011] The beneficial effect of adopting the above further scheme is: after the image is acquired, when it is difficult to accurately determine the type and parameters of the components through the inkjet coding information obtained by calculation, the components whose information can be accurately acquired are first located, and then the shaking device is slightly shaken to slightly change the placement posture in the material box, and the image in the material box is acquired again, and the components that have been accurately acquired are removed, and the edge contours, stitches, etc. of the components that cannot be identified are further acquired for further judgment, and this is repeated three to five times. The components that ultimately cannot obtain accurate information are removed from the grasping information, and will no longer be grasped in the subsequent grasping process. The position and posture of the components with accurate information are accurately recorded for precise grasping by the subsequent grasping robot.

[0012] Further, in step S300, after the calculator obtains the material coding information, it extracts feature information in the coding image, such as coding clarity, contrast and edge information, through a computer vision algorithm, and matches the coding feature information with the coding data entered into the database through a BP neural network to obtain accurate material information.

[0013] The beneficial effect of adopting the above further scheme is that the key features such as cleanliness, contrast and edge information in the inkjet image can be extracted through the computer vision algorithm, which is beneficial for the subsequent BP neural network to improve the accuracy of material recognition by learning the relationship between these features and the inkjet data.

[0014] Further, in step S200, the edge detection algorithm includes Sobel, Canny or Prewitt operator, and the algorithm parameters are adaptively adjusted according to the material characteristics.

[0015] The beneficial effects of adopting the above further scheme are: multiple adaptive algorithms can dynamically adjust the learning rate and parameter update strategy according to different problems and data sets, have strong adaptability, can handle various complex training tasks, reduce the risk of overfitting, and improve the generalization performance of the model on new data.

[0016] Furthermore, a genetic algorithm is used to optimize the network weights and biases during the training process of the BP neural network.

[0017] The beneficial effect of adopting the above further scheme is that the BP neural network genetic algorithm can optimize these parameters and reduce the dependence on random initialization. Furthermore, after step S400, the following steps are further included: Step S500, that is, after the sorting is completed, the sorting results are rechecked, the accuracy of the installation position is evaluated using the deep learning model, and the feedback is fed back to the deep learning model for self-learning and optimization.

[0018] The beneficial effects of adopting the above further scheme are: through re-inspection and evaluation, the high accuracy of the sorting results can be ensured, and the subsequent problems caused by incorrect sorting can be reduced. The deep learning model can self-learn and optimize according to the feedback, and continuously adjust and improve the model parameters to improve the performance of future sorting tasks. The feedback loop allows the model to be updated in real time and respond quickly to changes in the production process.

[0019] Furthermore, after step S500, the following steps are further included: S600. During the material sorting process, the robot’s visual system performance is monitored in real time, and the visual recognition parameters are dynamically adjusted based on performance feedback.

[0020] The beneficial effects of adopting the above further scheme are: real-time monitoring can ensure that the visual system always maintains the best performance state, timely detects and corrects recognition errors, improves the accuracy of material sorting, and dynamically adjusts parameters so that the visual system can adapt to different working environments and changes in conditions, such as lighting, background noise, etc., to maintain stable recognition performance.

[0021] Furthermore, in step S300, the computer uses a fuzzy matching algorithm to improve the fault tolerance of the matching when comparing the input data in the database.

[0022] The beneficial effect of adopting the above further solution is that the fuzzy matching algorithm allows similar items to be found when there are errors or incomplete matches in the data, thereby improving the accuracy of component matching and improving the intelligence level of the system.

[0023] Further, step S100 includes the following steps: S110. graying, reducing noise and enhancing contrast of the acquired image; S120. Use image segmentation technology to separate the material from the background and pass the separated information to the deep learning model.

[0024] The beneficial effects of adopting the above further scheme are: grayscale, noise reduction and contrast enhancement can significantly improve the visual effect of the image, improve the clarity and readability of the image, and these preprocessing steps help to enhance the key features in the image, making the characteristics of the components more prominent and facilitating subsequent identification and analysis.

[0025] A material sorting robot visual recognition device based on deep learning is applied to the material sorting robot visual recognition method based on deep learning as described above, comprising: A material box, wherein a vibration device is installed on the side wall of the material box; A sorting robot, wherein the sorting robot is mounted on a sliding device, and the sorting robot has a grabbing arm and a CCD camera, and the grabbing arm and the CCD camera are located on an upper side of the material box; A communication device is electrically connected to the vibration device and the sorting robot, and the communication device is connected to a calculator and a database. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flowchart of a specific embodiment of the present invention; Figure 2 This is a schematic diagram of the mechanism of the material sorting robot visual recognition device based on deep learning in the present invention.

[0027] In the accompanying drawings, the components represented by the reference numerals are listed as follows: 1. Material box; 2. Vibration device; 3. Sorting robot; 4. Sliding device; 5. Grabbing arm; 6. CCD camera; 7. Communication device. DETAILED DESCRIPTION

[0029] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0030] In the description of the present invention, it is necessary to understand that the terms "center", "length", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "inside", "outside", "peripheral", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred system or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0031] In the description of the present invention, “plurality” means at least two, for example, two, three, etc., unless otherwise clearly and specifically defined.

[0032] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] Embodiment 1 like Figure 1 As shown, a material sorting robot visual recognition method based on deep learning includes the following steps: S100. Image acquisition and preprocessing: The image of the electronic components to be sorted in the material box and the coding information on the surface of the components are acquired through the camera of the sorting robot, and the image is imported into the deep learning model obtained through training; In the specific steps, S110. grayscale, denoise and contrast enhancement are performed on the acquired image to improve image quality and enhance the recognizability of component features.

[0034] S120. Use image segmentation technology to separate materials from the background and pass the separated information to the deep learning model to provide accurate image data for subsequent recognition and positioning.

[0035] S200. Edge detection and material positioning: The image uses edge detection algorithms to obtain the material's outline, pin information, and the material's location information in the material box; In the specific steps, S210. After the image obtains the outline of the material through the edge detection algorithm, the accurately identified material outline is marked as a recognizable material, and the material outline that cannot be accurately identified is marked as an unrecognizable material, and the positions of recognizable non-material and unrecognizable material are continuously located. The edge detection algorithm includes Sobel, Canny or Prewitt operators, and the algorithm parameters are adaptively adjusted according to the material characteristics to adapt to different material characteristics and image conditions.

[0036] S220. During the continuous positioning process, the shaking device shakes the material box so that the posture of the material in the material box changes, and the robot camera obtains the image of the marked unrecognizable material in the material box, and transmits the image information to the trained deep learning model to obtain the contour, pin information and position information of the material for the second time; S230. Repeat steps S210 to S220 three to five times, mark the materials that cannot be identified as unused materials, and transmit the contour information of all identifiable materials to the calculator.

[0037] S300. Calculate the type of the current material through the contour, and pass the material type and coding information to the calculator. The calculator compares the material type and coding information with the input data in the training database, calculates the detailed information of the material, and passes the detailed information to the sorting robot; In the specific steps, after the calculator obtains the material coding information, it extracts the characteristic information in the coding image, such as coding clarity, contrast and edge information, through the computer vision algorithm, and matches the coding characteristic information with the coding data entered into the database through the BP neural network to obtain accurate information of the material. The genetic algorithm is used to optimize the network weight and bias during the training process of the BP neural network. When comparing the entered data in the database, the calculator uses a fuzzy matching algorithm to improve the matching fault tolerance.

[0038] S400. Material grabbing and conveying: The sorting robot adjusts the grabbing posture according to the detailed information and position information. After grabbing the material according to the posture information, the sorting robot conveys it to the circuit board installation position according to the installation posture.

[0039] After step S400, the following steps are also included: Rechecking and self-learning of sorting results: Step S500, that is, after the sorting is completed, rechecking the sorting results, using the deep learning model to evaluate the accuracy of the installation position, and feeding back to the deep learning model for self-learning and optimization.

[0040] After step S500, the following steps are also included: Real-time monitoring and parameter adjustment: During the material sorting process, the S600 monitors the robot's visual system performance in real time and dynamically adjusts the visual recognition parameters based on performance feedback.

[0041] Embodiment 2 The only difference between the second embodiment and the first embodiment is that a camera is used to capture images in the material box, identify, locate, and shake to continue identifying. For some components that are small in size, it is difficult to locate them when they are shaken after identification, resulting in a large amount of calculation for the subsequent calculator.

[0042] In the second embodiment, the following steps are included: S100. Obtain images of electronic components to be sorted in the material box and coding information on the surface of the components through the sorting robot camera, and import the images into the deep learning model obtained through training; S200. The image obtains the material contour, pin information, and location information of the material in the material box through an edge detection algorithm; S300. Calculate the type of the current material through the contour, and pass the material type and coding information to the calculator. The calculator compares the material type and coding information with the input data in the training database, calculates the detailed information of the material, and passes the detailed information to the sorting robot; S400. The sorting robot adjusts the gripping posture according to the detailed information and position information, grabs the identified components from the material box and moves them to the placement box, and records the position of the components in the placement box; S500. Repeat steps S100 to S400 until all components in the material box are identified, and are placed in the placement box one by one according to the placement requirements, and the placement position and placement posture of each component are recorded; S600. During the patch work, the sorting robot picks up the components in the placement box one by one according to the installation posture according to the patch requirements and transports them to the installation position of the patch machine.

[0043] Embodiment 3 like Figure 2 As shown, the present invention also provides a material sorting robot visual recognition device based on deep learning, which is applied to the material sorting robot visual recognition method based on deep learning as described above, including A material box 1, a vibration device 2 is installed on the side wall of the material box 1; A sorting robot 3, wherein the sorting robot 3 is mounted on a sliding device 4, and the sorting robot 3 has a grabbing arm 5 and a CCD camera 6, and the grabbing arm 5 and the CCD camera 6 are located on the upper side of the material box 1; The communication device 7 is electrically connected to the vibration device 2 and the sorting robot 3, and the communication device 7 is connected to a calculator and a database.

[0044] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0045] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A material sorting robot visual recognition method based on deep learning, characterized in that: The following steps are involved: S100. Obtain images of electronic components to be sorted in the material box and coding information on the surface of the components through the sorting robot camera, and import the images into the deep learning model obtained through training; S200. The image obtains the material contour, pin information, and location information of the material in the material box through an edge detection algorithm; S300. Calculate the type of the current material through the contour, and pass the material type and coding information to the calculator. The calculator compares the material type and coding information with the input data in the training database, calculates the detailed information of the material, and passes the detailed information to the sorting robot; S400. The sorting robot adjusts the grasping posture according to the detailed information and position information. After grasping the material according to the posture information, the sorting robot transports it to the circuit board installation position according to the installation posture.

2. The material sorting robot visual recognition method based on deep learning according to claim 1 is characterized in that: Step S200 includes the following steps: S210. After the image obtains the outline of the material through the edge detection algorithm, the outline of the material that is accurately identified is marked as a recognizable material, and the outline of the material that cannot be accurately identified is marked as an unrecognizable material, and the position of the recognizable non-material and unrecognizable material is continuously located; S220. During the continuous positioning process, the shaking device shakes the material box so that the posture of the material in the material box changes, and the robot camera obtains the image of the marked unrecognizable material in the material box, and transmits the image information to the trained deep learning model to obtain the contour, pin information and position information of the material for the second time; S230. Repeat steps S210 to S220 three to five times, mark the materials that cannot be identified as unused materials, and transmit the contour information of all identifiable materials to the calculator.

3. The material sorting robot visual recognition method based on deep learning according to claim 1 is characterized in that: In step S300, after the calculator obtains the material coding information, it extracts the feature information in the coding image, such as coding clarity, contrast and edge information, through the computer vision algorithm, and matches the coding feature information with the coding data entered into the database through the BP neural network to obtain accurate information of the material.

4. The material sorting robot visual recognition method based on deep learning according to claim 1 is characterized in that: In step S200, the edge detection algorithm includes Sobel, Canny or Prewitt operator, and the algorithm parameters are adaptively adjusted according to the material characteristics.

5. The material sorting robot visual recognition method based on deep learning according to claim 3 is characterized in that: Genetic algorithm is used to optimize network weights and biases during the training process of BP neural network.

6. The material sorting robot visual recognition method based on deep learning according to claim 1 is characterized in that: After step S400, the following steps are also included: Step S500, that is, after the sorting is completed, the sorting results are rechecked, the accuracy of the installation position is evaluated using the deep learning model, and the feedback is fed back to the deep learning model for self-learning and optimization.

7. The material sorting robot visual recognition method based on deep learning according to claim 6 is characterized in that: After step S500, the following steps are also included: S600. During the material sorting process, the robot’s visual system performance is monitored in real time, and visual recognition parameters are dynamically adjusted based on performance feedback.

8. The material sorting robot visual recognition method based on deep learning according to claim 6 is characterized in that: In step S300, the computer uses a fuzzy matching algorithm to improve the fault tolerance of the matching when comparing the input data in the database.

9. The material sorting robot visual recognition method based on deep learning according to claim 1 is characterized in that: Step S100 includes the following steps: S110. graying, reducing noise and enhancing contrast of the acquired image; S120. Use image segmentation technology to separate the material from the background and pass the separated information to the deep learning model.

10. A material sorting robot visual recognition device based on deep learning, applied to the material sorting robot visual recognition method based on deep learning according to any one of claims 1 to 9, characterized in that: include A material box, wherein a vibration device is installed on the side wall of the material box; A sorting robot, wherein the sorting robot is mounted on a sliding device, and the sorting robot has a grabbing arm and a CCD camera, and the grabbing arm and the CCD camera are located on an upper side of the material box; A communication device is electrically connected to the vibration device and the sorting robot, and the communication device is connected to a calculator and a database.

Images