Lamp bead sorting control method and system based on image recognition and storage medium

Abstract

The invention provides a lamp bead sorting control method and system based on image recognition and a storage medium, and the method comprises the steps: firstly, collecting an initial image of a lamp bead through a camera, calculating a distance ratio of a pin spacing to a ball head diameter in real time, and controlling a rotating mechanism to circularly adjust until the distance ratio reaches the standard; secondly, pin length analysis and internal structure feature recognition are synchronously executed, and positive and negative electrodes are comprehensively judged by adopting a neural network model through fusion of double-path confidence scoring; then, current characteristic detection and light-emitting state capture are carried out on an image recognition result, and polarity accuracy is verified through double physical signal cross validation; and finally, the feeding and discharging states of the lamp beads are controlled based on the polarity judgment result, so that the braid production cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of lamp bead patch, and more specifically, to a lamp bead sorting control method, system and storage medium based on image recognition. Background Art

[0002] With the popularization of surface mount technology (SMT) and the expansion of LED applications (such as lighting, display screens, in-vehicle electronics, etc.), the demand for efficient and precise component feeding in electronic manufacturing has increased sharply. Traditional feeding methods such as tape feeding have the problem of high cost due to reliance on customized carrier tapes, and at the same time, it is difficult to meet the production requirements of small batches and multiple varieties, especially the processing of bulk LED lamp beads. Moreover, a single carrier tape is only suitable for fixed-specification lamp beads and is difficult to cope with the increasing demand for customized and bulk material mixed-line production. In addition, before customizing the carrier tape, the control of the polarity direction consistency is weak through manual sorting or vibrating disk feeding, resulting in a decrease in the mounting yield. Therefore, there is an urgent need for a feeding solution that can synchronously achieve precise sorting of bulk materials and dynamic polarity verification. Summary of the Invention

[0003] In view of the above problems, the purpose of the present invention is to provide a lamp bead sorting control method, system and storage medium based on image recognition. First, by controlling the rotation mechanism to cycle and adjust based on the lamp bead image until the distance ratio reaches the standard, the problem of randomness of the bulk material posture is solved; secondly, the pin length analysis and internal structure recognition are synchronously executed, and the confidence scores of the two paths are fused, and the positive and negative poles are comprehensively determined through a neural network model to improve the adaptability to complex shapes; then, the current characteristic detection and the luminous state capture are carried out, and the accuracy of the polarity is cross-verified through dual physical signals to form a closed-loop error correction mechanism; finally, the feeding state of the lamp beads is controlled based on the polarity judgment result; through image recognition and physical detection, the recognition rate of the lamp bead polarity is improved, and thus the production tape cost is reduced.

[0004] The first aspect of the present invention provides a lamp bead sorting control method based on image recognition, and the method includes: Grasping the lamp bead by a vacuum suction nozzle and taking the first image information of the lamp bead; Matching the first image information with a preset distance reference vector to obtain the first distance ratio information; Rotating the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, and then taking the second image information of the lamp bead; Generating a polarity feature vector according to the second image information; Based on a preset recognition model, obtaining the polarity information according to the polarity feature vector; Rotating the lamp bead according to the polarity information and moving it to the electrical retest position; Based on a preset electrical retest mechanism, judging whether the polarity information is correct; If so, place the LED bead into the discharge channel according to the polarity information; If not, record a log according to the polarity feature vector, and place the LED bead into the feed channel to wait for re-detection.

[0005] In this solution, it also includes adjusting the rotation deviation angle, specifically: Match a preset distance reference vector according to the first image information, identify the pins and the ball head of the LED bead, and obtain the pin pitch information and the ball head diameter information of the LED bead; Calculate the ratio of the pin pitch information to the ball head diameter information of the LED bead to obtain the first distance ratio information; Based on a preset ratio-angle mapping relationship, obtain the rotation deviation angle according to the first distance ratio information; Control the rotation mechanism to adaptively adjust the orientation of the LED bead according to the deviation angle, and repeatedly execute the first image acquisition and rotation deviation angle calculation operations until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the LED bead.

[0006] In this solution, the distance ratio threshold is a static distance ratio threshold or a dynamic distance ratio threshold, specifically: Judge whether the LED bead specification information is obtained according to the first image information; If so, based on a preset static threshold mapping table, obtain a static threshold according to the LED bead specification information to set the distance ratio threshold; If not, adopt a dynamic detection method, continuously collect at least three groups of distance ratios during the rotation of the LED bead to obtain a distance ratio sequence; If the distance ratio sequence meets a preset rule of rising first and then falling, extract the maximum value of the distance ratio sequence to obtain the second distance ratio information; Based on a preset deviation tolerance mechanism, obtain the distance ratio threshold according to the second distance ratio information.

[0007] In this solution, obtaining the polarity information based on the preset recognition model according to the polarity feature vector is specifically: Based on the recognition model, obtain the pin length information and the first confidence information; Obtain the first polarity information according to the pin length information; Based on the recognition model, identify the narrowing part shape and the expanding part shape of the LED bead, and the second confidence information; Obtain the second polarity information according to the narrowing part shape and the expanding part shape; Obtain the first weight value and the second weight value according to the first confidence information and the second confidence information; Obtain the polarity information based on the first polarity information, the second polarity information, the first weight, and the second weight.

[0008] In this solution, the electrical remeasurement mechanism is specifically as follows: Apply a forward voltage to the lamp bead according to the polarity information, measure the first current information or identify the light-emitting state of the lamp bead by photographing an image; Determine whether the first current information exceeds a preset current threshold; If so, the first polarity determination information is in the correct state; If not, the first polarity determination information is in the wrong state; Determine whether the lamp bead is in the light-emitting state; If so, the second polarity determination information is in the correct state; If not, the second polarity determination information is in the wrong state; Determine whether the first polarity determination information is the same as the second polarity determination information; If so, set the determination result of the polarity information according to the first polarity determination information; If not, the polarity information is in the wrong state.

[0009] In this solution, recording a log according to the polarity feature vector and placing the lamp bead into the feeding channel for re-detection further includes: Set a first identifier for the lamp bead with incorrect polarity identification; Put the lamp bead into the return material channel through a pneumatic valve; According to the first identifier, when it is determined that the same lamp bead returns to the feeding port three times continuously, trigger a system alarm and move it to the waste bin.

[0010] A second aspect of the present invention provides a lamp bead sorting control system based on image recognition, including a lamp bead sorting control method program based on image recognition. When the lamp bead sorting control method program based on image recognition is executed by the processor, the following steps are implemented: Grab the lamp bead with a vacuum suction nozzle and capture the first image information of the lamp bead; Match the first image information with a preset distance reference vector to obtain the first distance ratio information; Rotate the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the lamp bead; Generate a polarity feature vector according to the second image information; Based on a preset recognition model, obtain the polarity information according to the polarity feature vector; Rotate the lamp bead according to the polarity information and move it to the electrical remeasurement position; Based on a preset electrical remeasurement mechanism, determine whether the polarity information is correct; If so, place the lamp beads into the discharge channel according to the polarity information; If not, record a log according to the polarity feature vector and place the lamp beads into the feed channel to wait for re-detection.

[0011] In this solution, it also includes adjusting the rotation deviation angle, specifically: Match a preset distance reference vector according to the first image information, identify the pins and the lamp bead ball head, and obtain the pin pitch information and the lamp bead ball head diameter information; Calculate the ratio of the pin pitch information to the lamp bead ball head diameter information to obtain the first distance ratio information; Based on a preset ratio-angle mapping relationship, obtain the rotation deviation angle according to the first distance ratio information; Control the rotation mechanism to adaptively adjust the orientation of the lamp beads according to the deviation angle, and repeatedly execute the first image acquisition and rotation deviation angle calculation operations until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the lamp beads.

[0012] In this solution, the distance ratio threshold is a static distance ratio threshold or a dynamic distance ratio threshold, specifically: According to the first image information, determine whether lamp bead specification information is obtained; If so, based on a preset static threshold mapping table, obtain a static threshold according to the lamp bead specification information to set the distance ratio threshold; If not, adopt a dynamic detection method to continuously collect at least three groups of distance ratios during the rotation of the lamp beads to obtain a distance ratio sequence; If the distance ratio sequence satisfies a preset rule of rising first and then falling, extract the maximum value of the distance ratio sequence to obtain the second distance ratio information; Based on a preset deviation tolerance mechanism, obtain the distance ratio threshold according to the second distance ratio information.

[0013] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for a lamp bead sorting control method based on image recognition. When the program for the lamp bead sorting control method based on image recognition is executed by a processor, the steps of the lamp bead sorting control method based on image recognition as described in any one of the above are implemented.

[0014] The present invention provides a method, system and storage medium for controlling the sorting of lamp beads based on image recognition. First, an initial image of the lamp beads is collected by a camera to calculate the distance ratio between the pin pitch and the ball head diameter in real time, and the rotation mechanism is cyclically adjusted until the distance ratio reaches the standard. Secondly, the pin length analysis and the internal structure feature recognition are synchronously executed, and the positive and negative poles are comprehensively determined by a neural network model through fusing the confidence scores of the two paths. Then, the current characteristics of the image recognition result are detected and the light emitting state is captured, and the polarity accuracy is cross-verified by dual physical signals. Finally, the feeding and discharging states of the lamp beads are controlled based on the polarity judgment result, thereby reducing the production and taping cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope.

[0016] Figure 1 Shows the image recognition feature vector diagram of the lamp beads provided by the embodiments of the present invention; Figure 2 Shows the flowchart of a method for controlling the sorting of lamp beads based on image recognition according to the present invention; Figure 3 Shows the flowchart of continuously adjusting the first distance ratio information provided by the embodiments of the present invention; Figure 4 Shows the flowchart of setting the distance ratio threshold provided by the embodiments of the present invention; Figure 5 Shows the block diagram of a system for controlling the sorting of lamp beads based on image recognition according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] Unless otherwise defined, all terms (including technical and scientific terms) used in the embodiments of the present invention have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless clearly defined in the embodiments of the present invention.

[0019] In the embodiments of the present invention, the terms "first", "second" and similar terms do not denote any order, quantity or importance, but are only used to distinguish different components. The terms "a", "an" or "the" and similar terms do not denote a limitation of quantity, but mean that there is at least one. Similarly, the terms "comprising" or "including" and similar terms mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms "connected" or "coupled" and similar terms are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The steps before or after the method of the embodiments of the present invention do not necessarily need to be carried out precisely in order. On the contrary, various steps can be carried out in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0020] In addition, in each embodiment of the present invention, each functional module can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0021] Figure 1 The image recognition feature vector diagram of the lamp beads provided by the embodiments of the present invention is shown.

[0022] As shown in 1, Figure 1 is the feature map obtained after image recognition. The pin pitch is L1 and the ball head diameter of the lamp bead is D1. Among them, both L1 and D1 are pixel distances expressed in pixel points; the long pin 101 is determined as the positive pin under normal circumstances; the short pin 102 is determined as the negative pin under normal circumstances; the narrowing part 103 is usually the internal morphological feature of the positive pole of the lamp bead; the expansion part 104 is usually the internal morphological feature of the negative pole of the lamp bead. As Figure 1 shown, the plane formed by the positive and negative two pins is defined as the front of the lamp bead; the front of the lamp bead is perpendicular to the camera shooting angle, which is recognized as the preset reference plane of the lamp bead. At this time, the image effect of the internal morphology of the lamp bead is the best; according to the relationship between the pin pitch and the ball head diameter, when the ratio of the pin pitch to the ball head diameter is larger, the included angle between the front of the lamp bead and the reference plane of the lamp bead is smaller, and the image effect of the internal morphology of the lamp bead is better.

[0023] Figure 2 The flowchart of a lamp bead sorting control method based on image recognition according to the present invention is shown.

[0024] As Figure 2 shown, in the first aspect of the present invention, a lamp bead sorting control method based on image recognition is disclosed, and the method includes: S202, the suction nozzle grabs the lamp bead and captures the first image information of the lamp bead; S204. Match the preset distance reference vector according to the first image information to obtain the first distance ratio information; S206. Rotate the lamp bead until the first distance ratio information is greater than the preset distance ratio threshold, and then capture the second image information of the lamp bead; S208. Generate a polarity feature vector according to the second image information; S210. Based on the preset recognition model, obtain the polarity information according to the polarity feature vector; S212. Rotate the lamp bead according to the polarity information and move it to the electrical retest position; S214. Based on the preset electrical retest mechanism, determine whether the polarity information is correct; S216. If so, put the lamp bead into the discharge channel according to the polarity information; S218. If not, record a log according to the polarity feature vector and put the lamp bead into the feeding channel to wait for re - detection.

[0025] It should be noted that this embodiment provides a control process for lamp bead sorting based on image recognition. In this embodiment, first, the lamp bead is grabbed by a vacuum suction nozzle and the image of the lamp bead is captured by an industrial camera, which is the first image information. Based on the first image information, after identifying the lamp bead pins and the lamp bead ball head, the pixel distance between the pins and the pixel distance of the ball head diameter are respectively collected to calculate the ratio of the pin pitch to the ball head diameter in real - time, obtaining the first distance ratio information. The suction nozzle is driven to rotate by a rotating mechanism, thereby driving the lamp bead to rotate, continuously adjusting the lamp bead attitude until the first distance ratio information is greater than the preset distance ratio threshold, and capturing the second image information, indicating that the front - facing direction of the lamp bead has met the picture - taking requirements for subsequent polarity recognition. By continuously rotating and detecting the distance ratio, the front - facing direction of the lamp bead is ensured, improving the image quality during subsequent image recognition and solving the problem of randomness in the posture of bulk materials. Secondly, according to the second image information, through the preset feature extraction process, a polarity feature vector is obtained. Based on the preset neural network recognition model, the polarity feature vector is matched and recognized, and the method of synchronously performing pin length analysis and internal structure feature extraction is used to identify the lamp bead polarity. Then, according to the identified lamp bead polarity, the lamp bead is rotated and moved to the electrical retest position for electrical detection. By detecting the current characteristics and capturing the light - emitting state, the accuracy of the identified lamp bead polarity is cross - verified through dual physical signals, thereby forming a closed - loop error - correction mechanism. Finally, the lamp beads with correct polarity verification are put into the discharge channel and neatly arranged in the correct direction and conveyed to the pick - up position of the mounter in sequence; the lamp beads with incorrect polarity verification are sent back to the feeding channel to wait for re - detection, and the recognition error status is also recorded through logs and identifiers.

[0026] Figure 3 Shows a flowchart of continuously adjusting the first distance ratio information provided by an embodiment of the present invention.

[0027] According to an embodiment of the present invention, as Figure 3 shown, it further includes adjusting the rotation deviation angle, specifically: S302, matching a preset distance reference vector according to the first image information, identifying the pins and the ball heads of the lamp beads, and obtaining the pin pitch information and the ball head diameter information of the lamp beads; S304, calculating the ratio of the pin pitch information to the ball head diameter information of the lamp beads to obtain the first distance ratio information; S306, based on a preset ratio-angle mapping relationship, obtaining the rotation deviation angle according to the first distance ratio information; S308, controlling the rotation mechanism to adaptively adjust the orientation of the lamp bead according to the deviation angle, and circularly executing the first image acquisition and rotation deviation angle calculation operations until the first distance ratio information is greater than a preset distance ratio threshold, and then capturing the second image information of the lamp bead.

[0028] It should be noted that this embodiment provides a logic for continuously adjusting the rotation deviation angle. The rotation deviation angle is the angle between the front of the lamp bead and the reference plane of the lamp bead. In this embodiment, during the process of rotating the lamp bead, by calculating the pixel distance ratio of the pin pitch of the lamp bead to the ball head diameter in real time, and then based on a preset ratio-angle mapping relationship, the rotation deviation angle is calculated. Among them, the smaller the rotation deviation angle, the closer the front of the lamp bead is to being parallel to the reference plane of the lamp bead, and at this time, the internal features extracted from the captured lamp bead image are clearer. By continuously rotating the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, it is then determined that the second image information captured at this time meets the subsequent polarity identification requirements. This embodiment ensures the acquisition of a standard front image by controlling the rotation mechanism to circularly adjust until the ratio meets the standard, and solves the problem of the randomness of the posture of the bulk materials.

[0029] Figure 4 shows a flowchart for setting a distance ratio threshold provided by an embodiment of the present invention.

[0030] According to an embodiment of the present invention, as Figure 4 shown, the distance ratio threshold is a static distance ratio threshold or a dynamic distance ratio threshold, specifically: S402, judging whether lamp bead specification information is obtained according to the first image information; S404, if so, based on a preset static threshold mapping table, obtaining a static threshold according to the lamp bead specification information to set the distance ratio threshold; S406, if not, then using a dynamic detection method, continuously collecting at least three groups of distance ratios during the rotation of the lamp bead to obtain a distance ratio sequence; S408. If the distance ratio sequence satisfies a preset rule of first increasing and then decreasing, extract the maximum value of the distance ratio sequence to obtain second distance ratio information. S410. Based on a preset deviation tolerance mechanism, obtain the distance ratio threshold according to the second distance ratio information.

[0031] It should be noted that if the lamp bead is an industry standard part, its pin pitch and ball head diameter need to conform to the reference values set by the industry, and the reference values constitute the lamp bead specification information. By identifying the lamp bead brand, execution specification number, standard product image comparison or manual input, etc., it is determined whether the lamp beads in the current batch are industry standard parts, that is, it is used to judge whether the lamp bead specification information is obtained. If the lamp bead specification information is obtained, the static threshold method is used to obtain the distance ratio threshold; that is, based on a preset static threshold mapping table, the static threshold is queried according to the lamp bead specification and used as the distance ratio threshold. If the lamp bead specification information is not obtained, the dynamic detection method is used to set the distance ratio threshold. The execution process of the dynamic detection method includes continuously rotating the lamp bead and calculating multiple groups of distance ratios in real time to combine into a distance ratio sequence. Based on methods such as difference comparison and slope fitting, the distance ratio sequence is tested. When it is determined that the elements in the distance ratio sequence satisfy the rule of first increasing and then decreasing, the maximum value in the distance ratio sequence is extracted as the second distance ratio information. Finally, based on the set deviation tolerance mechanism, the distance ratio threshold is obtained according to the second distance ratio information; for example, 80% of the second distance ratio information is used as the distance ratio threshold.

[0032] According to an embodiment of the present invention, obtaining the polarity information according to the polarity feature vector based on the preset recognition model specifically includes: Based on the recognition model, obtain the pin length information and the first confidence information. According to the pin length information, obtain the first polarity information. Based on the recognition model, identify the narrowing part shape and the expanding part shape of the lamp bead, and the second confidence information. According to the narrowing part shape and the expanding part shape, obtain the second polarity information. According to the first confidence information and the second confidence information, obtain the first weight and the second weight. According to the first polarity information, the second polarity information, the first weight and the second weight, obtain the polarity information.

[0033] It should be noted that this embodiment provides a dual-path fusion polarity recognition mechanism. The pin length analysis and internal structure feature extraction are performed synchronously, and confidence scores are set respectively; combining the polarity judgment results and confidence scores of the dual paths, the polarity information of the lamp bead is calculated. In this embodiment, first, through a preset neural network recognition model, the pins in the polarity feature vector are recognized and located, and the lengths of the two pins are measured to obtain the first polarity information; among them, the first polarity information is used to judge the polarity of the lamp bead by the length of the pins. And, based on information such as the pin length relationship, pin bending condition, and pin breakage condition, the first confidence information is obtained through conversion by the neural network model. Then, through a preset neural network recognition model, the internal structure features in the polarity feature vector are recognized and located, including but not limited to the morphological features of the narrowing part and the expanding part of the lamp bead, to obtain the second polarity information; among them, the second polarity information is used to judge the polarity of the lamp bead by the internal features. And, based on information such as the rotation deviation angle and the morphological matching degree between the narrowing part and the expanding part, the second confidence information is obtained through conversion by the neural network model. Finally, according to the first confidence information and the second confidence information, the first weight and the second weight are obtained according to a preset weight distribution algorithm. As a real-time method, according to the lamp bead image, when the left side is the positive electrode and the right side is the negative electrode, the first polarity information and the second polarity information are quantified as 1; when the left side is the negative electrode and the right side is the positive electrode, the first polarity information and the second polarity information are quantified as -1; combining the corresponding first weight and second weight, the polarity calculation result is obtained. If the polarity calculation result is greater than 0, it means that the left side of the lamp bead is the positive electrode and the right side is the negative electrode; if the polarity calculation result is less than 0, it means that the left side of the lamp bead is the negative electrode and the right side is the positive electrode; if the polarity calculation result is equal to 0, it means that the recognition is abnormal. This embodiment fuses the dual-path confidence scores and comprehensively determines the positive and negative electrodes through the neural network model, improving the adaptability to complex morphologies.

[0034] According to an embodiment of the present invention, the electrical retest mechanism is specifically: Apply a positive voltage to the lamp bead according to the polarity information, measure the first current information or identify the light-emitting state of the lamp bead by taking a photo image; Judge whether the first current information exceeds a preset current threshold; If so, the first polarity determination information is in the correct state; If not, the first polarity determination information is in the wrong state; Judge whether the lamp bead is in the light-emitting state; If so, the second polarity determination information is in the correct state; If not, the second polarity determination information is in the wrong state; Judge whether the first polarity determination information is the same as the second polarity determination information; If so, set the determination result of the polarity information according to the first polarity determination information; If not, the polarity information is in an error state.

[0035] It should be noted that in this embodiment, the electrically retested the polarity of the identified lamp beads, and cross-validated the polarity accuracy through dual physical signals. In this embodiment, according to the identified lamp bead polarity, it is rotated and then moved into the electrical retest position to apply a forward voltage to the lamp beads. Then, according to the unidirectional conductivity of the lamp bead LED, the first polarity determination information is obtained through the current of the lamp bead; and the second polarity determination information is obtained according to the light-emitting state of the lamp bead; wherein, in this embodiment, the retest result is represented by the first polarity determination information and the second polarity determination information. Finally, combining the first polarity determination information and the second polarity determination information, only when both are determined correctly, the lamp bead polarity identification is considered correct.

[0036] According to an embodiment of the present invention, the method of recording a log according to the polarity feature vector and placing the lamp beads in the feeding channel for waiting for re-detection further includes: Set a first identifier for the lamp beads with incorrect polarity identification; Put the lamp beads into the return chute through a pneumatic valve; According to the first identifier, when it is determined that the same lamp bead returns to the inlet three times continuously, trigger a system alarm and move it into the waste bin.

[0037] It should be noted that this embodiment provides a fault tolerance mechanism. When the lamp bead polarity identification is incorrect, the lamp bead is marked and placed in the return chute and transferred to the inlet for re-detection. If the same lamp bead fails to be identified three times in a row, trigger a system alarm to report the abnormality of the lamp bead; at the same time, transfer the lamp bead to the waste bin.

[0038] It is worth mentioning that it further includes: In response to a preset production efficiency improvement instruction, based on a preset adjustment step, lower the distance ratio threshold; If the distance ratio threshold is lower than a preset distance ratio lower limit, set the distance ratio threshold according to the distance ratio lower limit and feedback an efficiency improvement abnormality; In response to a preset precision priority instruction, based on a preset adjustment step, raise the distance ratio threshold; If the distance ratio threshold is higher than a preset distance ratio upper limit, set the distance ratio threshold according to the distance ratio upper limit and an accuracy improvement abnormality.

[0039] It should be noted that this embodiment also provides an efficiency control mechanism and a precision control mechanism. In response to a production efficiency improvement instruction, the adjustment step is set according to the current distance ratio threshold to control the downward adjustment of the current distance ratio threshold; if the distance ratio threshold is lower than the preset lower limit of the distance ratio, the distance ratio threshold is maintained at the lower limit of the distance ratio, and an efficiency improvement anomaly is fed back to the background. In this embodiment, by lowering the distance ratio threshold, the time for the lamp bead to rotate to the set angle is reduced, thereby achieving the purpose of improving production efficiency. In response to a precision priority instruction, the adjustment step is set according to the current distance ratio threshold to control the upward adjustment of the current distance ratio threshold; if the distance ratio threshold is higher than the preset upper limit of the distance ratio, the distance ratio threshold is maintained at the upper limit of the distance ratio, and a precision improvement anomaly is fed back to the background. In this embodiment, by increasing the distance ratio threshold, the quality of the second image information is improved, thereby improving the accuracy of polarity recognition.

[0040] It is worth mentioning that it further includes: Set the training weight according to the result of the electrical retest; Combine the second image information, the training weight and the optical retest result to obtain a data set; Input the data set into the recognition model for backpropagation training; Adjust the neural network weights of the recognition model based on the training results.

[0041] It should be noted that this embodiment provides a training mechanism for a neural network model. Based on the results of the electrical retest, for the image data whose retest indicates correct polarity, a lower training weight is set; for the image data whose retest indicates incorrect polarity, a higher training weight is set. Combine the second image information, the training weight and the optical retest result to obtain a training data set for backpropagation training of the neural network recognition model. Based on the training results, adjust the neural network weights of the recognition model; that is, by establishing an abnormal data feedback training link, continuously optimize the neural network model to improve the accuracy of image recognition.

[0042] Figure 5 The block diagram of a lamp bead sorting control system based on image recognition according to the present invention is shown.

[0043] As Figure 5 shown, the second aspect of the present invention discloses a lamp bead sorting control system 5 based on image recognition, including a memory 51 and a processor 52. The memory includes a lamp bead sorting control method program based on image recognition. When the lamp bead sorting control method program based on image recognition is executed by the processor, the following steps are implemented: Grab the lamp bead through a vacuum suction nozzle and capture the first image information of the lamp bead; Match the preset distance reference vector according to the first image information to obtain the first distance ratio information; Rotate the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the lamp bead; Generate a polarity feature vector according to the second image information; Based on a preset recognition model, obtain polarity information according to the polarity feature vector; Rotate the lamp bead according to the polarity information and move it to the electrical remeasurement position; Based on a preset electrical remeasurement mechanism, determine whether the polarity information is correct; If so, put the lamp bead into the discharge channel according to the polarity information; If not, record a log according to the polarity feature vector and put the lamp bead into the feed channel to wait for re-detection.

[0044] It should be noted that this embodiment provides a control process for sorting lamp beads based on image recognition. In this embodiment, first, a lamp bead is grabbed by a vacuum suction nozzle and the image of the lamp bead is captured by an industrial camera, which is the first image information. Based on the first image information, after identifying the lamp bead pins and the lamp bead ball head, the pixel distance between the pins and the pixel distance of the ball head diameter are respectively collected, which are used to calculate the ratio of the pin pitch to the ball head diameter in real time to obtain the first distance ratio information. The suction nozzle is rotated by a rotating mechanism, and then the lamp bead is driven to rotate. The posture of the lamp bead is continuously adjusted until the first distance ratio information is greater than the preset distance ratio threshold, and the second image information is captured, indicating that the front orientation of the lamp bead meets the picture shooting requirements for subsequent polarity recognition. By continuously rotating and detecting the distance ratio, the front orientation of the lamp bead is ensured, the image quality during subsequent image recognition is improved, and the problem of randomness of the posture of scattered materials is solved. Secondly, according to the second image information, after preset feature extraction processing, a polarity feature vector is obtained. Based on a preset neural network recognition model, the polarity feature vector is matched and recognized, and the method of synchronously performing pin length analysis and internal structure feature extraction is adopted to identify the polarity of the lamp bead. Then, according to the identified polarity of the lamp bead, the lamp bead is rotated and moved to the electrical remeasurement position for electrical detection. The accuracy of the identified polarity of the lamp bead is cross-verified by using current characteristic detection and luminous state capture, and then a closed-loop error correction mechanism is formed. Finally, the lamp beads with correct polarity verification are put into the discharge channel and arranged neatly in the correct direction and conveyed to the pick-up position of the mounter in sequence; the lamp beads with incorrect polarity verification are sent back to the feed channel to wait for re-detection, and the error recognition status is also recorded through logs and identifiers.

[0045] According to an embodiment of the present invention, it further includes adjusting the rotation deviation angle, specifically: Match a preset distance reference vector according to the first image information, identify the pins and the lamp bead ball head, and obtain the pin pitch information and the lamp bead ball head diameter information; Calculate the ratio of the pin pitch information to the ball head diameter information of the lamp bead to obtain the first distance ratio information; Based on a preset ratio-angle mapping relationship, obtain the rotation deviation angle according to the first distance ratio information; Control the rotation mechanism to adaptively adjust the orientation of the lamp bead according to the deviation angle, and repeatedly execute the first image acquisition and rotation deviation angle calculation operations until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the lamp bead.

[0046] It should be noted that this embodiment provides a logic for continuously adjusting the rotation deviation angle. The rotation deviation angle is the angle between the front surface of the lamp bead and the reference surface of the lamp bead. In this embodiment, during the process of rotating the lamp bead, by calculating the pixel distance ratio of the pin pitch of the lamp bead to the ball head diameter in real time, and then based on a preset ratio-angle mapping relationship, the rotation deviation angle is calculated. Among them, the smaller the rotation deviation angle, the closer the front surface of the lamp bead is to being parallel to the reference surface of the lamp bead, and at this time, the internal features extracted from the captured lamp bead image are clearer. By continuously rotating the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, it is then determined that the second image information captured at this time meets the subsequent polarity recognition requirements. This embodiment ensures the acquisition of a standard front image by controlling the rotation mechanism to continuously adjust until the ratio reaches the standard, and solves the problem of randomness of the posture of the loose materials.

[0047] According to an embodiment of the present invention, the distance ratio threshold is a static distance ratio threshold or a dynamic distance ratio threshold, specifically: Based on the first image information, determine whether the lamp bead specification information is obtained; If so, based on a preset static threshold mapping table, obtain a static threshold according to the lamp bead specification information to set the distance ratio threshold; If not, adopt a dynamic detection method to continuously collect at least three groups of distance ratios during the rotation of the lamp bead to obtain a distance ratio sequence; If the distance ratio sequence meets a preset rule of rising first and then falling, extract the maximum value of the distance ratio sequence to obtain the second distance ratio information; Based on a preset deviation tolerance mechanism, obtain the distance ratio threshold according to the second distance ratio information.

[0048] It should be noted that if the lamp beads are industry standard parts, the pin pitch and ball head diameter thereof need to conform to the reference values set by the industry, and the reference values constitute the lamp bead specification information. By identifying the lamp bead brand, execution specification number, standard product image comparison or manual input, etc., it is determined whether the lamp beads of the current batch are industry standard parts, that is, it is used to judge whether the lamp bead specification information is obtained. If the lamp bead specification information is obtained, the distance ratio threshold is obtained by using the static threshold method; that is, based on the preset static threshold mapping table, the static threshold is queried according to the lamp bead specification and used as the distance ratio threshold. If the lamp bead specification information is not obtained, the dynamic detection method is used to set the distance ratio threshold. The execution process of the dynamic detection method includes continuously rotating the lamp beads and calculating multiple sets of distance ratios in real time to form a distance ratio sequence. Based on methods such as difference comparison and slope fitting, the distance ratio sequence is tested. When it is judged that the elements in the distance ratio sequence have satisfied the rule of rising first and then falling, the maximum value in the distance ratio sequence is extracted as the second distance ratio information. Finally, based on the set deviation tolerance mechanism, the distance ratio threshold is obtained according to the second distance ratio information; for example, 80% of the second distance ratio information is used as the distance ratio threshold.

[0049] According to an embodiment of the present invention, based on the preset recognition model, the polarity information is obtained according to the polarity feature vector, specifically: Based on the recognition model, the pin length information and the first confidence information are obtained; According to the pin length information, the first polarity information is obtained; Based on the recognition model, the narrowing part shape and the expanding part shape of the lamp beads, and the second confidence information are identified; According to the narrowing part shape and the expanding part shape, the second polarity information is obtained; According to the first confidence information and the second confidence information, the first weight and the second weight are obtained; According to the first polarity information, the second polarity information, the first weight and the second weight, the polarity information is obtained.

[0050] It should be noted that this embodiment provides a dual-path fusion polarity recognition mechanism. The pin length analysis and internal structure feature extraction are executed synchronously, and confidence scores are respectively set; the polarity information of the lamp bead is calculated by combining the polarity judgment results of the dual paths and the confidence scores. In this embodiment, first, through a preset neural network recognition model, the pins in the polarity feature vector are recognized and located, and the lengths of two pins are measured to obtain the first polarity information; wherein, the first polarity information is used to judge the polarity of the lamp bead by the lengths of the pins. And, based on information such as the pin length relationship, pin bending condition, and pin fracture condition, the first confidence information is obtained through conversion by the neural network model. Then, through a preset neural network recognition model, the internal structure features in the polarity feature vector are recognized and located, including but not limited to the morphological features of the narrowing part and the expanding part of the lamp bead, to obtain the second polarity information; wherein, the second polarity information is used to judge the polarity of the lamp bead by the internal features. And, based on information such as the rotation deviation angle and the morphological matching degree between the narrowing part and the expanding part, the second confidence information is obtained through conversion by the neural network model. Finally, according to the first confidence information and the second confidence information, the first weight and the second weight are obtained according to a preset weight distribution algorithm. As a real-time method, when the left side of the lamp bead image is the positive electrode and the right side is the negative electrode, the first polarity information and the second polarity information are quantified as 1; when the left side is the negative electrode and the right side is the positive electrode, the first polarity information and the second polarity information are quantified as -1; combining the corresponding first weight and second weight, the polarity calculation result is obtained. If the polarity calculation result is greater than 0, it means that the left side of the lamp bead is the positive electrode and the right side is the negative electrode; if the polarity calculation result is less than 0, it means that the left side of the lamp bead is the negative electrode and the right side is the positive electrode; if the polarity calculation result is equal to 0, it means that the recognition is abnormal. This embodiment fuses the confidence scores of the dual paths and comprehensively determines the positive and negative electrodes through the neural network model, improving the adaptability to complex morphologies.

[0051] According to an embodiment of the present invention, the electrical retest mechanism is specifically as follows: Apply a positive voltage to the lamp bead according to the polarity information, measure the first current information or recognize the light-emitting state of the lamp bead by taking a photo image; Judge whether the first current information exceeds a preset current threshold; If so, the first polarity determination information is in the correct state; If not, the first polarity determination information is in the wrong state; Judge whether the lamp bead is in the light-emitting state; If so, the second polarity determination information is in the correct state; If not, the second polarity determination information is in the wrong state; Judge whether the first polarity determination information is the same as the second polarity determination information; If yes, setting the determination result of the polarity information according to the first polarity determination information; If not, the polarity information is in an error state.

[0052] It should be noted that, in this embodiment, the polarity of the identified lamp bead is electrically retested, and the polarity accuracy is cross-verified by dual physical signals. In this embodiment, according to the identified polarity of the lamp bead, it is rotated and moved into the electrical retest position to apply a forward voltage to the lamp bead. Then, according to the unidirectional conductivity of the lamp bead LED, the first polarity determination information is obtained through the current of the lamp bead; and the second polarity determination information is obtained according to the luminous state of the lamp bead; wherein, in this embodiment, the retest result is represented by the first polarity determination information and the second polarity determination information. Finally, combining the first polarity determination information and the second polarity determination information, only when both are determined correctly, is it determined that the polarity identification of the lamp bead is correct.

[0053] According to an embodiment of the present invention, the method of recording a log according to the polarity characteristic vector and placing the lamp beads into a feeding channel for re-detection further includes: For the lamp beads with incorrect polarity recognition, a first mark is set; Put the lamp beads into the return channel through the pneumatic valve; According to the first identification, when it is determined that the same lamp bead returns to the feed port three times in a row, the system alarm is triggered and the lamp bead is moved into the waste bin.

[0054] It should be noted that this embodiment provides a fault-tolerant mechanism. When the polarity recognition of the lamp bead is wrong, the lamp bead is marked and put into the return channel to be transferred to the feed port for re-testing. If the same lamp bead fails to be recognized three times in a row, the system alarm is triggered and the lamp bead abnormality is reported; at the same time, the lamp bead is moved to the waste bin.

[0055] It is worth mentioning that it also includes: In response to a preset production efficiency improvement instruction, lowering the distance ratio threshold based on a preset adjustment step; If the distance ratio threshold is lower than a preset distance ratio lower limit, the distance ratio threshold is set according to the distance ratio lower limit, and an efficiency improvement abnormality is fed back; In response to a preset accuracy priority instruction, the distance ratio threshold is increased based on a preset adjustment step; If the distance ratio threshold is higher than a preset distance ratio upper limit, the distance ratio threshold is set according to the distance ratio upper limit, and the accuracy is improved abnormally.

[0056] It should be noted that this embodiment also provides an efficiency control mechanism and a precision control mechanism. In response to a production efficiency improvement instruction, the adjustment step is set according to the current distance ratio threshold to control the downward adjustment of the current distance ratio threshold; if the distance ratio threshold is lower than the preset lower limit of the distance ratio, the distance ratio threshold is maintained at the lower limit of the distance ratio, and an efficiency improvement anomaly is fed back to the background. In this embodiment, by reducing the distance ratio threshold, the time for the lamp beads to rotate to the set angle is reduced, thereby achieving the purpose of improving production efficiency. In response to a precision priority instruction, the adjustment step is set according to the current distance ratio threshold to control the upward adjustment of the current distance ratio threshold; if the distance ratio threshold is higher than the preset upper limit of the distance ratio, the distance ratio threshold is maintained at the upper limit of the distance ratio, and a precision improvement anomaly is fed back to the background. In this embodiment, by increasing the distance ratio threshold, the quality of the second image information is improved, thereby improving the accuracy of polarity recognition.

[0057] It is worth mentioning that it further includes: Set the training weight according to the result of the electrical retest; Combine the second image information, the training weight and the optical retest result to obtain a data set; Input the data set into the recognition model for backpropagation training; Based on the training result, adjust the neural network weights of the recognition model.

[0058] It should be noted that this embodiment provides a training mechanism for a neural network model. Based on the result of the electrical retest, a lower training weight is set for the image data whose retest indicates correct polarity; a higher training weight is set for the image data whose retest indicates incorrect polarity. Combine the second image information, the training weight and the optical retest result to obtain a training data set for backpropagation training of the neural network recognition model. Based on the training result, adjust the neural network weights of the recognition model; that is to say, by establishing an abnormal data feedback training link, continuously optimize the neural network model to improve the accuracy of image recognition.

[0059] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for a lamp bead sorting control method based on image recognition. When the program for the lamp bead sorting control method based on image recognition is executed by a processor, the steps of the lamp bead sorting control method based on image recognition as described in any one of the above are implemented.

[0060] In summary, the present invention provides a method, system, and storage medium for controlling the sorting of lamp beads based on image recognition. First, an initial image of the lamp beads is collected by a camera for calculating the distance ratio between the pin pitch and the ball head diameter in real time, and the rotation mechanism is cyclically adjusted until the distance ratio meets the standard. Secondly, the pin length analysis and the internal structure feature recognition are executed synchronously, and the positive and negative poles are comprehensively determined by a neural network model through fusing the confidence scores of the two paths. Then, the current characteristics of the image recognition result are detected and the lighting state is captured, and the polarity accuracy is cross-validated by dual physical signals. Finally, the feeding and discharging states of the lamp beads are controlled based on the polarity judgment result, thereby reducing the production and taping cost.

[0061] If the above functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0062] The foregoing is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for controlling the sorting of lamp beads based on image recognition, characterized in that, The method includes: Grasping the lamp bead by a vacuum suction nozzle and taking the first image information of the lamp bead; Matching a preset distance reference vector according to the first image information to obtain first distance ratio information; Rotating the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, and then taking the second image information of the lamp bead; Generating a polarity feature vector according to the second image information; Based on a preset recognition model, obtaining polarity information according to the polarity feature vector; Rotating the lamp bead according to the polarity information and moving it to the electrical re-test position; Based on a preset electrical re-test mechanism, judging whether the polarity information is correct; If so, putting the lamp bead into the discharge channel according to the polarity information; If not, recording a log according to the polarity feature vector and putting the lamp bead into the feed channel to wait for re-detection.

2. The method for controlling the sorting of lamp beads based on image recognition according to claim 1, wherein It further includes adjusting the rotation deviation angle, specifically: Matching a preset distance reference vector according to the first image information, identifying the pin and the lamp bead ball head to obtain pin pitch information and lamp bead ball head diameter information; Calculating the ratio of the pin pitch information to the lamp bead ball head diameter information to obtain first distance ratio information; Based on a preset ratio-angle mapping relationship, obtaining the rotation deviation angle according to the first distance ratio information; Controlling the rotation mechanism to adaptively adjust the orientation of the lamp bead according to the deviation angle, and circularly executing the first image acquisition and rotation deviation angle calculation operations until the first distance ratio information is greater than a preset distance ratio threshold, and then taking the second image information of the lamp bead.

3. The method for controlling the sorting of lamp beads based on image recognition according to claim 2, characterized in that, The distance ratio threshold is a static distance ratio threshold or a dynamic distance ratio threshold, specifically: Judging whether lamp bead specification information is obtained according to the first image information; If so, based on a preset static threshold mapping table, obtaining a static threshold according to the lamp bead specification information to set the distance ratio threshold; If not, adopting a dynamic detection method to continuously collect at least three groups of distance ratios during the rotation of the lamp bead to obtain a distance ratio sequence; If the distance ratio sequence meets a preset first-rising-then-falling rule, extracting the maximum value of the distance ratio sequence to obtain second distance ratio information; Based on a preset deviation tolerance mechanism, obtaining the distance ratio threshold according to the second distance ratio information.

4. The method for controlling the sorting of lamp beads based on image recognition according to claim 1, wherein, The obtaining of the polarity information based on a preset recognition model according to the polarity feature vector is specifically: Based on the recognition model, obtaining pin length information and first confidence information; Obtaining first polarity information according to the pin length information; Based on the recognition model, identifying the narrowing part shape and widening part shape of the lamp bead, and second confidence information; Obtaining second polarity information according to the narrowing part shape and the widening part shape; Obtaining a first weight and a second weight according to the first confidence information and the second confidence information; Obtaining the polarity information according to the first polarity information, the second polarity information, the first weight and the second weight.

5. The method for controlling the sorting of lamp beads based on image recognition according to claim 1, wherein The electrical re-test mechanism is specifically: Applying a positive voltage to the lamp bead according to the polarity information, measuring first current information or identifying the light-emitting state of the lamp bead by photographing an image; Judging whether the first current information exceeds a preset current threshold; If so, the first polarity determination information is in the correct state; If not, the first polarity determination information is in the wrong state; Determine whether the lamp bead is in the lighting state; If so, the second polarity determination information is in the correct state; If not, the second polarity determination information is in the wrong state; Determine whether the first polarity determination information is the same as the second polarity determination information; If so, set the determination result of the polarity information according to the first polarity determination information; If not, the polarity information is in the wrong state.

6. The method for controlling the sorting of lamp beads based on image recognition according to claim 1, wherein, The step of recording a log according to the polarity feature vector and placing the lamp bead in the feeding channel for re-detection further includes: Set a first identifier for the lamp bead with incorrect polarity recognition; Put the lamp bead into the return channel through a pneumatic valve; According to the first identifier, when it is determined that the same lamp bead returns to the inlet three times in a row, trigger a system alarm and move it to the waste bin.

7. A light bead sorting control system based on image recognition, characterized in that, The system includes a memory and a processor. The memory includes a program for a method of controlling lamp bead sorting based on image recognition. When the program for the method of controlling lamp bead sorting based on image recognition is executed by the processor, the following steps are implemented: Grab the lamp bead through a vacuum suction nozzle and capture the first image information of the lamp bead; Match the first image information with a preset distance reference vector to obtain first distance ratio information; Rotate the lamp bead until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the lamp bead; Generate a polarity feature vector according to the second image information; Based on a preset recognition model, obtain polarity information according to the polarity feature vector; Rotate the lamp bead according to the polarity information and move it to the electrical re-test position; Based on a preset electrical re-test mechanism, determine whether the polarity information is correct; If so, put the lamp bead into the discharge channel according to the polarity information; If not, record a log according to the polarity feature vector and place the lamp bead in the feeding channel for re-detection.

8. An image recognition-based light bead sorting control system according to claim 7, characterized in that It further includes adjusting the rotation deviation angle, specifically: Match the first image information with a preset distance reference vector, identify the pins and the lamp bead ball head, and obtain the pin pitch information and the lamp bead ball head diameter information; Calculate the ratio of the pin pitch information to the lamp bead ball head diameter information to obtain first distance ratio information; Based on a preset ratio-angle mapping relationship, obtain the rotation deviation angle according to the first distance ratio information; Control the rotation mechanism to adaptively adjust the orientation of the lamp bead according to the deviation angle, and repeatedly execute the first image acquisition and rotation deviation angle calculation operations until the first distance ratio information is greater than a preset distance ratio threshold, and then capture the second image information of the lamp bead.

9. The bead sorting control system based on image recognition according to claim 8, characterized in that The distance ratio threshold is a static distance ratio threshold or a dynamic distance ratio threshold, specifically: According to the first image information, determine whether lamp bead specification information is obtained; If so, based on a preset static threshold mapping table, obtain a static threshold according to the lamp bead specification information to set the distance ratio threshold; If not, adopt a dynamic detection method to continuously collect at least three groups of distance ratios during the rotation of the lamp bead to obtain a distance ratio sequence; If the distance ratio sequence meets a preset rule of rising first and then falling, extract the maximum value of the distance ratio sequence to obtain second distance ratio information; Based on a preset deviation tolerance mechanism, obtain the distance ratio threshold according to the second distance ratio information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer-readable storage medium includes a program for a method of controlling bead sorting based on image recognition. When the program for the method of controlling bead sorting based on image recognition is executed by a processor, the steps of the method of controlling bead sorting based on image recognition according to any one of claims 1 to 6 are implemented.

Images