A method and device for directly inserting LED lamp beads based on machine vision

Abstract

The present invention provides a method and device for directly soldering LED lamp beads based on machine vision. The method comprises: collecting a pre-welding image and performing image solder joint recognition to obtain a set of solder joint location coordinates; performing welding path planning based on the set of solder joint location coordinates to obtain welding path information; obtaining first auxiliary information based on historical welding defect information and welding path information of a first welding device; performing lamp bead welding based on the welding path information and the first auxiliary information to obtain a welding image set, performing welding effect detection to obtain first correction information, and obtaining a set of solder joint location coordinate corrections; obtaining adjusted welding path information based on the set of solder joint location coordinate corrections to complete lamp bead welding based on the pre-welding image. This method solves the technical problem that existing directly soldering LED lamp bead welding equipment has weak solder joint recognition capabilities, unstable welding accuracy and quality, and low welding efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method and device for welding direct-insertion LED lamp beads based on machine vision. Background Art

[0002] LED lights (light-emitting diodes) are categorized as either direct-plug or surface-mount (SMD) due to their packaging methods. Direct-plug LED lamps are often used outdoors due to their excellent waterproof performance and superior stability, such as in outdoor displays, decorative light strings, and decorative light signs. The soldering quality and welding technology of direct-plug LED lamps affect the reliability, service life, and market competitiveness of lighting, displays, and other electronic components. Traditional soldering is manual, and human factors significantly impact product quality, resulting in poor product quality stability. With the advancement of welding technology, large-scale automated welding platforms have emerged. However, these platforms are large in scale, and purchasing complete sets of equipment is expensive and costly. This makes it difficult for some smaller businesses to afford these systems, resulting in low soldering efficiency for direct-plug LED lamps. Furthermore, some emerging semi-automated soldering equipment has limited ability to identify the leads of direct-plug LED lamps, resulting in low soldering accuracy.

[0003] The existing technology has technical problems such as weak ability of direct-insert LED lamp bead welding equipment to identify welding sites, unstable welding accuracy and welding quality, resulting in low welding efficiency. Summary of the Invention

[0004] This application provides a machine vision-based in-line LED lamp bead soldering method and device, addressing the technical issues of existing in-line LED lamp bead soldering equipment, such as weak soldering site recognition capabilities, unstable soldering accuracy and quality, and low soldering efficiency. By performing image recognition and edge detection on pre-welding images, the soldering sites are accurately identified, allowing for planning of appropriate soldering paths. Pre-welding trajectory correction is performed, ensuring soldering accuracy during mass production and improving the stability of soldering quality.

[0005] In view of the above problems, the present application provides a method and device for directly inserting LED lamp beads based on machine vision.

[0006] In the first aspect, the present application provides a method for directly plugging LED lamp beads into the soldering machine based on machine vision, wherein the method includes: obtaining a first pre-welding image based on the image acquisition device; performing solder joint identification on the first pre-welding image to obtain a first solder joint positioning coordinate set; performing welding path planning based on the first solder joint positioning coordinate set to obtain first welding path information; obtaining historical welding defect information of the first welding device, and obtaining first auxiliary information based on the historical welding defect information and the first welding path information; performing lamp bead welding based on the first welding path information and the first auxiliary information, and obtaining a first welding image set based on the image acquisition device; performing welding effect detection on the first welding image set to obtain first correction information; obtaining a first solder joint positioning coordinate correction set based on the first correction information; obtaining first adjustment welding path information based on the first solder joint positioning coordinate correction set to complete the lamp bead welding of the first pre-welding image.

[0007] On the other hand, the present application provides a direct-insert LED lamp bead welding device based on machine vision, wherein the device includes: a first obtaining unit, the first obtaining unit is used to obtain a first pre-welding image based on an image acquisition device; a second obtaining unit, the second obtaining unit is used to perform solder joint recognition on the first pre-welding image to obtain a first solder joint positioning coordinate set; a third obtaining unit, the third obtaining unit is used to perform welding path planning based on the first solder joint positioning coordinate set to obtain first welding path information; a fourth obtaining unit, the fourth obtaining unit is used to obtain historical welding defect information of the first welding device, and based on the historical welding defect information and the first welding path path information to obtain first auxiliary information; a fifth obtaining unit, the fifth obtaining unit being used to perform lamp bead welding based on the first welding path information and the first auxiliary information, and obtain a first welding image set according to the image acquisition device; a sixth obtaining unit, the sixth obtaining unit being used to perform welding effect detection on the first welding image set to obtain first correction information; a seventh obtaining unit, the seventh obtaining unit being used to obtain a first welding point positioning coordinate correction set based on the first correction information; a first execution unit, the first execution unit being used to obtain first adjusted welding path information based on the first welding point positioning coordinate correction set, and complete the lamp bead welding of the first pre-welding image.

[0008] In a third aspect, the present application provides a direct-insert LED lamp bead welding system based on machine vision, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the program.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0010] Due to the adoption of a technical solution of capturing a first pre-welding image based on an image acquisition device, and performing welding point identification on the first pre-welding image to obtain a first welding point positioning coordinate set; performing welding path planning based on the first welding point positioning coordinate set to obtain first welding path information; obtaining first auxiliary information based on the historical welding defect information of the first welding device and the first welding path information; performing lamp bead welding based on the first welding path information and the first auxiliary information, and capturing images again based on the image acquisition device to obtain a first welding image set; performing welding effect detection on the first welding image set to obtain first correction information; obtaining a first welding point positioning coordinate correction set based on the first correction information; obtaining first adjustment welding path information based on the first welding point positioning coordinate correction set, and completing the lamp bead welding of the first pre-welding image, the present application provides a direct-insertion LED lamp bead welding method and device based on machine vision, which achieves the technical effect of accurately identifying the welding site by performing image recognition and edge detection on the pre-welding image, thereby planning a suitable welding path, and performing trajectory correction through pre-welding, thereby ensuring welding accuracy during mass production and improving the stability of welding quality.

[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a flow chart of a method for directly inserting LED lamp beads based on machine vision according to an embodiment of the present application;

[0013] Figure 2 This is a flow chart of obtaining the first soldering point positioning coordinate set in a method for directly inserting LED lamp beads based on machine vision according to an embodiment of the present application;

[0014] Figure 3 This is a schematic diagram of a process for performing similarity analysis on a second pre-welding image and a first pre-welding image in a method for directly inserting LED lamp beads based on machine vision according to an embodiment of the present application;

[0015] Figure 4 This is a structural diagram of a direct-insertion LED lamp bead welding device based on machine vision according to an embodiment of the present application;

[0016] Figure 5 This is a schematic diagram of the structure of an exemplary electronic device according to an embodiment of the present application.

[0017] Explanation of the accompanying drawings: first obtaining unit 11, second obtaining unit 12, third obtaining unit 13, fourth obtaining unit 14, fifth obtaining unit 15, sixth obtaining unit 16, seventh obtaining unit 17, first execution unit 18, electronic device 300, memory 301, processor 302, communication interface 303, bus architecture 304. DETAILED DESCRIPTION

[0018] This application provides a machine vision-based in-line LED lamp bead soldering method and device, addressing the technical issues of existing in-line LED lamp bead soldering equipment, such as weak soldering site recognition capabilities, unstable soldering accuracy and quality, and low soldering efficiency. By performing image recognition and edge detection on pre-welding images, the soldering sites are accurately identified, allowing for planning of appropriate soldering paths. Pre-welding trajectory correction is performed, ensuring soldering accuracy during mass production and improving the stability of soldering quality.

[0019] The welding quality and welding technology of plug-in LED lamp beads affect the product reliability, service life and market competitiveness of lighting, display screens and other electronic components. The traditional form of welding is manual welding, and human factors have a significant impact on product quality, resulting in poor product quality stability. With the development of welding technology, large-scale machine automated welding platforms have emerged, but these welding platforms are large in scale, and purchasing complete sets of equipment is expensive and costly. Some small-scale enterprises find it difficult to afford, resulting in low efficiency in the welding of plug-in LED lamp beads. Some emerging semi-automatic welding equipment has a weak ability to identify the lamp bead leads during the soldering process of plug-in LED lamp beads, and the welding accuracy is not high. This leads to technical problems such as the weak ability of the plug-in LED lamp bead welding equipment to identify the welding site, unstable welding accuracy and welding quality, and low welding efficiency.

[0020] In response to the above technical problems, the overall idea of ​​the technical solution provided by this application is as follows:

[0021] The present application provides a method for directly inserting LED lamp beads and welding based on machine vision, wherein the method includes: acquiring a first pre-welding image based on an image acquisition device, and performing solder joint identification on the first pre-welding image to obtain a first solder joint positioning coordinate set; performing welding path planning based on the first solder joint positioning coordinate set to obtain first welding path information; obtaining first auxiliary information based on historical welding defect information of the first welding device and the first welding path information; performing lamp bead welding based on the first welding path information and the first auxiliary information, and acquiring images again based on the image acquisition device to obtain a first welding image set; performing welding effect detection on the first welding image set to obtain first correction information; obtaining a first solder joint positioning coordinate correction set based on the first correction information; obtaining first adjustment welding path information based on the first solder joint positioning coordinate correction set to complete the lamp bead welding of the first pre-welding image.

[0022] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically introduced in conjunction with the drawings in the specification.

[0023] Example 1

[0024] like Figure 1 As shown, an embodiment of the present application provides a method for directly inserting LED lamp beads and soldering them based on machine vision, wherein the method is applied to a directly inserting LED lamp bead soldering device, the device being communicatively connected to an image acquisition device, and the method comprising:

[0025] Step S100: obtaining a first pre-welding image based on the image acquisition device;

[0026] Step S200: performing welding point recognition on the first pre-welding image to obtain a first welding point location coordinate set;

[0027] Specifically, the image acquisition device is integrated into the welding device used. In order to meet different welding tasks, different numbers of image acquisition devices can be integrated according to the scale of the enterprise and the welding task. If the enterprise is small and the daily welding work is less difficult, very few welding devices will integrate more image acquisition devices (for example, integrating 3 or more image acquisition devices), and most welding devices will only need to integrate 1 to 2 image acquisition devices. The image acquisition devices include but are not limited to CCD high-speed cameras, CMOS high-speed cameras, etc. CCD high-speed cameras and CMOS high-speed cameras are industrial cameras with high image stability, high transmission capacity and high anti-interference ability, and are suitable for image acquisition of automated equipment.

[0028] Before performing direct-insert LED lamp bead welding, the lamp beads to be welded have been inserted into the corresponding positions, and based on the image acquisition device, the image of the target to be welded is captured to obtain the first pre-welding image. Image edge detection in image recognition technology is used to obtain the outlines of the positive and negative leads of the LED lamp bead from the detection results. Based on the first pre-welding image, a two-dimensional coordinate system is constructed (the origin of the two-dimensional coordinate system is set according to corporate practices, and can be set at the center of the image, the upper left corner, the lower left corner of the image, etc., and is not specifically limited here), so as to obtain the first solder point positioning coordinate set based on the two-dimensional coordinate system, that is, the coordinate set of all solder points identified for the first time based on machine vision. Capturing pre-welding images through machine vision and obtaining solder point positioning sets through edge detection can provide a data basis for machine welding path design and automatic welding.

[0029] Step S300: performing welding path planning according to the first welding point positioning coordinate set to obtain first welding path information;

[0030] Furthermore, the welding path planning is performed based on the first welding point positioning coordinate set to obtain the first welding path information. In the embodiment of the present application, step S300 further includes:

[0031] Step S310: obtaining the movement mode information of the robot arm of the first welding device;

[0032] Step S320: obtaining a welding path set based on the first welding point positioning coordinate set and the motion mode information;

[0033] Step S330: performing path length analysis on the welding path set to obtain shortest welding path information, and using the shortest welding path information as the first welding path information.

[0034] Specifically, the complexity of the welding task determines the complexity of the welding path. For example, if the welding task involves welding light signs with different images, the image complexity affects the complexity of the welding path. To improve welding speed and efficiency, welding path planning is required based on the first weld point location coordinate set. Specifically, the path length is designed based on the motion of the welding device and the path distance connecting the location coordinates. The first welding device is capable of completing the welding task and includes a robotic arm for moving the welding device. Motion information of the robotic arm of the first welding device is obtained. This motion information includes, but is not limited to, radial motion and horizontal motion. The welding path set includes all possible paths to complete the welding task and is obtained by simulating the paths according to the motion of the robotic arm based on the first weld point location coordinate set. Path length analysis is performed on the paths in the welding path set. Specifically, the corresponding path coordinate information is obtained from the paths, and the distance between the coordinate points is calculated to determine the length of each path. The path information with the shortest path length is used as the first welding path information. The first welding path information includes the welding path information.

[0035] Step S400: obtaining historical welding defect information of a first welding device, and obtaining first auxiliary information based on the historical welding defect information and the first welding path information;

[0036] Furthermore, the obtaining of historical welding defect information of the first welding device and obtaining first auxiliary information based on the historical welding defect information and the first welding path information may further include:

[0037] Step S410: Analyzing the problem of the first welding device itself based on historical welding defect information of the first welding device to obtain a first analysis result;

[0038] Step S420: performing a device debugging according to the first analysis result to obtain a first welding device debugging result;

[0039] Step S430: performing pre-welding based on the first welding device debugging result and the first welding path information to obtain a pre-welding result;

[0040] Step S440: performing secondary device debugging according to the pre-welding result to obtain the first auxiliary information.

[0041] Specifically, the historical welding defect information of the first welding device is data on welding product quality issues encountered during the first welding device's usage history, including quality issues caused by device aging and malfunction, as well as product quality issues caused by manual errors in previous processes. Based on this historical welding defect information, the first welding device is troubleshooted to determine whether maintenance is required, whether the machine shaft is loose, and whether the machine's service life meets the required standards. A first analysis result is obtained, and the first welding device is debugged in real time based on the first analysis result to obtain a first welding device debugging result. The first welding device debugging result includes inspection and maintenance data for the first welding device. For example, if the first analysis result indicates a loose machine shaft, the first welding device debugging result includes a record of tightening the machine shaft.

[0042] After the primary device debugging, pre-welding is performed based on the debugging results of the first welding device and the first welding path. This pre-welding test and troubleshooting allows for troubleshooting of machine operational issues before large-scale welding. Pre-welding results are obtained, including the welding program debugging, the operational status of each axis of the welding machine, and the effectiveness of the pre-welding. Based on these pre-welding results, the first welding device is then secondary debugged. The debugged parameters serve as the first auxiliary information to assist in the LED soldering process. Performing both primary and secondary device debugging on the first welding device can effectively improve production line efficiency and stabilize welding quality.

[0043] Step S500: performing lamp bead welding based on the first welding path information and the first auxiliary information, and obtaining a first welding image set according to the image acquisition device;

[0044] Step S600: performing welding effect detection on the first welding image set to obtain first correction information;

[0045] Specifically, to improve welding accuracy, a smaller number of LED welding passes are initially performed based on the first welding path information and the first auxiliary information. The number of passes can be set based on the complexity of the welding task. For example, for Company A, welding task a is relatively complex. To further refine the welding coordinates, Company A's technicians may first perform 10 to 20 welds.

[0046] Before and after each LED soldering session, an image acquisition device captures images to produce a first set of welding images. Based on welding quality inspection standards, image recognition technology is used to detect welding anomalies within the image set, such as offset welding, bubbles, and undercuts. This detected welding anomaly information is sent to the welder, who generates first correction information based on manual verification. Post-weld quality is inspected using image feature extraction technology, combined with manual inspection to generate accurate correction information. This compensates for welding errors introduced by machine vision, thereby reducing welder workload.

[0047] Step S700: obtaining a first welding point positioning coordinate correction set based on the first correction information;

[0048] Step S800: Based on the first welding point positioning coordinate correction set, first adjusted welding path information is obtained to complete the lamp bead welding of the first pre-welding image.

[0049] Specifically, based on the first correction information, the welder's correction result for the first weld point location coordinates is obtained, namely, a correction set of the first weld point location coordinates is obtained. Based on the corrections to the weld point coordinates, an adjusted welding path is obtained. Lamp-bead welding of the first pre-welding image is performed based on the first adjusted welding path information. Through actual verification and adjustment of the first welding path information, more accurate weld point location coordinates can be obtained, thereby obtaining more accurate welding path information, improving welding stability, reliability, and accuracy.

[0050] Further, such as Figure 2 As shown, the welding point recognition is performed on the first pre-welding image to obtain a first welding point positioning coordinate set. In this embodiment of the application, step S200 further includes:

[0051] Step S210: performing image grayscale transformation on the first pre-welding image to obtain a first grayscale image;

[0052] Step S220: After binarizing the first grayscale image, edge detection is performed using a Canny operator to obtain a first edge detection result;

[0053] Step S230: using the Otsu algorithm to perform adaptive threshold selection to obtain a first hysteresis threshold and a second hysteresis threshold;

[0054] Step S240: performing edge detection on the first edge detection result according to the first hysteresis threshold and the second hysteresis threshold to obtain a second edge detection result;

[0055] Step S250: constructing a first rectangular coordinate system;

[0056] Step S260: performing welding point identification according to the first rectangular coordinate system and the second edge detection result to obtain the first welding point location coordinate set.

[0057] Specifically, to accurately identify the soldering locations of in-line LED lamp beads, edge detection is performed on the first pre-welding image. First, the first pre-welding image captured by the image acquisition device is subjected to image grayscale conversion, converting the color image into a grayscale image to obtain a first grayscale image. Image grayscale conversion can be performed using any of the following methods: component method, maximum method, average method, or weighted average method. The first grayscale image is then binarized using existing image preprocessing techniques. Edge detection is further performed using the Canny operator, which is essentially a filtering algorithm. After smoothing the binarized image using a Gaussian filter, the gradient amplitude and direction of the de-noised image are calculated. Non-maximum suppression is then performed on the gradient amplitude. Edges are then detected and connected using a dual-threshold method to obtain the first edge detection result. The specific calculation formula and process are not detailed here. The threshold selection in the dual-threshold method is often manually set and has poor adaptability. Therefore, the first edge detection result is further tested. The Otsu algorithm is used for adaptive threshold selection, aiming to select a segmentation threshold that maximizes target-background separation and inter-class difference.

[0058] The Otsu algorithm is used to address issues in Canny adaptive edge detection. The maximum inter-class variance method is used to find the optimal segmentation threshold. The optimal segmentation threshold is considered to be found when the variance is maximized. The optimal segmentation threshold is the first hysteresis threshold, which is a high threshold and the second hysteresis threshold is a low threshold. The low threshold is set to 0.3 to 0.5 times the high threshold and can be selected based on the actual welding task. Edge detection is performed on the first edge detection result based on the first and second hysteresis thresholds to obtain the second edge detection result. Based on the Canny edge detection algorithm, the Otsu algorithm is used for adaptive threshold selection, achieving the technical effect of improving the positioning accuracy and robustness of edge detection. The first rectangular coordinate system is constructed based on the first pre-welding image. The first rectangular coordinate system is the two-dimensional coordinate system mentioned above. The coordinate information of the corresponding weld points is obtained in the first rectangular coordinate system based on the second edge detection results. Ultimately, an accurate set of first weld point positioning coordinates is obtained.

[0059] Furthermore, the welding effect detection is performed on the first welding image set to obtain first correction information. In the embodiment of the present application, step S600 further includes:

[0060] Step S610: obtaining a welding abnormality feature set;

[0061] Step S620: performing feature extraction on the first welding image set according to the welding abnormality feature set to obtain a first feature extraction result;

[0062] Step S630: obtaining welding inspection standard information;

[0063] Step S640: Obtain the first correction information according to the first feature extraction result and the welding detection standard information.

[0064] Specifically, a welding anomaly feature set is obtained based on the welding task type and historical welding anomaly data. The welding anomaly feature set includes features such as offset welding, bubbles, and undercuts. The first welding image set is captured by an image acquisition device before and after each test welding of the lamp bead. Feature extraction is performed on the first welding image using the welding anomaly feature set to obtain a first feature extraction result.

[0065] Obtain welding inspection standard information, including abnormality severity determination criteria and abnormality identification criteria. Manual verification is performed based on the extracted first feature extraction results and the welding inspection standard information to obtain the first correction information. The test welding results are then tested for welding effectiveness using feature extraction to further correct the welding path planned by machine vision recognition.

[0066] Further, such as Figure 3 As shown, step S800 in this embodiment of the application further includes:

[0067] Step S810: Obtaining a first preset similarity coefficient;

[0068] Step S820: obtaining a second pre-welding image based on the image acquisition device;

[0069] Step S830: performing similarity analysis on the second pre-welding image and the first pre-welding image to obtain a first similarity coefficient;

[0070] Step S840: If the first similarity coefficient is higher than the first preset similarity coefficient, performing image segmentation on the second pre-welding image to obtain identical image regions and different image regions;

[0071] Step S850: performing lamp bead welding in the same image area based on the first adjusted welding path information.

[0072] Furthermore, step S850 in this embodiment of the present application further includes:

[0073] Step S851: obtaining the final welding point positioning coordinates of the same area image according to the first adjusted welding path information;

[0074] Step S852: Based on the final solder joint positioning coordinates, perform path planning on the differential image region to obtain the second welding path information;

[0075] Step S853: According to the second welding path information, perform solder joint welding on the differential image region.

[0076] Specifically, after welding the first pre-welding image, if mass-producing similar images, welding of similar images can be performed based on the first adjusted welding path information of the first pre-welding image. Preset the first preset similarity coefficient, and obtain the second pre-welding image according to the image acquisition device. The second pre-welding image is any pre-welding image information other than the first pre-welding image. Perform similarity analysis on the second pre-welding image and the first pre-welding image to obtain the first similarity coefficient. Exemplarily, if the first pre-welding image is the standard character "Hu Hu Sheng Wei", and the second pre-welding image is the standard character "Hu Hu Sheng Feng", there is similarity between the first pre-welding image and the second pre-welding image. If the first similarity coefficient is higher than the first preset similarity coefficient, perform image segmentation on the second pre-welding image to obtain the same image region and the differential image region. In the above example, the same image region is the character "Hu Hu Sheng", and the differential image regions are the characters "Wei" and "Feng".

[0077] For the same image region, use the first adjusted welding path information for welding. After welding in the same image region is completed, obtain the final solder joint positioning coordinates. To ensure shortening the welding time interval, starting from the final solder joint positioning coordinates, combine the image information of the differential image region to perform path planning on the differential image region. For path planning, it is necessary to first perform edge detection on the image information of the differential image region, obtain the solder joint positioning coordinates, then perform path length analysis, and finally obtain the most suitable path, that is, the second welding path information. Perform solder joint welding according to the second welding path information. It achieves the technical effects of rapid path planning and processing of similar images and improving the welding efficiency of enterprises.

[0078] In summary, a method and device for soldering through-hole LED lamp beads based on machine vision provided by an embodiment of the present application have the following technical effects:

[0079] 1. Due to the adoption of a technical solution of capturing a first pre-welding image based on an image acquisition device and performing weld point recognition on the first pre-welding image to obtain a first weld point positioning coordinate set; performing welding path planning based on the first weld point positioning coordinate set to obtain first welding path information; obtaining first auxiliary information based on the historical welding defect information of the first welding device and the first welding path information; performing lamp bead welding based on the first welding path information and the first auxiliary information, and capturing images again according to the image acquisition device to obtain a first welding image set; performing welding effect detection on the first welding image set to obtain first correction information; obtaining a first weld point positioning coordinate correction set based on the first correction information; obtaining first adjusted welding path information based on the first weld point positioning coordinate correction set, and completing lamp bead welding of the first pre-welding image, the embodiment of the present application provides a direct-insertion LED lamp bead welding method and device based on machine vision, which achieves the technical effect of accurately identifying welding sites by performing image recognition and edge detection on pre-welding images, thereby planning a suitable welding path, and performing trajectory correction through pre-welding, thereby ensuring welding accuracy during mass production and improving the stability of welding quality.

[0080] 2. By adopting the method of similarity analysis of different pre-welding images, the images that meet the similarity are divided and the welding path planning is carried out quickly, thereby achieving the technical effect of fast path planning for similar images and fast welding, improving the welding accuracy of similar images while ensuring the welding speed, and improving the welding efficiency.

[0081] Example 2

[0082] Based on the same inventive concept as the above-mentioned embodiment of a method for directly inserting LED lamp beads based on machine vision, as Figure 4 As shown, the embodiment of the present application provides a direct-insertion LED lamp bead welding device based on machine vision, wherein the device includes:

[0083] A first obtaining unit 11, wherein the first obtaining unit 11 is configured to obtain a first pre-welding image based on an image acquisition device;

[0084] a second obtaining unit 12, configured to perform welding point recognition on the first pre-welding image to obtain a first welding point positioning coordinate set;

[0085] a third obtaining unit 13, configured to perform welding path planning according to the first welding point positioning coordinate set to obtain first welding path information;

[0086] a fourth obtaining unit 14 configured to obtain historical welding defect information of the first welding device, and obtain first auxiliary information based on the historical welding defect information and the first welding path information;

[0087] a fifth obtaining unit 15, configured to perform lamp bead welding based on the first welding path information and the first auxiliary information, and obtain a first welding image set according to the image acquisition device;

[0088] a sixth obtaining unit 16, configured to perform welding effect detection on the first welding image set to obtain first correction information;

[0089] a seventh obtaining unit 17, the seventh obtaining unit 17 being configured to obtain a first welding point positioning coordinate correction set based on the first correction information;

[0090] The first execution unit 18 is configured to obtain first adjusted welding path information based on the first welding point positioning coordinate correction set, and complete the lamp bead welding of the first pre-welding image.

[0091] Furthermore, the device comprises:

[0092] an eighth obtaining unit, configured to perform image grayscale transformation on the first pre-welding image to obtain a first grayscale image;

[0093] a ninth obtaining unit configured to perform edge detection using a Canny operator after binarizing the first grayscale image to obtain a first edge detection result;

[0094] a tenth obtaining unit, configured to perform adaptive threshold selection using an Otsu algorithm to obtain a first hysteresis threshold and a second hysteresis threshold;

[0095] an eleventh obtaining unit, configured to perform edge detection on the first edge detection result according to the first hysteresis threshold and the second hysteresis threshold to obtain a second edge detection result;

[0096] a first construction unit, wherein the first construction unit is used to construct a first rectangular coordinate system;

[0097] A twelfth obtaining unit is configured to perform welding point identification according to the first rectangular coordinate system and the second edge detection result, and obtain the first welding point positioning coordinate set.

[0098] Furthermore, the device comprises:

[0099] a thirteenth obtaining unit, configured to obtain movement mode information of the robot arm of the first welding device;

[0100] a fourteenth obtaining unit, configured to obtain a welding path set based on the first welding point positioning coordinate set and the movement mode information;

[0101] A fifteenth obtaining unit is configured to perform path length analysis on the welding path set to obtain shortest welding path information, and use the shortest welding path information as the first welding path information.

[0102] Furthermore, the device comprises:

[0103] a sixteenth obtaining unit, configured to analyze a problem of the first welding device according to historical welding defect information of the first welding device to obtain a first analysis result;

[0104] a seventeenth obtaining unit, configured to perform a device debugging according to the first analysis result to obtain a first welding device debugging result;

[0105] an eighteenth obtaining unit, configured to perform pre-welding based on the first welding device debugging result and the first welding path information to obtain a pre-welding result;

[0106] A nineteenth obtaining unit is configured to perform secondary device debugging according to the pre-welding result to obtain the first auxiliary information.

[0107] Furthermore, the device comprises:

[0108] a twentieth obtaining unit, the twentieth obtaining unit being used to obtain a welding abnormality feature set;

[0109] a twenty-first obtaining unit, configured to perform feature extraction on the first welding image set according to the welding abnormality feature set to obtain a first feature extraction result;

[0110] A twenty-second obtaining unit, the twenty-second obtaining unit being used to obtain welding inspection standard information;

[0111] The twenty-third obtaining unit is used to obtain the first correction information according to the first feature extraction result and the welding detection standard information.

[0112] Furthermore, the device comprises:

[0113] A twenty-fourth obtaining unit, the twenty-fourth obtaining unit being configured to obtain a first preset similarity coefficient;

[0114] a twenty-fifth obtaining unit, configured to obtain a second pre-welding image based on the image acquisition device;

[0115] a twenty-sixth obtaining unit, configured to perform a similarity analysis on the second pre-welding image and the first pre-welding image to obtain a first similarity coefficient;

[0116] a twenty-seventh obtaining unit, configured to perform image segmentation on the second pre-welding image to obtain identical image regions and different image regions if the first similarity coefficient is higher than the first preset similarity coefficient;

[0117] A second execution unit is configured to perform lamp bead welding in the same image area based on the first adjusted welding path information.

[0118] Furthermore, the device comprises:

[0119] a twenty-eighth obtaining unit, configured to obtain final welding point positioning coordinates of the same area image according to the first adjusted welding path information;

[0120] a twenty-ninth obtaining unit, configured to perform path planning on the difference image area based on the final welding point positioning coordinates to obtain second welding path information;

[0121] A third execution unit is configured to perform lamp bead welding in the difference image area according to the second welding path information.

[0122] Exemplary electronic devices

[0123] Reference below Figure 5 The electronic device of the embodiment of the present application is described. Based on the same inventive concept as the in-line LED lamp bead soldering method based on machine vision in the aforementioned embodiment, the embodiment of the present application further provides a in-line LED lamp bead soldering system based on machine vision, comprising: a processor, the processor being coupled to a memory, the memory being used to store a program, and when the program is executed by the processor, the system executes any one of the methods described in the first aspect.

[0124] The electronic device 300 includes: a processor 302, a communication interface 303, and a memory 301. Optionally, the electronic device 300 may further include a bus architecture 304. The communication interface 303, the processor 302, and the memory 301 may be interconnected via the bus architecture 304; the bus architecture 304 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus architecture 304 may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0125] The processor 302 may be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the present application.

[0126] The communication interface 303 uses any transceiver-like system for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), wired access network, etc.

[0127] The memory 301 can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory can exist independently and be connected to the processor via the bus architecture 304. The memory can also be integrated with the processor.

[0128] The memory 301 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 302. The processor 302 is used to execute the computer-executable instructions stored in the memory 301, thereby implementing a machine vision-based direct-insert LED lamp bead soldering method provided in the above embodiment of the present application.

[0129] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0130] An embodiment of the present application provides a method for directly inserting LED lamp beads and welding based on machine vision, wherein the method includes: acquiring a first pre-welding image based on an image acquisition device, and performing solder joint identification on the first pre-welding image to obtain a first solder joint positioning coordinate set; performing welding path planning based on the first solder joint positioning coordinate set to obtain first welding path information; obtaining first auxiliary information based on historical welding defect information of the first welding device and the first welding path information; performing lamp bead welding based on the first welding path information and the first auxiliary information, and acquiring images again based on the image acquisition device to obtain a first welding image set; performing welding effect detection on the first welding image set to obtain first correction information; obtaining a first solder joint positioning coordinate correction set based on the first correction information; obtaining first adjustment welding path information based on the first solder joint positioning coordinate correction set to complete the lamp bead welding of the first pre-welding image.

[0131] Those skilled in the art will understand that the various numerical numbers such as the first and second involved in this application are only for the convenience of description, and are not used to limit the scope of the embodiments of the present application, nor do they represent the order of precedence. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one" refers to one or more. At least two refers to two or more. "At least one", "any one" or similar expressions refer to any combination of these items, including any combination of single items (individuals) or plural items (individuals). For example, at least one item (individual, kind) of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0132] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0133] The various illustrative logic units and circuits described in the embodiments of the present application can be implemented or operated by the design of a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic system, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, alternatively, the general-purpose processor can also be any traditional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing systems, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration to implement.

[0134] The steps of the method or algorithm described in the embodiments of the present application can be directly embedded in hardware, software units executed by a processor, or a combination of the two. The software unit can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium in the art. For example, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be provided in an ASIC, and the ASIC can be provided in a terminal. Optionally, the processor and the storage medium can also be provided in different components in the terminal. These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0135] Although the present application has been described with reference to specific features and embodiments thereof, it will be apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and drawings are merely illustrative of the present application as defined herein and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, various modifications and variations may be made to the present application by those skilled in the art without departing from the scope of the present application. Thus, the present application is intended to include such modifications and variations as would fall within the scope of the present application and its equivalents.

Claims

1. A method for directly inserting LED lamp beads based on machine vision, characterized in that: The method is applied to a direct-insertion LED lamp bead welding device, the device is communicatively connected to an image acquisition device, and the method includes: Based on the image acquisition device, obtaining a first pre-welding image; Performing welding point recognition on the first pre-welding image to obtain a first welding point positioning coordinate set; Performing welding path planning according to the first welding point positioning coordinate set to obtain first welding path information; Obtaining historical welding defect information of a first welding device, and obtaining first auxiliary information based on the historical welding defect information and the first welding path information; Performing lamp bead welding based on the first welding path information and the first auxiliary information, and obtaining a first welding image set according to the image acquisition device; Performing welding effect detection on the first welding image set to obtain first correction information includes: Obtaining a welding abnormality feature set; performing feature extraction on the first welding image set according to the welding abnormality feature set to obtain a first feature extraction result; Obtain information on welding inspection standards; Obtaining the first correction information according to the first feature extraction result and the welding detection standard information; Based on the first correction information, obtaining a first welding point positioning coordinate correction set; Based on the first welding point positioning coordinate correction set, first adjusted welding path information is obtained to complete the lamp bead welding of the first pre-welding image.

2. The method according to claim 1, wherein The method of performing welding point recognition on the first pre-welding image to obtain a first welding point positioning coordinate set includes: Performing image grayscale transformation on the first pre-welding image to obtain a first grayscale image; After binarizing the first grayscale image, edge detection is performed using a Canny operator to obtain a first edge detection result; Using the Otsu algorithm to perform adaptive threshold selection, a first hysteresis threshold and a second hysteresis threshold are obtained; Performing edge detection on the first edge detection result according to the first hysteresis threshold and the second hysteresis threshold to obtain a second edge detection result; Construct the first rectangular coordinate system; Welding point identification is performed based on the first rectangular coordinate system and the second edge detection result to obtain the first welding point positioning coordinate set.

3. The method according to claim 1, wherein The method of performing welding path planning based on the first welding point positioning coordinate set to obtain first welding path information includes: Obtaining movement information of the robot arm of the first welding device; Obtaining a welding path set based on the first welding point positioning coordinate set and the motion mode information; Performing path length analysis on the welding path set to obtain shortest welding path information, and using the shortest welding path information as the first welding path information.

4. The method according to claim 1, wherein The method of obtaining historical welding defect information of the first welding device and obtaining first auxiliary information based on the historical welding defect information and the first welding path information includes: Analyzing the problem of the first welding device itself based on historical welding defect information of the first welding device to obtain a first analysis result; Performing a device debugging according to the first analysis result to obtain a first welding device debugging result; Perform pre-welding based on the first welding device debugging result and the first welding path information to obtain a pre-welding result; Secondary device debugging is performed according to the pre-welding result to obtain the first auxiliary information.

5. The method according to claim 1, wherein The method comprises: Obtaining a first preset similarity coefficient; Based on the image acquisition device, obtaining a second pre-welding image; performing a similarity analysis on the second pre-welding image and the first pre-welding image to obtain a first similarity coefficient; If the first similarity coefficient is higher than the first preset similarity coefficient, performing image segmentation on the second pre-welding image to obtain identical image areas and different image areas; Lamp bead welding is performed in the same image area based on the first adjusted welding path information.

6. The method according to claim 5, wherein The method comprises: Obtaining final welding point positioning coordinates of the same image area according to the first adjusted welding path information; Performing path planning on the difference image area based on the final welding point positioning coordinates to obtain second welding path information; The lamp beads in the difference image area are welded according to the second welding path information.

7. A direct-insert LED lamp bead welding device based on machine vision, characterized in that: The device is used to implement the method according to any one of claims 1 to 6, and the device comprises: a first obtaining unit, configured to obtain a first pre-welding image based on an image acquisition device; a second obtaining unit, configured to perform welding point recognition on the first pre-welding image to obtain a first welding point positioning coordinate set; a third obtaining unit, configured to perform welding path planning according to the first welding point positioning coordinate set to obtain first welding path information; a fourth obtaining unit, configured to obtain historical welding defect information of the first welding device, and obtain first auxiliary information based on the historical welding defect information and the first welding path information; a fifth obtaining unit, configured to perform lamp bead welding based on the first welding path information and the first auxiliary information, and obtain a first welding image set according to the image acquisition device; a sixth obtaining unit, configured to perform welding effect detection on the first welding image set to obtain first correction information; a seventh obtaining unit, configured to obtain a first welding point positioning coordinate correction set based on the first correction information; The first execution unit is used to obtain first adjusted welding path information based on the first welding point positioning coordinate correction set, and complete the lamp bead welding of the first pre-welding image.

8. A direct-insert LED lamp bead welding system based on machine vision, comprising: A processor is coupled to a memory, the memory is used to store a program, and when the program is executed by the processor, the system is configured to execute the method according to any one of claims 1 to 6.

Images