Method, apparatus and electronic device for detecting a solder print

Abstract

Embodiments of the present application provide a method and device for detecting a welding mark and an electronic device, which can improve the accuracy of welding mark classification. The method for detecting a welding mark comprises: obtaining a first picture, the first picture comprising a welding mark; and obtaining a classification type of the welding mark according to the first picture and a welding mark classification model, wherein the welding mark classification model is used to classify the welding mark according to a feature of the welding mark.

Description

Technical Field

[0001] This application relates to the field of battery technology, and in particular to a method, apparatus and electronic device for detecting solder marks. Background Technology

[0002] With the development of the times, electric vehicles, due to their advantages such as high environmental friendliness, low noise, and low operating costs, have a huge market prospect and can effectively promote energy conservation and emission reduction, which is beneficial to social development and progress. For electric vehicles and related fields, battery technology is a crucial factor in their development.

[0003] In each stage of battery production, various components of the battery need to be tested to ensure that the battery is qualified at every stage, thus guaranteeing that the final product meets quality requirements. Ensuring the accuracy of battery testing remains a pressing issue that needs to be addressed. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for detecting solder marks, which can avoid background interference during the solder mark detection process, improve the accuracy of solder mark classification, and thereby reduce the over-detection rate and under-detection rate of solder mark detection.

[0005] In a first aspect, a method for measuring solder marks is provided, comprising: acquiring a first image, the first image including solder marks; and obtaining a classification type of the solder marks based on the first image and a solder mark classification model, wherein the solder mark classification model is used to classify the solder marks according to their features.

[0006] By classifying weld marks in the first image using a weld mark classification model, there is no need to set a grayscale threshold for the first image. This avoids misclassification of weld mark types caused by factors such as the shooting location and equipment affecting the grayscale of the captured image when judging images with the same grayscale threshold. Furthermore, analyzing the data in the image using the weld mark classification model improves the accuracy of the classification results, thereby reducing the over-detection and under-detection rates of weld marks.

[0007] In some embodiments, the characteristics of the solder mark include at least one of the shape, color, and area of ​​the solder mark.

[0008] By processing the features of solder marks that clearly distinguish between qualified and defective solder marks, the solder mark classification model can classify solder marks more accurately and improve the accuracy of the classification results.

[0009] In some embodiments, obtaining the classification type of the solder mark based on the first image and the solder mark classification model includes: obtaining a first classification value based on the first image and the solder mark classification model; and obtaining the classification type of the solder mark based on the first classification value.

[0010] The solder stamp classification model can process only the image and data, and then assign the first classification value to the specific classification type in subsequent processing. This can reduce the complexity of the solder stamp classification model in processing the data in the first image and improve processing efficiency.

[0011] In some embodiments, obtaining the first image includes: obtaining a second image, the second image including the welding area where the solder mark is located; and segmenting the first image from the second image based on the second image and the solder mark segmentation model.

[0012] Segmenting the weld stamp portion in the second image using a weld stamp segmentation model helps to extract and analyze the weld stamp portion, avoiding the influence of other areas in the image on the weld stamp classification type, and thus improving the accuracy of weld stamp classification.

[0013] In some embodiments, the solder stamp segmentation model is used to segment the first image from the second image based on the boundary shape of the solder stamp.

[0014] This allows for the rapid identification of solder joint areas without requiring analysis and processing of each individual pixel, thereby improving segmentation efficiency and reducing data processing time.

[0015] In some embodiments, obtaining the second image includes: obtaining a third image, the third image including the surface area where the welding area is located; extracting the welding area from the third image to obtain the second image.

[0016] By extracting the welding area from images directly captured by the camera, the detection range can be narrowed, processing time reduced, and detection accuracy increased. Simultaneously, a third image can be acquired over a larger area. When multiple objects need to be inspected, this reduces the number of shots required and the number of industrial cameras needed, decreasing the time the product being inspected is displayed during image capture and improving work efficiency.

[0017] In some embodiments, the classification type includes qualified, faulty, or defective.

[0018] This allows for rapid classification results and termination of the inspection process when weld marks are clearly qualified or defective, thus shortening the processing time for inspecting weld marks. In cases where the classification type is "explosion point," further analysis and inspection of the weld marks can be performed to improve inspection accuracy and reduce the false detection rate.

[0019] In some embodiments, the method further includes: if the solder mark is classified as a burst point, extracting a target region in the first image, the target region being a region in the first image with a color abrupt change; and determining whether the solder mark is qualified or defective based on the target region.

[0020] By conducting secondary inspections on weld stamps whose classification as either acceptable or defective cannot be determined, these weld stamps can be further classified more accurately, thereby improving classification accuracy and reducing the over-detection and under-detection rates. Simultaneously, by further analyzing the characteristics of smaller areas within the weld stamps, inspection can be performed with even higher precision, improving the accuracy of weld stamp classification and reducing the false detection rate.

[0021] In some embodiments, determining whether the solder mark is qualified or defective based on the target area includes: determining whether the type of the solder mark is qualified or defective based on the target area and a region classification model.

[0022] By analyzing and processing images of the target area using deep learning models, weld marks classified as "explosion points" can be further subdivided more accurately. This classification method can improve the accuracy of weld mark detection, avoid the influence of grayscale deviations in the image on the detection results, and thus reduce the over-detection rate and the under-detection rate.

[0023] In some embodiments, the region classification model is used to classify the solder marks according to the shape of the target region.

[0024] By processing the features of the target area that clearly distinguish between qualified and defective solder marks, the region classification model can more accurately differentiate solder mark types, avoid the influence of grayscale deviations in the target area on the classification results, and ensure the accuracy of detection.

[0025] In some embodiments, determining whether the type of the solder mark is qualified or defective based on the target area and the area classification model includes: obtaining a second classification value based on the target area and the area classification model; and determining whether the type of the solder mark is qualified or defective based on the second classification value.

[0026] Outputting a second classification value from the region classification model can reduce the complexity of data processing for the region classification model and improve detection efficiency without affecting the processing accuracy.

[0027] In some embodiments, extracting the target region in the first image includes: obtaining the target region based on the first image and a region segmentation model, wherein the region segmentation model is used to segment regions with abrupt color changes in the first image into the target region.

[0028] This allows for accurate segmentation of the target area in the first image, which is beneficial for subsequent classification of weld marks based on the target area. It also avoids the influence of other areas outside the target area on the detection results, thereby improving detection accuracy.

[0029] In some embodiments, determining whether the solder mark is qualified or defective based on the target area includes: determining whether the solder mark is qualified or defective based on the grayscale and / or area of ​​the target area.

[0030] This method has a relatively simple calculation process and can quickly further subdivide the solder marks classified as burst points, thereby determining whether the solder marks are qualified or defective.

[0031] In some embodiments, determining the type of solder mark as qualified or defective based on the grayscale and / or area of ​​the target area includes: determining the type of solder mark as defective when the area of ​​the target area is greater than or equal to a first threshold, or when the grayscale of the target area is greater than or equal to a second threshold; or determining the type of solder mark as qualified when the area of ​​the target area is less than the first threshold and when the grayscale of the target area is less than the second threshold.

[0032] By comparing the results with a threshold, the condition of the target area can be judged quickly to determine whether the solder joint is qualified or defective, thus improving processing efficiency.

[0033] In some embodiments, the first classification value includes a first probability, a second probability, and a third probability, wherein the first probability is the probability that the classification type is qualified, the second probability is the probability that the classification type is a fault, and the third probability is the probability that the classification type is defective; obtaining the classification type of the solder mark based on the first classification value includes: determining the classification type corresponding to the maximum value among the first probability, the second probability, and the third probability as the classification type of the solder mark.

[0034] This allows for a more accurate determination of the solder mark classification type when the classification type in the first image is similar to several other types. This reduces the possibility of misclassification and helps lower the over-detection and under-detection rates of solder mark detection.

[0035] In some embodiments, the solder mark is an anode solder mark, and the solder mark classification model is a classification model of the anode solder mark, which is trained from the data of the anode solder mark; or, the solder mark is a cathode solder mark, and the solder mark classification model is a classification model of the cathode solder mark, which is trained from the data of the cathode solder mark.

[0036] Training the weld mark classification models for detecting anode and cathode weld marks separately helps to more accurately determine the weld mark classification type based on the characteristics of the anode or cathode weld marks, thereby improving the accuracy of weld mark detection.

[0037] In a second aspect, an apparatus for detecting solder marks is provided, comprising: a processor and a memory, the memory storing instructions which, when executed by the processor, cause the apparatus to perform the method as described in any embodiment of the first aspect above.

[0038] Thirdly, an electronic device is provided, comprising: a device for detecting solder marks as described in any embodiment of the second aspect above.

[0039] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed, performs the method as described in any of the embodiments of the first aspect above. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of a system architecture provided in an embodiment of this application.

[0042] Figure 2 This is a schematic flowchart of a method for detecting solder marks provided in an embodiment of this application.

[0043] Figure 3 This is a schematic diagram of a first image provided in an embodiment of this application.

[0044] Figure 4 This is a schematic diagram of a second image provided in an embodiment of this application.

[0045] Figure 5 This is a schematic diagram illustrating the extraction of a target region from a first image, provided in an embodiment of this application.

[0046] Figure 6 This is a schematic flowchart of another method for detecting solder marks provided in an embodiment of this application.

[0047] Figure 7 This is a schematic diagram of a device for detecting solder marks provided in an embodiment of this application.

[0048] The accompanying drawings are not drawn to scale. Detailed Implementation

[0049] The embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to illustrate the principles of this application by way of example, but should not be used to limit the scope of this application, that is, this application is not limited to the described embodiments.

[0050] In the description of this application, it should be noted that, unless otherwise stated, "a plurality of" means two or more; the terms "upper," "lower," "left," "right," "inner," and "outer," etc., indicating orientation or positional relationships, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. Furthermore, the terms "first," "second," and "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. "Vertical" is not vertical in the strict sense, but within the allowable tolerance range. "Parallel" is not parallel in the strict sense, but within the allowable tolerance range.

[0051] The directional terms used in the following description refer to the directions shown in the figures and are not intended to limit the specific structure of this application. It should also be noted in the description of this application that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0052] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0053] In battery manufacturing, welding is often used to connect various components on a single battery cell. The weld marks produced during welding are a crucial indicator of welding quality. However, current methods for inspecting weld marks typically involve directly capturing images of the product using an industrial camera, analyzing connected regions of different gray levels within the image, and setting a gray level threshold to determine the weld mark's quality. However, in this image-based inspection and judgment process, the background can severely interfere with the delineation of weld mark areas, and scanning accuracy can also affect the precision of weld mark extraction. Furthermore, capturing weld marks at different workstations using different equipment can influence the gray level of the images, potentially leading to misjudgments when using the same gray level threshold for weld mark type identification. Therefore, setting different thresholds is necessary to ensure the accuracy of the inspection results.

[0054] In view of this, this application provides a method for detecting solder marks. By inputting captured images into a neural network model, the solder marks are classified to determine their type and whether the welding quality at the corresponding location on the product is up to standard. This method for detecting solder marks is unaffected by background interference and does not require adjustment of grayscale thresholds, thus enabling more accurate detection and judgment of solder marks and reducing the over-detection and under-detection rates.

[0055] Figure 1 A schematic diagram of a system architecture applicable to an embodiment of this application is shown. Figure 1 In this process, the data acquisition device 160 is used to collect training data. After collecting the training data, the data acquisition device 160 stores the training data in the database 130, and the training device 120 trains the target model / rule 101 based on the training data maintained in the database 130.

[0056] The aforementioned target model / rule 101 can be used to implement the solder mark detection method of this application embodiment, specifically a neural network. It should be noted that in practical applications, the training data maintained in the database 130 may not all come from the data acquisition device 160; it may also be received from other devices. Furthermore, it should be noted that the training device 120 may not necessarily train the target model / rule 101 entirely based on the training data maintained in the database 130; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0057] The target model / rule 101 trained using training device 120 can be applied to different systems or devices, such as... Figure 1The execution device 110 shown can be a terminal, such as a mobile phone terminal, tablet computer, laptop computer, etc., or it can be a server or cloud service. Figure 1 In this embodiment, the execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with external devices. Users can input data to the I / O interface 112 through the client device 140. The input data may include the video or image to be processed input by the client device 140.

[0058] In some implementations, the client device 140 may be the same device as the execution device 110. For example, both the client device 140 and the execution device 110 may be terminal devices.

[0059] In other embodiments, the client device 140 may be a different device from the execution device 110. For example, the client device 140 may be a terminal device, while the execution device 110 may be a cloud device, a server, or other such device. The client device 140 may interact with the execution device 310 through a communication network of any communication mechanism / standard. The communication network may be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0060] The calculation module 111 of the execution device 110 is used to process the input data (such as the image to be processed) received by the I / O interface 112. During the calculation and other related processing performed by the calculation module 111 of the execution device 110, the execution device 110 can call data, code, etc. in the data storage system 150 for corresponding processing, and can also store the data, instructions, etc. obtained from the corresponding processing into the data storage system 150.

[0061] Finally, I / O interface 112 returns the processing results, such as the obtained solder mark classification results, to customer device 140, thereby providing them to the user.

[0062] It is worth noting that, Figure 1 This is merely a schematic diagram of a system architecture provided in an embodiment of this application. The positional relationships between the devices, components, modules, etc., shown in the diagram do not constitute any limitation. For example, in Figure 1 In this case, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110.

[0063] like Figure 1As shown, the target model / rule 101 is trained by the training device 120. In this embodiment of the application, the target model / rule 101 can be a neural network. Specifically, the neural network in this embodiment of the application can be a CNN, a region convolutional neural network (RCNN), a faster region convolutional neural network (faster RCNN), or other types of neural networks, etc. This application does not make any specific limitations on this.

[0064] The following is combined with Figure 2 The method 200 for detecting solder marks provided in this application will be described in detail. Figure 2 This is a schematic flowchart illustrating a method for detecting solder marks according to an embodiment of this application. This method 200 can be executed by a device for detecting solder marks, which can be... Figure 1 The execution device 110 is used in this application. For ease of explanation, this embodiment uses a part requiring welding during battery production as an example. It should be understood that the method provided in this embodiment can also be used to detect solder marks in other scenarios. Method 200 may include at least some of the following.

[0065] S210: The device for detecting solder marks acquires a first image, the first image including the solder mark.

[0066] S220: The device for detecting solder marks obtains the classification type of solder marks based on the first image and the solder mark classification model, wherein the solder mark classification model is used to classify solder marks according to their characteristics.

[0067] The first image can be obtained by photographing solder marks generated during the actual production process, and the first image must include at least the solder mark to be inspected. Taking the battery production process as an example, the solder mark can be a solder mark left at a location where welding is required during battery production, such as the solder mark formed by welding the connecting components of a battery cell to the electrode terminals. The first image may also include the periphery of the solder mark. Taking the solder mark formed by welding the connecting components of a battery cell to the electrode terminals as an example, the first image may include the entire area of ​​the electrode terminals, not just the welded portion of the electrode terminals. The first image may also include other parts of the battery cell in the area where the solder mark is located, for example, it may include the area around the electrode terminals. In one possible implementation, the solder mark to be inspected occupies most of the first image, while the periphery or background portion of the solder mark occupies a smaller proportion of the first image.

[0068] A solder stamp classification model refers to a model that classifies solder stamps in a first image. This model can be, for example, a neural network. During the training process, a first image with known classification results can be input into the solder stamp classification model. The model analyzes and processes the first image to obtain a first classification result. This first classification result is compared with the known classification results, and the computational parameters in the solder stamp classification model are adjusted to ensure a match between the first and known classification results. By training the solder stamp classification model with a large number of first images with known classification results and continuously adjusting the computational parameters, it becomes possible to achieve a classification result that matches the actual solder stamp type represented by the first image when only one first image is input. In other words, within a certain error range, the classification result output by the solder stamp classification model can be considered accurate.

[0069] In the process of classifying the first image using the solder stamp classification model, the first image with unknown classification results is input into the solder stamp classification model. Based on the parameters determined during the training process of the model, the data of the first image is analyzed and processed. The classification type of the solder stamp on the first image can be considered as an accurate classification result within a certain error range.

[0070] In the process of classifying weld stamps, their characteristics are a crucial basis for classification. These characteristics can be physical properties of the weld stamp itself, such as its shape, color, and area. During the analysis and processing of the weld stamps in the first image, the weld stamp classification model can extract the weld stamps from the image into data based on their characteristics, and then analyze and process the data. By distinguishing weld stamps from different welding conditions based on their characteristics, weld stamp classification can be achieved.

[0071] By classifying weld marks in the first image using a weld mark classification model, there is no need to set a grayscale threshold for the first image. This avoids misclassification of weld mark types caused by factors such as the shooting location and equipment affecting the grayscale of the captured image when judging images with the same grayscale threshold. Furthermore, analyzing the data in the image using the weld mark classification model improves the accuracy of the classification results, thereby reducing the over-detection and under-detection rates of weld marks.

[0072] According to some embodiments of this application, optionally, the characteristics of the solder stamp include at least one of the shape, color, and area of ​​the solder stamp.

[0073] This application provides examples of weld marks formed by welding the electrode terminals of the anode to the connecting member, and weld marks formed by welding the electrode terminals of the cathode to the connecting member. Figure 3 As shown, Figure 3(a) shows the solder marks in the anode area. Figure 3 (b) shows the solder mark in the cathode area.

[0074] from Figure 3 As can be seen, qualified anode solder marks are typically a fairly standard circle, without excessive dark areas, and occupy a certain proportion of the area on the anode electrode terminal. Qualified cathode solder marks are typically a fairly standard circle with three separate protruding parts around its perimeter, also without excessive dark areas, and occupy a certain proportion of the area on the cathode electrode terminal. Solder mark classification models can be used to classify solder marks by comparing them with qualified solder marks during the inspection of anode or cathode solder marks, using at least one of the following criteria: shape, color, or area.

[0075] It should be understood that shape, color, and area can be judged individually, or any combination of these factors can be used for judgment. When judging by combining these factors, each factor must meet the qualification criteria for the weld mark to be classified as a qualified weld mark; if any factor fails to meet the qualification criteria, the weld mark cannot be classified as a qualified weld mark.

[0076] By processing the features of solder marks that clearly distinguish between qualified and defective solder marks, the solder mark classification model can classify solder marks more accurately and improve the accuracy of the classification results.

[0077] According to some embodiments of this application, optionally, the device for detecting solder marks obtains a first classification value based on a first image and a solder mark classification model; and obtains the classification type of the solder mark based on the first classification value.

[0078] The first classification value can be the direct output of the solder stamp classification model, meaning the model converts the information in the first image into data and performs analysis and processing to obtain the result. Different first classification values ​​correspond to different classification types. After obtaining the first classification value, it can be assigned to a specific classification type in subsequent processing.

[0079] For example, the first classification value can be set to 0 or 1, where the classification type corresponding to the first classification value of 0 is qualified and the classification type corresponding to the first classification value of 1 is defective. The solder stamp classification model analyzes and processes the data extracted from the first image and can obtain the first classification value as 0 or 1. Then, based on the specific first classification value obtained, the classification type of the solder stamp in the first image is determined.

[0080] Optionally, multiple first category values ​​can be set according to the needs of different classification types. For example, when three types need to be set, the first category value can be set to 0, 1, or 2, with each first category value corresponding to a different classification type.

[0081] The solder stamp classification model can process only the image and data, and then assign the first classification value to the specific classification type in subsequent processing. This can reduce the complexity of the solder stamp classification model in processing the data in the first image and improve processing efficiency.

[0082] According to some embodiments of this application, optionally, the device for detecting solder marks acquires a second image, the second image including the welding area where the solder mark is located; and a first image is segmented from the second image based on the second image and the solder mark segmentation model.

[0083] Weld marks are traces left from welding within a certain area. Therefore, when photographing weld marks, it cannot be guaranteed that the captured image only includes the weld mark; it may also include the surrounding area. To reduce the interference of irrelevant areas on weld mark classification, a weld mark segmentation model can be used to segment the second image to obtain a first image containing the weld mark.

[0084] The second image can be obtained by photographing the weld marks generated during the actual production process. For example, the second image shows the weld mark and the surrounding area, while the first image is obtained by inputting the second image into a weld mark segmentation model and segmenting the weld mark separately. Figure 4 For example, Figure 4 Image (a) is a picture of the anode electrode terminal, a portion of which is the anode solder mark; Figure 4 Image (b) is a picture of the cathode electrode terminal, with a portion of the area showing the cathode solder mark. Figure 4 Both (a) and (b) can be used as the second image, input into the solder joint segmentation model, and the resulting first image will correspond to... Figure 3 (a) and (b) in the example.

[0085] Segmenting the weld stamp portion in the second image using a weld stamp segmentation model helps to extract and analyze the weld stamp portion, avoiding the influence of other areas in the image on the weld stamp classification type, and thus improving the accuracy of weld stamp classification.

[0086] According to some embodiments of this application, optionally, a solder stamp segmentation model is used to segment a first image from a second image based on the boundary shape of the solder stamp.

[0087] Solder stamp segmentation models can segment solder stamps, for example, using UNet semantic segmentation. Semantic segmentation detects each pixel and its surrounding pixels, classifying them accordingly. For instance, when analyzing a pixel, if the model determines it belongs to the solder stamp area, it marks the pixel as 1; if it belongs to the surrounding area, it marks it as 0. The solder stamp segmentation model can then segment all pixels marked as 1, resulting in the first image.

[0088] In one possible implementation, during the analysis of pixels in the second image, the relative position of the next possible pixel to the current pixel can be determined based on the boundary shape of the solder joint. For example, in an image of an anode electrode terminal, the circular boundary shape of the solder joint can be used as a reference to determine the next pixel to be detected.

[0089] In another possible implementation, the solder stamp segmentation model can record the number of pixels while analyzing them, converting this number into area. The solder stamp can then be classified based on the area of ​​the segmented region, thus determining whether the solder stamp is acceptable. For example, the area of ​​an acceptable solder stamp should fall within a certain numerical range. If the area recorded by the solder stamp segmentation model is outside this range, the solder stamp can be directly considered defective. If the area recorded by the solder stamp segmentation model is within this range, other characteristics of the solder stamp can be further analyzed to determine its classification type.

[0090] This allows for the rapid identification of solder joint areas without requiring analysis and processing of each individual pixel, thereby improving segmentation efficiency and reducing data processing time.

[0091] According to some embodiments of this application, optionally, the apparatus for detecting solder marks acquires a third image, the third image including the surface area where the solder area is located; the solder area in the third image is extracted to obtain a second image.

[0092] In actual production, multiple parts requiring inspection can be captured in a single shot; that is, a single image can include multiple weld marks that need to be inspected. Specifically, the image obtained by the industrial camera can be a third image, which can include the surface area where the weld area is located. The surface area can refer to the plane where the weld area is located, and multiple weld areas can exist on this plane. In one possible implementation, the camera can take a picture in a direction perpendicular to the surface where the weld area is located to obtain a third image. This third image can include the weld area, such as the anode electrode terminal, and may also include the area surrounding the weld area. Further optionally, it may also include other weld areas, such as the cathode electrode terminal.

[0093] During the extraction of the welding area from the third image, the region of interest (ROI) can be extracted to narrow down the scope of the third image that needs to be analyzed and processed. For example, if the third image includes the anode electrode terminal, the cathode electrode terminal, and the end faces of both, the welding area of ​​the anode electrode terminal or the cathode electrode terminal can be extracted from the third image and used as the second image for further data processing.

[0094] If the corresponding ROI is not extracted from the third image, it means that there is no part to be detected in the third image, and the detection process can be terminated directly.

[0095] By extracting the welding area from images directly captured by the camera, the detection range can be narrowed, processing time reduced, and detection accuracy increased. Simultaneously, a third image can be acquired over a larger area. When multiple objects need to be inspected, this reduces the number of shots required and the number of industrial cameras needed, decreasing the time the product being inspected is displayed during image capture and improving work efficiency.

[0096] According to some embodiments of this application, the classification type may optionally include qualified, faulty, or defective.

[0097] In one possible implementation, solder marks can be classified into the three categories mentioned above. A "qualified" classification indicates that the product corresponding to the solder mark can proceed to the next production step; a "defective" classification indicates that the product corresponding to the solder mark needs to be scrapped or returned to the previous production step for adjustment; and a "burst point" classification indicates that there is a part on the solder mark that may affect the welding quality, but it is currently impossible to determine whether this part will have a substantial adverse effect on the product. In the case of a solder mark classified as a "burst point," further assessment is required.

[0098] This allows for rapid classification results and termination of the inspection process when weld marks are clearly qualified or defective, thus shortening the processing time for inspecting weld marks. In cases where the classification type is "explosion point," further analysis and inspection of the weld marks can be performed to improve inspection accuracy and reduce the false detection rate.

[0099] According to some embodiments of this application, optionally, method 200 further includes: when the classification type of solder mark is burst point, the solder mark detection device extracts a target area in the first image, the target area being the area in the first image where the color changes abruptly; and determines whether the solder mark is qualified or defective based on the target area.

[0100] like Figure 5 As shown, Figure 5 This diagram illustrates the extraction of the target region from the first image. Figure 5 (a) in the image represents an anode solder mark, which is classified as a burst point. Extracting the area with a color abrupt change in (a) yields the target area, which is the portion circled by the solid black line in (b). It should be understood that the solid black line in (b) is added solely to indicate the target area and may not be displayed during actual processing. Similarly, Figure 5 (c) in the diagram represents a cathode solder mark, which is also classified as a burst point. Extracting the area with a color abrupt change in (c) reveals the target area, which is the portion circled by the black solid line in (d). Analyzing this target area allows for further detection of whether the solder mark will have a substantial impact on the product, thus further classifying the burst point solder mark as either acceptable or defective.

[0101] By conducting secondary inspections on weld stamps whose classification as either acceptable or defective cannot be determined, these weld stamps can be further classified more accurately, thereby improving classification accuracy and reducing the over-detection and under-detection rates. Simultaneously, by further analyzing the characteristics of smaller areas within the weld stamps, inspection can be performed with even higher precision, improving the accuracy of weld stamp classification and reducing the false detection rate.

[0102] According to some embodiments of this application, optionally, the device for detecting solder marks determines whether the solder mark is qualified or defective based on the target area and the area classification model.

[0103] During the analysis of the target area, neural networks can be used. Specifically, a region classification model can be trained for the target area, and the extracted target area image can be input into the region classification model to determine whether the solder mark is qualified or defective.

[0104] By analyzing and processing images of the target area using deep learning models, weld marks classified as "explosion points" can be further subdivided more accurately. This classification method can improve the accuracy of weld mark detection, avoid the influence of grayscale deviations in the image on the detection results, and thus reduce the over-detection rate and the under-detection rate.

[0105] According to some embodiments of this application, optionally, a region classification model is used to classify solder marks based on the characteristics of a target region.

[0106] In classifying solder marks using a region classification model, the solder marks can be classified based on the characteristics of the target region. For example, the characteristics of the target region can be at least one of shape, color, and area.

[0107] Taking shape as an example, whether the target area is regular in shape, concentrated in a small area, or long and narrow with a wide extended area can all be used as a basis for classifying solder marks.

[0108] Taking color as an example, if the target area is the part in the first image where the color changes abruptly, then the color intensity of the target area can also be used as a basis for classifying solder marks.

[0109] Taking area as an example, whether the area of ​​the target area is within a certain threshold range. If the area of ​​the target area is small and has little impact on the welding quality, it can be classified as qualified; if the area of ​​the target area is large and its impact on the welding quality cannot be ignored, it can be classified as unqualified.

[0110] By processing the features of the target area that clearly distinguish between qualified and defective solder marks, the region classification model can more accurately differentiate solder mark types, avoid the influence of grayscale deviations in the target area on the classification results, and ensure the accuracy of detection.

[0111] According to some embodiments of this application, optionally, the device for detecting solder marks obtains a second classification value based on the target area and the area classification model; and determines the type of solder mark as qualified or defective based on the second classification value.

[0112] The second classification value can be the direct output of the region classification model after processing the image of the target region. Different second classification values ​​correspond to different classification types. In the process of inspecting weld beads, weld beads that require further analysis and processing of the target region are classified as burst points by the weld bead classification model. Therefore, in the inspection of this part of the weld bead target region, it is only necessary to determine whether the weld bead is qualified or defective. In this case, the second classification value can include two values, one corresponding to qualified and the other to defective. The region classification model outputs one of these two values ​​based on the data processing results, and then in subsequent processing, the second classification value is assigned to the specific classification type.

[0113] Outputting a second classification value from the region classification model can reduce the complexity of data processing for the region classification model and improve detection efficiency without affecting the processing accuracy.

[0114] According to some embodiments of this application, optionally, the device for detecting solder marks obtains a target region based on a first image and a region segmentation model, wherein the region segmentation model is used to segment regions with abrupt color changes in the first image into the target region.

[0115] In the process of extracting the target region from the first image, a deep learning model can be used to segment the target region, i.e., a region segmentation model. For example, a UNet semantic segmentation model can segment the target region, where semantic segmentation detects each pixel and its surrounding pixels, classifying different pixels. Specifically, a region segmentation model can determine whether a pixel belongs to the target region by judging whether it has a color abrupt change compared to at least some of its surrounding pixels.

[0116] This allows for accurate segmentation of the target area in the first image, which is beneficial for subsequent classification of weld marks based on the target area. It also avoids the influence of other areas outside the target area on the detection results, thereby improving detection accuracy.

[0117] According to some embodiments of this application, optionally, the apparatus for detecting solder marks determines whether the solder mark is qualified or defective based on the gray level and / or area of ​​the target area.

[0118] In one possible implementation, the grayscale and / or area of ​​the target region can also be used as the basis for detecting solder marks. For example, connected regions in the first image can be extracted using blob visual analysis as the target region. Simultaneously, the grayscale of the target region can be acquired, and a grayscale threshold can be set for the target region to determine whether the solder marks in the first image are acceptable or defective. Optionally, the area of ​​the target region can also be acquired, and an area threshold can be set for the target region to determine whether the solder marks in the first image are acceptable or defective.

[0119] This method has a relatively simple calculation process and can quickly further subdivide the solder marks classified as burst points, thereby determining whether the solder marks are qualified or defective.

[0120] According to some embodiments of this application, optionally, the solder joint type is determined to be defective if the area of ​​the target region is greater than or equal to a first threshold, or if the gray level of the target region is greater than or equal to a second threshold; or, the solder joint type is determined to be qualified if the area of ​​the target region is less than the first threshold and the gray level of the target region is less than the second threshold.

[0121] In determining the type of solder joint based on the area and grayscale of the target region, thresholds can be set for both. If either is outside the acceptable threshold range, the solder joint is determined to be defective; only when both are within the acceptable threshold range can the solder joint be determined to be acceptable.

[0122] By comparing the results with a threshold, the condition of the target area can be judged quickly to determine whether the solder joint is qualified or defective, thus improving processing efficiency.

[0123] According to some embodiments of this application, optionally, the first classification value includes a first probability, a second probability, and a third probability, where the first probability is the probability that the classification type is qualified, the second probability is the probability that the classification type is a burst point, and the third probability is the probability that the classification type is defective. The classification type corresponding to the maximum value among the first probability, the second probability, and the third probability is determined as the classification type of the solder mark.

[0124] The solder stamp classification model can directly output probability values ​​corresponding to different classification types. When the solder stamp classification type is determined to be three categories, the first classification value obtained from the model can include a first probability, a second probability, and a third probability. During the analysis and processing of the first image, the model calculates the probability that the solder stamp in the first image is of the type of acceptable, burst point, or defective. These three probabilities represent the likelihood of the solder stamp being acceptable, burst point, or defective. The classification type with the highest probability is taken as the classification type of the solder stamp.

[0125] This allows for a more accurate determination of the solder mark classification type when the classification type in the first image is similar to several other types. This reduces the possibility of misclassification and helps lower the over-detection and under-detection rates of solder mark detection.

[0126] According to some embodiments of this application, optionally, the solder mark is an anode solder mark, the solder mark classification model is an anode solder mark classification model, and the anode solder mark classification model is trained from anode solder mark data; or...

[0127] The weld stamps are cathode weld stamps, and the weld stamp classification model is a classification model for cathode weld stamps. The classification model for cathode weld stamps is trained from the data of cathode weld stamps.

[0128] from Figures 3 to 5 As can be seen, anode and cathode weld marks differ significantly in shape, leading to substantial differences in their classification. Therefore, anode and cathode weld marks can be detected separately. In one possible implementation, a weld mark classification model is trained using anode weld mark data, such as anode weld mark images, to obtain an anode weld mark classification model. This anode weld mark classification model is then used to determine its classification type. Similarly, a cathode weld mark classification model is trained using cathode weld mark data, such as cathode weld mark images, to obtain a cathode weld mark classification model. This cathode weld mark classification model is then used to determine its classification type.

[0129] Training the weld mark classification models for detecting anode and cathode weld marks separately helps to more accurately determine the weld mark classification type based on the characteristics of the anode or cathode weld marks, thereby improving the accuracy of weld mark detection.

[0130] In another alternative implementation, the method for detecting solder marks provided in this application embodiment may include, as follows: Figure 6 The process 600 shown may include at least some of the following.

[0131] 601. The device for detecting solder marks acquires a third image, for example, the third image may be an image directly captured by an industrial camera.

[0132] 602. Detect the region of interest (ROI) in the third image to confirm whether the third image contains the weld mark that needs to be detected. If yes, extract the ROI from the third image and use it as the second image for subsequent processing; otherwise, end the process. The second image may include the welding area where the weld mark is located.

[0133] 603. Obtain a first image, for example, the first image may be obtained from a second image. In one possible implementation, the device for detecting solder marks may use Unet semantic segmentation to segment the solder marks in the second image to obtain the first image.

[0134] 604. Obtain the classification type of the welding mark according to the first picture and the welding mark classification model. Input the first picture into the welding mark classification model, and use the trained model to analyze and process the first picture to obtain the classification type of the welding mark. Among them, the classification type obtained by using the welding mark classification model can be qualified, explosion point or defective. When the welding mark is classified as qualified or defective, it can be used as the final classification result of the welding mark, and the process ends. When the welding mark is classified as an explosion point, the welding mark can be further analyzed.

[0135] 605. When the welding mark in the first picture is classified as an explosion point, extract the target area in the first picture. Among them, the target area can be the part in the first picture that may affect the welding quality. For example, the area with a sudden color change in the first picture.

[0136] 606. Analyze the target area to determine that the final classification type of the welding mark is qualified or defective. Optionally, the target area can be analyzed by using a region classification model or setting a grayscale threshold, etc., so as to obtain the classification type of the welding mark.

[0137] The method for detecting welding marks provided by the embodiments of the present application improves the accuracy of welding mark classification by segmenting the welding marks, so that the process of welding mark detection can be free from background interference. At the same time, using a deep learning model to classify the welding marks also does not require relying on the adjustment of the grayscale threshold, making the classification of the welding marks more accurate. In addition, further detecting the welding marks that cannot be directly classified as qualified or defective can detect the welding marks with higher precision, while reducing the data processing volume of the device for detecting welding marks, which is beneficial to improving the detection efficiency.

[0138] The present application also provides a device 700 for detecting welding marks, as Figure 7 shown, including: a processor 701 and a memory 702. When the instructions stored in the memory 702 are executed by the processor 701, the device 700 executes the method described in any of the above embodiments.

[0139] The present application also provides an electronic device, including the device 700 for detecting welding marks described in the above embodiments.

[0140] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, it executes the method described in any of the above embodiments in the first aspect.

[0141] Although this application has been described with reference to preferred embodiments, various modifications can be made thereto and components can be replaced with equivalents without departing from the scope of this application. In particular, the technical features mentioned in the various embodiments can be combined in any manner, provided there is no structural conflict. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method of detecting a weld, characterized by, include: Obtain a first image, which includes solder marks; Based on the first image and the solder stamp classification model, the classification type of the solder stamp is obtained, wherein the solder stamp classification model is used to classify the solder stamp according to its characteristics, and the classification type includes qualified, burst point, or defective; If the solder mark is classified as a burst point, the target region in the first image is extracted, and the target region is the region in the first image where the color changes abruptly. The solder mark is determined to be either acceptable or defective based on the target area.

2. The method of claim 1, wherein, The characteristics of the solder mark include at least one of the following: shape, color, and area.

3. The method of claim 1, wherein, The step of obtaining the classification type of the solder mark based on the first image and the solder mark classification model includes: Based on the first image and the solder mark classification model, a first classification value is obtained; The classification type of the solder mark is obtained based on the first classification value.

4. The method of claim 1, wherein, The process of obtaining the first image includes: Obtain a second image, which includes the welding area where the solder mark is located; Based on the second image and the solder joint segmentation model, the first image is segmented from the second image.

5. The method of claim 4, wherein, The solder stamp segmentation model is used to segment the first image from the second image based on the boundary shape of the solder stamp.

6. The method of claim 4, wherein, The process of obtaining the second image includes: Obtain a third image, the third image including the surface area where the welding area is located; The welding area is extracted from the third image to obtain the second image.

7. The method of claim 1, wherein, The step of determining whether the solder mark is qualified or defective based on the target area includes: Based on the target area and the area classification model, the type of the solder mark is determined to be either qualified or defective.

8. The method of claim 7, wherein, The region classification model is used to classify the solder marks based on the characteristics of the target region.

9. The method of claim 7, wherein, The step of determining whether the solder mark is qualified or defective based on the target area and the area classification model includes: Based on the target region and the region classification model, a second classification value is obtained; The type of solder mark is determined as qualified or defective based on the second classification value.

10. The method according to claim 1, characterized in that, Extracting the target region from the first image includes: Based on the first image and the region segmentation model, the target region is obtained. The region segmentation model is used to segment regions with abrupt color changes in the first image into the target region.

11. The method according to claim 1, characterized in that, The step of determining whether the solder mark is qualified or defective based on the target area includes: The solder mark is determined to be qualified or defective based on the grayscale and / or area of ​​the target area.

12. The method according to claim 11, characterized in that, The step of determining whether the solder mark is qualified or defective based on the grayscale and / or area of ​​the target area includes: If the area of ​​the target region is greater than or equal to a first threshold, or if the grayscale value of the target region is greater than or equal to a second threshold, the solder joint is determined to be defective; or... If the area of ​​the target region is less than the first threshold and the gray level of the target region is less than the second threshold, the type of the solder mark is determined to be qualified.

13. The method according to claim 3, characterized in that, The first classification value includes a first probability, a second probability, and a third probability. The first probability is the probability that the classification type is qualified, the second probability is the probability that the classification type is a hot spot, and the third probability is the probability that the classification type is undesirable. The step of obtaining the classification type of the solder mark based on the first classification value includes: The classification type corresponding to the maximum value among the first probability, the second probability, and the third probability is determined as the classification type of the solder mark.

14. The method according to any one of claims 1 to 13, characterized in that, The solder mark is an anode solder mark, and the solder mark classification model is a classification model for the anode solder mark, which is trained from the data of the anode solder mark; or, The weld mark is a cathode weld mark, and the weld mark classification model is a classification model of the cathode weld mark, which is obtained by training the cathode weld mark data.

15. An apparatus for detecting solder marks, characterized in that, include: A processor and a memory, the memory storing instructions that, when executed by the processor, cause the apparatus to perform the method as described in any one of claims 1 to 14.

16. An electronic device, characterized in that, include: The apparatus for detecting solder marks as described in claim 15.

Images