A method, device, electronic device and storage medium for detecting solder joint images

Abstract

The present invention discloses a weld image detection method, device, electronic device, and storage medium. The method involves acquiring a weld image to be detected in real time; inputting the weld image to be detected into a pre-built weld image feature detection model to obtain a weld image detection result; and comparing each confidence value in the weld image detection result with a preset confidence threshold to determine a weld image grade detection result corresponding to the weld image to be detected. The method solves the problems of difficulty in quantifying the quality evaluation of steel mesh welds and inaccurate weld image detection, enables accurate quality evaluation of steel mesh welds, improves the accuracy and efficiency of weld image detection, and reduces labor and time costs.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a solder joint image detection method, device, electronic equipment and storage medium. Background Art

[0002] In construction engineering, steel mesh is composed of multiple rebars arranged horizontally and vertically. This mesh is typically fabricated by spot welding each intersection of the horizontal and longitudinal bars. Reinforcement meshes are often enormous, with a single layer often comprising thousands or even tens of thousands of rebar. Therefore, the quality of these welds directly impacts the safety and reliability of the mesh throughout its installation.

[0003] In the process of realizing the present invention, the inventors found that the existing technology has the following defects: At present, the quality inspection of the weld points of the steel mesh usually requires professionals to make qualitative judgments on whether the weld points are qualified or unqualified one by one. It is difficult to make a quantitative evaluation of the inspection results of the weld point quality, which consumes a lot of manpower and time costs, and also has relatively high requirements for professionals. Summary of the Invention

[0004] The present invention provides a solder joint image detection method, device, electronic equipment and storage medium to improve the accuracy and efficiency of solder joint image detection and reduce labor costs and time costs.

[0005] According to one aspect of the present invention, a method for detecting solder joint images is provided, comprising:

[0006] Real-time acquisition of images of solder joints to be inspected;

[0007] Inputting the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result;

[0008] The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value;

[0009] Each of the confidence values ​​in the weld spot image detection result is compared with a preset confidence threshold value to determine a weld spot image grade detection result corresponding to the weld spot image to be detected.

[0010] According to another aspect of the present invention, a welding spot image detection device is provided, comprising:

[0011] A module for acquiring an image of a solder joint to be inspected, used for acquiring an image of the solder joint to be inspected in real time;

[0012] A solder joint image detection result determination module is used to input the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result;

[0013] The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value;

[0014] The solder joint image grade detection result determination module is used to compare each of the confidence values ​​in the solder joint image detection result with a preset confidence threshold value to determine the solder joint image grade detection result corresponding to the solder joint image to be detected.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a solder joint image detection method according to any embodiment of the present invention when executing the computer program.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a solder joint image detection method according to any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention obtains a weld image to be inspected in real time; inputs the weld image to be inspected into a pre-built weld image feature detection model to obtain a weld image detection result; and compares each confidence value in the weld image detection result with a preset confidence threshold to determine a weld image grade detection result corresponding to the weld image to be inspected. This solves the problems of difficult quantitative quality evaluation of steel mesh welds and inaccurate weld image detection, enabling accurate quality evaluation of steel mesh welds, improving the accuracy and efficiency of weld image detection, and reducing labor and time costs.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flow chart of a solder joint image detection method provided in accordance with the first embodiment of the present invention;

[0021] Figure 2 is a flow chart of another solder joint image detection method provided according to the second embodiment of the present invention;

[0022] Figure 3 2 is a schematic structural diagram of a welding spot image detection device provided according to a third embodiment of the present invention;

[0023] Figure 4 It is a structural diagram of an electronic device provided according to the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "target", "current", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Example 1

[0027] Figure 1 A flowchart of a weld image detection method is provided for the first embodiment of the present invention. This embodiment is applicable to the situation where quality inspection of welds in steel mesh is performed. The method can be performed by a weld image detection device, which can be implemented in the form of hardware and / or software.

[0028] Correspondingly, such as Figure 1 As shown, the method includes:

[0029] S110 , acquiring an image of the solder joint to be inspected in real time.

[0030] The weld spot images to be detected may be images of weld spots on the steel mesh collected in real time, and multiple images of weld spots to be detected may be collected from the steel mesh to determine the quality of the weld spots.

[0031] S120: Input the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result.

[0032] The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value.

[0033] The solder joint image feature detection model may be a model capable of analyzing a solder joint image to obtain a detection result corresponding to the solder joint image to be detected. The solder joint image detection result may be a parameter pair describing the solder joint image.

[0034] Specifically, the weld image detection results include multiple image features and multiple confidence values, and there is a one-to-one correspondence between the image features and the confidence values. The image features can be features that describe the weld image, and can include full image features, pit image features, crack image features, deviated image features, incomplete image features, misaligned image features, and leaky weld image features.

[0035] The confidence value may be used to describe the reliability of each of the image features, or may be referred to as a confidence level and a confidence coefficient.

[0036] Exemplarily, the weld image to be inspected is inspected by a weld image feature detection model to obtain a weld image detection result, which can be: {(fullness: 0.09), (pit: 0.81), (crack: 0.21), (deviation: 0.52), (incompleteness: 0.13), (misalignment: 0.02), (leakage: 0.09)}.

[0037] Furthermore, the multiple image features are full image features, pit image features, cracked image features, deviated image features, incomplete image features, dislocated image features and leaky weld image features; the confidence values ​​corresponding to the above image features are: 0.09, 0.81, 0.21, 0.52, 0.13, 0.02 and 0.09 respectively.

[0038] S130: Compare each of the confidence values ​​in the weld spot image detection result with a preset confidence threshold value to determine a weld spot image grade detection result corresponding to the weld spot image to be detected.

[0039] The confidence threshold may be a value corresponding to a preset confidence threshold. There are two confidence thresholds, namely a first confidence threshold and a second confidence threshold. The solder joint image grade detection result may be a detection result describing the grade corresponding to the solder joint image.

[0040] Specifically, the solder joint image grade detection results may include five grades: good, better, qualified, poor, and bad. Specifically, the corresponding solder joint image grade detection results may be obtained by comparing each confidence level with a confidence threshold.

[0041] Optionally, the image features include at least one of the following: a full image feature, a pit image feature, a crack image feature, a deviation image feature, a defective image feature, a misaligned image feature and a leaky weld image feature; the confidence values ​​in the weld image detection results are compared with preset confidence thresholds to determine the weld image level detection results corresponding to the weld image to be detected, including: judging whether the confidence values ​​in the weld image detection results are less than a preset first confidence threshold, and if so, assigning each target confidence value that meets the conditions to 0 to obtain each filtering confidence value; obtaining a misaligned filtering confidence value corresponding to the misaligned image feature and a leaky weld filtering confidence value corresponding to the leaky weld image feature in the weld image detection results from each filtering confidence value; and determining the weld image level detection result corresponding to the weld image to be detected if any one of the misaligned filtering confidence value and the leaky weld filtering confidence value is greater than a preset second confidence threshold.

[0042] The first confidence threshold may be a pre-set confidence threshold that can filter each confidence value. The filtered confidence value may be a confidence value obtained by filtering each confidence value. The misalignment filtering confidence value may be a filtering confidence value corresponding to the misalignment image feature. The leaking weld filtering confidence value may be a filtering confidence value corresponding to the leaking weld image feature. The second confidence threshold may be a pre-set confidence threshold that can compare the misalignment filtering confidence value and the leaking weld filtering confidence value to determine the weld spot image grade detection result.

[0043] Continuing with the previous example, assuming the first confidence threshold is 0.1 and the second confidence threshold is 0.5, the confidence values ​​corresponding to the image features are 0.09, 0.81, 0.21, 0.52, 0.13, 0.02, and 0.09, respectively.

[0044] It is necessary to compare the above confidence values ​​with the first confidence threshold, that is, to determine whether the confidence values ​​in the solder joint image detection results are less than the preset first confidence threshold. If so, the target confidence values ​​that meet the conditions are assigned to 0 to obtain the filtered confidence values. Since 0.09, 0.02, and 0.09 are all less than the first confidence threshold, the filtered confidence values ​​obtained are: 0, 0.81, 0.21, 0.52, 0.13, 0, and 0. In other words, the filtered solder joint image detection results can be: {(Full: 0), (Pit: 0.81), (Crack: 0.21), (Deviation: 0.52), (Incomplete: 0.13), (Misalignment: 0), (Leakage: 0)}.

[0045] Furthermore, a misalignment filtering confidence value corresponding to the misalignment image feature and a leaking weld filtering confidence value corresponding to the leaking weld image feature are obtained. That is, the misalignment filtering confidence value corresponding to the misalignment image feature and the leaking weld filtering confidence value corresponding to the leaking weld image feature are 0. Since neither the misalignment filtering confidence value nor the leaking weld filtering confidence value is greater than a preset second confidence threshold, additional operations are required to further determine the weld image grade detection result corresponding to the weld image to be detected.

[0046] Assuming that the misalignment filtering confidence value is 0.6, since the misalignment filtering confidence value is greater than the second confidence threshold, the solder joint image grade detection result corresponding to the solder joint image to be detected is determined, and it is assumed that the solder joint image grade detection result is poor.

[0047] The advantage of this setting is that by comparing each confidence value with the first confidence threshold and the second confidence threshold, a specific solder joint image grade detection result is obtained, which can improve the accuracy of determining the solder joint image quality grade.

[0048] Optionally, after the misalignment filtering confidence value and the weld leakage filtering confidence value, it also includes: if the misalignment filtering confidence value and the weld leakage filtering confidence value are both not greater than the second confidence threshold, then constructing the membership feature vector corresponding to the weld spot image to be detected according to each of the filtering confidence values; calculating the membership feature vector and the pre-constructed standard membership matrix through fuzzy multiplication operation to obtain the evaluation feature vector corresponding to the weld spot image to be detected; and calculating the weld spot image grade detection result corresponding to the weld spot image to be detected through defuzzification operation based on the evaluation feature vector.

[0049] The membership feature vector can be a feature vector obtained by filtering the confidence values. Specifically, membership is a type of fuzzy set that describes how each element in a fuzzy set belongs to the fuzzy set. A standard membership matrix can be constructed based on the membership values ​​and a historical fuzzy relationship matrix and can be calculated using fuzzy multiplication. The evaluation feature vector can be a feature vector that evaluates the solder joint image to be inspected.

[0050] Continuing with the previous example, since neither the misalignment filter confidence value nor the leaking weld filter confidence value is greater than the preset second confidence threshold, additional operations are required to further determine the weld image grade detection result corresponding to the weld image to be inspected. In this case, the filtered weld image detection result may be: {(Full: 0), (Pit: 0.81), (Crack: 0.21), (Deviation: 0.52), (Incomplete: 0.13), (Misalignment: 0), (Leaking weld: 0)}.

[0051] Furthermore, a membership feature vector corresponding to the solder joint image to be detected is constructed according to each of the filtering confidence values, that is, the membership feature vector is {0, 0.81, 0.21, 0.52, 0.13, 0, 0}.

[0052] Accordingly, the membership eigenvector and the pre-constructed standard membership matrix are calculated using fuzzy multiplication to obtain the evaluation eigenvector corresponding to the solder joint image to be inspected. Assume that the standard membership matrix is ​​R, and R is a 7×5 matrix. Since the membership eigenvector is a 1×7 eigenvector, it can be calculated that the evaluation eigenvector is a 1×5 eigenvector.

[0053] Furthermore, according to the evaluation feature vector, a defuzzification operation is performed to calculate a solder joint image grade detection result corresponding to the solder joint image to be detected.

[0054] Optionally, the membership feature vector and the pre-constructed standard membership matrix are calculated through fuzzy multiplication operation to obtain the evaluation feature vector corresponding to the weld spot image to be detected, including: according to the membership feature vector, the evaluation feature vector B corresponding to the weld spot image to be detected is calculated through the formula B=A°R; wherein A represents the membership feature vector, R represents the standard membership matrix, and ° represents the operator corresponding to the fuzzy multiplication operation.

[0055] In this embodiment, the evaluation feature vector is calculated using the above formula B=A°R.

[0056] In addition, the fuzzy multiplication operation can be taking the smaller first and then the larger, the product first and then the larger, or the smaller first and then the sum.

[0057] Optionally, the method of calculating the weld image grade detection result corresponding to the weld image to be detected based on the evaluation feature vector through defuzzification operation includes: normalizing the evaluation feature vector to obtain a normalized evaluation feature vector; calculating the weld image score corresponding to the weld image to be detected based on the normalized evaluation feature vector through a pre-constructed membership rule; and determining the weld image grade detection result corresponding to the weld image to be detected based on the weld image score.

[0058] The normalized evaluation feature vector may be a feature vector obtained by normalizing the evaluation feature vector, and the solder joint image scoring may be a scoring process performed on the solder joint image.

[0059] Continuing from the previous example, assume that the evaluation feature vector B is {b1, b2, ..., b m}, and normalize the evaluation feature vector to obtain the normalized evaluation feature vector C.

[0060] Specifically, the normalization process is The normalized evaluation feature vector C is calculated. Further, the score vector V corresponding to the membership rule is obtained = {v1, v2, ..., v m}, according to the formula The solder joint image score is calculated and then the solder joint image grade detection result is determined. Where i is a variable between [1, m].

[0061] The advantage of this setting is that the solder joint image grade detection result is determined by performing normalization processing and other operations on the evaluation feature vector. The solder joint image grade detection result obtained in this way is more accurate, and the quantitative operation of determining the solder joint image quality can be realized, thereby improving the efficiency of solder joint image quality detection.

[0062] The technical solution of the embodiment of the present invention obtains a weld image to be inspected in real time; inputs the weld image to be inspected into a pre-built weld image feature detection model to obtain a weld image detection result; and compares each confidence value in the weld image detection result with a preset confidence threshold to determine a weld image grade detection result corresponding to the weld image to be inspected. This solves the problems of difficult quantitative quality evaluation of steel mesh welds and inaccurate weld image detection, enabling accurate quality evaluation of steel mesh welds, improving the accuracy and efficiency of weld image detection, and reducing labor and time costs.

[0063] Example 2

[0064] Figure 2A flowchart of another weld image detection method provided in Example 2 of the present invention is provided. This embodiment is optimized based on the above embodiments. In this embodiment, before the real-time acquisition of the weld image to be detected, the specific operation process of training the weld image feature detection model is also included.

[0065] Correspondingly, such as Figure 2 As shown, the method includes:

[0066] S210: Acquire historical solder joint images, perform feature extraction on the historical solder joint images using discrete fuzzy language, and obtain at least one historical image feature corresponding to each of the historical solder joint images.

[0067] The historical weld point images may be historical weld point images collected from the steel mesh.

[0068] Optionally, after acquiring the historical weld spot images, performing feature extraction on the historical weld spot images by discrete fuzzy language, and obtaining the historical image features and historical weld spot evaluation features corresponding to the historical weld spot images, the method further includes: performing feature extraction on the historical weld spot images by discrete fuzzy language, and obtaining at least one historical weld spot evaluation feature corresponding to the historical weld spot images; constructing a historical feature fuzzy set based on each of the historical image features, and constructing a historical evaluation fuzzy set based on the historical weld spot evaluation features; constructing a historical fuzzy relationship matrix based on the historical feature fuzzy set and the historical evaluation fuzzy set; matching corresponding membership values ​​according to each parameter pair in the historical fuzzy relationship matrix, and constructing a standard membership matrix based on each membership value.

[0069] The historical image features may be features extracted from historical solder joint images. Specifically, the historical solder joint evaluation features may be features obtained by evaluating historical solder joints, and may include five levels of solder joint evaluation features: good, relatively good, qualified, poor, and bad.

[0070] The historical feature fuzzy set can be a feature fuzzy set constructed based on historical image features. The historical evaluation fuzzy set can be a feature fuzzy set constructed based on historical solder joint evaluation features. The historical fuzzy relationship matrix can be a matrix constructed based on the historical feature fuzzy set and the historical evaluation fuzzy set. The membership value can be an empirical membership value obtained by analyzing historical solder joint images.

[0071] In this embodiment, it is assumed that there are n historical image features, namely {x1, x2, ..., x n}, the historical feature fuzzy set can be constructed as X={u X (x1),u X (x2),...,u X (xn )}. Assume that there are m historical image evaluation features, namely {y1, y2, ..., y m}, the historical evaluation fuzzy set can be constructed as Y = {u Y (y1),u Y (y2),...,u Y (y n )}.

[0072] Furthermore, the historical fuzzy relationship matrix is ​​constructed based on the historical characteristic fuzzy set and the historical evaluation fuzzy set. The historical fuzzy relationship matrix is ​​as follows: According to the membership value obtained, the historical fuzzy relationship matrix is ​​quantified and the standard membership matrix can be obtained as follows:

[0073] The advantage of this setting is that the fuzzy set is constructed through historical image features and historical weld evaluation features, and the standard membership matrix is ​​calculated. The standard membership matrix obtained in this way is more accurate, so that the quality evaluation of the weld images collected on the steel mesh can be more accurate, thereby improving the accuracy and efficiency of weld image detection.

[0074] S220: Using at least one of the historical image features as image feature annotation information, and training a weld image feature detection model using an image detection algorithm based on the image feature annotation information and the historical weld images.

[0075] The image feature annotation information may be composed of a plurality of historical image features and is used to annotate information of historical welding point images.

[0076] In this embodiment, at least one historical image feature is obtained by performing feature extraction on historical weld spot images, and a weld spot image feature detection model is trained based on each of the historical image features as image feature annotation information.

[0077] For example, assume there are three historical weld spot images: historical weld spot image 1, historical weld spot image 2, and historical weld spot image 3. The historical image features extracted from historical weld spot image 1 are: plump image features and pit image features; the historical image features extracted from historical weld spot image 2 are: leaky weld image features; and the historical image features extracted from historical weld spot image 3 are: plump image features, pit image features, cracked image features, and incomplete image features.

[0078] Furthermore, it can be determined that the image feature annotation information corresponding to historical solder joint image 1 is full and pitted; the image feature annotation information corresponding to historical solder joint image 2 is leaking; and the image feature annotation information corresponding to historical solder joint image 3 is full, pitted, cracked, and incomplete.

[0079] Correspondingly, according to the image feature annotation information and the corresponding historical solder joint images, the solder joint image feature detection model is trained through the image detection algorithm.

[0080] S230 , acquiring an image of the solder joint to be inspected in real time.

[0081] S240: Input the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result.

[0082] The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value.

[0083] S250: Compare each of the confidence values ​​in the weld spot image detection result with a preset confidence threshold value to determine a weld spot image grade detection result corresponding to the weld spot image to be detected.

[0084] The technical solution of the embodiment of the present invention obtains historical weld spot images, extracts features from the historical weld spot images using discrete fuzzy language, and obtains at least one historical image feature corresponding to each of the historical weld spot images; uses the at least one historical image feature as image feature annotation information, and trains a weld spot image feature detection model based on the image feature annotation information and the historical weld spot images using an image detection algorithm; obtains weld spot images to be detected in real time; inputs the weld spot images to be detected into a pre-built weld spot image feature detection model to obtain weld spot image detection results; and compares each confidence value in the weld spot image detection result with a preset confidence threshold to determine the weld spot image grade detection result corresponding to the weld spot image to be detected. The method can accurately evaluate the quality of steel mesh welds, improve the accuracy and efficiency of weld spot image detection, and reduce labor and time costs.

[0085] Example 3

[0086] Figure 3 This is a structural diagram of a solder joint image detection device provided in the third embodiment of the present invention. The solder joint image detection device provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal device or server to implement a solder joint image detection method in the embodiment of the present invention. Figure 3 As shown, the device includes: a welding spot image acquisition module 310 to be detected, a welding spot image detection result determination module 320 and a welding spot image grade detection result determination module 330.

[0087] The image acquisition module 310 of the weld spot to be inspected is used to acquire the image of the weld spot to be inspected in real time;

[0088] The solder joint image detection result determination module 320 is configured to input the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result;

[0089] The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value;

[0090] The solder joint image grade detection result determination module 330 is configured to compare each of the confidence values ​​in the solder joint image detection result with a preset confidence threshold value to determine a solder joint image grade detection result corresponding to the solder joint image to be detected.

[0091] The technical solution of the embodiment of the present invention obtains a weld image to be inspected in real time; inputs the weld image to be inspected into a pre-built weld image feature detection model to obtain a weld image detection result; and compares each confidence value in the weld image detection result with a preset confidence threshold to determine a weld image grade detection result corresponding to the weld image to be inspected. This solves the problems of difficult quantitative quality evaluation of steel mesh welds and inaccurate weld image detection, enabling accurate quality evaluation of steel mesh welds, improving the accuracy and efficiency of weld image detection, and reducing labor and time costs.

[0092] Optionally, it also includes a weld image feature detection model training module, which can be specifically used to: before the real-time acquisition of the weld image to be detected, obtain historical weld images, perform feature extraction on the historical weld images through discrete fuzzy language, and obtain at least one historical image feature corresponding to the historical weld images; use at least one of the historical image features as image feature annotation information, and train a weld image feature detection model based on the image feature annotation information and the historical weld images through an image detection algorithm.

[0093] Optionally, the weld image feature detection model training module can also be specifically used for: after acquiring the historical weld image, performing feature extraction on the historical weld image through discrete fuzzy language to obtain the historical image features and historical weld evaluation features corresponding to the historical weld image, performing feature extraction on the historical weld image through discrete fuzzy language to obtain at least one historical weld evaluation feature corresponding to the historical weld image; constructing a historical feature fuzzy set based on each of the historical image features, and constructing a historical evaluation fuzzy set based on the historical weld evaluation features; constructing a historical fuzzy relationship matrix based on the historical feature fuzzy set and the historical evaluation fuzzy set; matching the corresponding membership values ​​according to each parameter pair in the historical fuzzy relationship matrix, and constructing a standard membership matrix based on each membership value.

[0094] Optionally, the image feature includes at least one of the following: a full image feature, a pit image feature, a cracked image feature, a deviated image feature, a defective image feature, a dislocated image feature, and a leaky weld image feature.

[0095] Optionally, the weld image level detection result determination module 330 can be specifically used to: respectively determine whether each of the confidence values ​​in the weld image detection result is less than a preset first confidence threshold; if so, assign each target confidence value that meets the condition to 0 to obtain each filtering confidence value; from each filtering confidence value, obtain the misalignment filtering confidence value corresponding to the misalignment image feature and the leaking welding filtering confidence value corresponding to the leaking welding image feature in the weld image detection result; among the misalignment filtering confidence value and the leaking welding filtering confidence value, if any one is greater than the preset second confidence threshold, determine the weld image level detection result corresponding to the weld image to be detected.

[0096] Optionally, the weld image grade detection result determination module 330 can also be specifically used for: after the misalignment filtering confidence value and the weld leakage filtering confidence value, if the misalignment filtering confidence value and the weld leakage filtering confidence value are both not greater than the second confidence threshold, then constructing the membership feature vector corresponding to the weld image to be detected according to each of the filtering confidence values; calculating the membership feature vector and the pre-constructed standard membership matrix through fuzzy multiplication operation to obtain the evaluation feature vector corresponding to the weld image to be detected; and calculating the weld image grade detection result corresponding to the weld image to be detected through defuzzification operation based on the evaluation feature vector.

[0097] Optionally, the weld image grade detection result determination module 330 can also be specifically used to: calculate the evaluation feature vector B corresponding to the weld image to be detected according to the membership feature vector through the formula B=A°R; wherein A represents the membership feature vector, R represents the standard membership matrix, and ° represents the operator corresponding to the fuzzy multiplication operation.

[0098] Optionally, the weld image grade detection result determination module 330 can also be specifically used to: normalize the evaluation feature vector to obtain a normalized evaluation feature vector; calculate the weld image score corresponding to the weld image to be detected based on the normalized evaluation feature vector through a pre-constructed membership rule; and determine the weld image grade detection result corresponding to the weld image to be detected based on the weld image score.

[0099] The solder joint image detection device provided in the embodiment of the present invention can execute the solder joint image detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0100] Example 4

[0101] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement the fourth embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0102] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0103] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0104] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the solder joint image inspection method.

[0105] In some embodiments, the solder joint image detection method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the solder joint image detection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the solder joint image detection method in any other suitable manner (e.g., via firmware).

[0106] The method includes: acquiring a weld image to be detected in real time; inputting the weld image to be detected into a pre-built weld image feature detection model to obtain a weld image detection result; and comparing each confidence value in the weld image detection result with a preset confidence threshold to determine a weld image grade detection result corresponding to the weld image to be detected.

[0107] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0109] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0113] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0114] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

[0115] Example 5

[0116] Embodiment 5 of the present invention further provides a computer-readable storage medium, wherein the computer-readable instructions, when executed by a computer processor, are used to execute a weld image detection method, the method comprising: acquiring a weld image to be detected in real time; inputting the weld image to be detected into a pre-built weld image feature detection model to obtain a weld image detection result; and comparing each of the confidence values ​​in the weld image detection result with a preset confidence threshold to determine a weld image grade detection result corresponding to the weld image to be detected.

[0117] Of course, the computer-readable storage medium provided in the embodiment of the present invention has computer-executable instructions that are not limited to the above-described method operations, and can also execute related operations in the solder joint image detection method provided in any embodiment of the present invention.

[0118] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0119] It is worth noting that in the embodiment of the above-mentioned weld spot image detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0120] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A solder joint image detection method, characterized in that: include: Real-time acquisition of images of solder joints to be inspected; Inputting the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result; The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value; Comparing each of the confidence values ​​in the weld spot image detection result with a preset confidence threshold value to determine a weld spot image grade detection result corresponding to the weld spot image to be detected; Before acquiring the image of the weld to be inspected in real time, the method further includes: Acquire historical solder joint images, perform feature extraction on the historical solder joint images using discrete fuzzy language, and obtain at least one historical image feature corresponding to each of the historical solder joint images; At least one of the historical image features is used as image feature annotation information, and a weld image feature detection model is trained using an image detection algorithm based on the image feature annotation information and the historical weld images.

2. The method according to claim 1, characterized in that After acquiring the historical solder joint images and extracting features of the historical solder joint images using discrete fuzzy language to obtain historical image features corresponding to the historical solder joint images, the method further includes: Extract features from historical solder joint images using discrete fuzzy language to obtain at least one historical solder joint evaluation feature corresponding to each of the historical solder joint images; According to each of the historical image features, a historical feature fuzzy set is constructed, and according to each of the historical solder joint evaluation features, a historical evaluation fuzzy set is constructed; Constructing a historical fuzzy relationship matrix based on the historical characteristic fuzzy set and the historical evaluation fuzzy set; According to each parameter pair in the historical fuzzy relationship matrix, the corresponding membership values ​​are matched respectively, and according to each membership value, a standard membership matrix is ​​constructed.

3. The method according to claim 2, characterized in that The image feature includes at least one of the following: a full image feature, a pit image feature, a crack image feature, a deviation image feature, a defective image feature, a dislocation image feature, and a leaking weld image feature; The step of comparing each of the confidence values ​​in the solder joint image detection result with a preset confidence threshold value to determine a solder joint image grade detection result corresponding to the solder joint image to be detected includes: Determine whether each of the confidence values ​​in the solder joint image detection result is less than a preset first confidence threshold, and if so, assign each target confidence value that meets the condition to 0 to obtain each filtered confidence value; From each of the filtering confidence values, obtain a misalignment filtering confidence value corresponding to the misalignment image feature and a weld leak filtering confidence value corresponding to the weld leak image feature in the weld spot image detection result; If any one of the misalignment filtering confidence value and the weld leak filtering confidence value is greater than a preset second confidence threshold, a weld image grade detection result corresponding to the weld image to be detected is determined.

4. The method according to claim 3, characterized in that After the misalignment filter confidence value and the leak welding filter confidence value, the following is also included: If both the misalignment filtering confidence value and the weld leak filtering confidence value are not greater than the second confidence threshold, constructing a membership feature vector corresponding to the weld spot image to be detected according to each of the filtering confidence values; Calculating the membership feature vector and a pre-constructed standard membership matrix through fuzzy multiplication to obtain an evaluation feature vector corresponding to the solder joint image to be inspected; According to the evaluation feature vector, a weld spot image grade detection result corresponding to the weld spot image to be detected is calculated through a defuzzification operation.

5. The method according to claim 4, characterized in that The step of calculating the membership characteristic vector and the pre-constructed standard membership matrix through fuzzy multiplication to obtain an evaluation characteristic vector corresponding to the solder joint image to be inspected includes: According to the membership characteristic vector, the evaluation characteristic vector B corresponding to the solder joint image to be detected is calculated by the formula B=A°R; Wherein, A represents the membership eigenvector, R represents the standard membership matrix, and ° represents the operator corresponding to the fuzzy multiplication operation.

6. The method according to claim 4, characterized in that The step of calculating a solder joint image grade detection result corresponding to the solder joint image to be detected by performing a defuzzification operation based on the evaluation feature vector includes: Normalizing the evaluation feature vector to obtain a normalized evaluation feature vector; Calculating a solder joint image score corresponding to the solder joint image to be inspected based on the normalized evaluation feature vector using a pre-established membership rule; A weld image grade detection result corresponding to the weld image to be detected is determined according to the weld image score.

7. A welding spot image detection device, characterized in that: include: A module for acquiring an image of a solder joint to be inspected, used for acquiring an image of the solder joint to be inspected in real time; A solder joint image detection result determination module is used to input the solder joint image to be detected into a pre-built solder joint image feature detection model to obtain a solder joint image detection result; The solder joint image detection result includes: at least one image feature and at least one confidence value; wherein one image feature corresponds to one confidence value; a solder joint image grade detection result determination module, configured to compare each of the confidence values ​​in the solder joint image detection result with a preset confidence threshold value, and determine a solder joint image grade detection result corresponding to the solder joint image to be detected; Among them, it also includes a weld image feature detection model training module, which is used to: before the real-time acquisition of the weld image to be detected, obtain historical weld images, perform feature extraction on the historical weld images through discrete fuzzy language, and obtain at least one historical image feature corresponding to the historical weld images; use at least one of the historical image features as image feature annotation information, and train a weld image feature detection model based on the image feature annotation information and the historical weld images through an image detection algorithm.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, a solder joint image detection method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a welding spot image detection method according to any one of claims 1 to 6 when executed.

Images