Defect detection method, device, apparatus and storage medium

Abstract

The present application belongs to the technical field of defect detection, and discloses a kind of defect detection method, device, equipment and storage medium, comprising: mesh product image is identified to mesh, and the identified mesh is used as node to build directed graph;The dual of directed graph is calculated, and the dual graph is obtained;With the edge in dual graph as reference, set the region of interest, segment product image, obtain edge image set;Each image in edge image set is scored by the classification model obtained by pre-training, and the scoring result is pulled back to the dual graph, to obtain weighted directed graph;According to weighted directed graph, defect classification is carried out.The present application indirectly locates the edge of mesh in mesh product image through dual graph, generates edge image set, scores each edge image in edge image set through classification model, obtains weighted directed graph, and then classifies defects through weighted directed graph, to ensure that mesh product can be automatically detected for defects, and the robustness and accuracy of mesh product defect detection are improved.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a defect detection method, apparatus, equipment and storage medium. Background Technology

[0002] Mesh structures are ubiquitous in industrial production and product appearance. Currently, defect detection of mesh structures is generally carried out manually. However, due to the unique symmetrical periodic structure of mesh structures, the accuracy and efficiency of manual inspection are low. For example, when the mesh structure is obviously deformed or in a semi-transparent state, manual inspection cannot achieve both accuracy and efficiency. Therefore, how to improve the accuracy and efficiency of mesh structure defect detection has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main objective of this invention is to provide a defect detection method, apparatus, device, and storage medium, aiming to solve the technical problems of low accuracy and efficiency in the detection of defects in mesh structures in the prior art.

[0004] To achieve the above objectives, the present invention provides a defect detection method, the method comprising the following steps:

[0005] Mesh recognition is performed on the product image of the mesh product, and a directed graph is constructed using the recognized meshes as nodes. The directed graph is represented by an adjacency list.

[0006] Calculate the dual of the directed graph to obtain the dual graph;

[0007] Using the edges in the dual graph as a reference, a region of interest is set, and the product image is segmented to obtain an edge image set;

[0008] The pre-trained classification model scores each image in the edge image set, and the scoring results are then pulled back to the dual graph to obtain a weighted directed graph.

[0009] Defect classification is performed based on the weighted directed graph.

[0010] Optionally, the step of performing mesh hole recognition on the product image of the mesh product and constructing a directed graph using the recognized mesh holes as nodes includes:

[0011] Acquire product images of mesh products;

[0012] The product image is registered at multiple levels to determine the position of the calibration mesh corresponding to each mesh in the product image.

[0013] A directed graph is constructed based on the calibration mesh positions corresponding to each mesh.

[0014] Optionally, the step of performing multi-level registration on the product image to determine the calibration mesh position corresponding to each mesh in the product image includes:

[0015] The initial mesh position of each mesh in the product image is located by a preset template matching algorithm;

[0016] The initial mesh positions are calibrated by constructing a gradient using the central difference method and then performing gradient descent to obtain the calibrated mesh positions corresponding to each mesh.

[0017] Optionally, the step of constructing a directed graph based on the calibration mesh positions corresponding to each mesh includes:

[0018] The center coordinates of each mesh are determined based on the calibration mesh position of each mesh.

[0019] A directed graph corresponding to the product image is constructed based on the center coordinates of each mesh and a preset side length threshold.

[0020] Optionally, the step of scoring each image in the edge image set using the pre-trained classification model and then pulling the scoring results back to the dual graph to obtain a weighted directed graph includes:

[0021] Rotate each edge image in the edge image set, and construct a sample set based on the processed edge images;

[0022] The pre-trained classification model is used to score each edge image in the sample set to obtain a score result set;

[0023] The scores corresponding to each edge image contained in the scoring result set are pulled back to the dual graph to obtain a weighted directed graph.

[0024] Optionally, the defect classification based on the weighted directed graph includes:

[0025] Defect elements are determined based on the defect scores corresponding to each edge in the weighted directed graph.

[0026] Based on the topology of the weighted directed graph, determine the adjacency relationships between each defective element;

[0027] Defects are classified based on the aforementioned adjacency relationships.

[0028] Optionally, the defect classification based on the adjacency relationship includes:

[0029] Based on the adjacency relationships between each defect element, cluster analysis is performed on each defect element to construct a defect subgraph set;

[0030] The defect subgraph set is classified using a graph classification algorithm;

[0031] The defects of the mesh product are classified according to the defect subgraph set after classification.

[0032] Furthermore, to achieve the above objectives, the present invention also proposes a defect detection device, the device comprising:

[0033] A construction module is used to perform mesh recognition on product images of mesh products, and to construct a directed graph using the recognized meshes as nodes. The directed graph is represented by an adjacency list.

[0034] The calculation module is used to calculate the dual of the directed graph to obtain the dual graph;

[0035] The segmentation module is used to set the region of interest based on the edges in the dual graph, segment the product image, and obtain an edge image set;

[0036] The scoring module is used to score each image in the edge image set using a pre-trained classification model, and pull the scoring results back to the dual graph to obtain a weighted directed graph.

[0037] The grading module is used to grade defects based on the weighted directed graph.

[0038] Furthermore, to achieve the above objectives, the present invention also proposes a defect detection device, the device comprising: a memory, a processor, and a defect detection program stored in the memory and executable on the processor, the defect detection program being configured to implement the steps of the defect detection method as described above.

[0039] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a defect detection program, which, when executed by a processor, implements the steps of the defect detection method described above.

[0040] This invention performs mesh recognition on product images of mesh products, constructing a directed graph using the identified mesh holes as nodes; calculates the dual graph to obtain a dual graph; sets regions of interest based on the edges in the dual graph, segments the product image, and obtains an edge image set; uses a pre-trained classification model to score defects in each image in the edge image set, and pulls the scoring results back into the dual graph to obtain a weighted directed graph; and performs defect classification based on the weighted directed graph. This invention indirectly locates the edges of mesh holes in mesh product images through the dual graph, generates an edge image set, scores defects in each edge image in the edge image set using a classification model to obtain a weighted directed graph, and then performs defect classification based on the weighted directed graph, ensuring automated defect detection of mesh products and improving the robustness and accuracy of defect detection for mesh products. Attached Figure Description

[0041] Figure 1This is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiments of the present invention;

[0042] Figure 2 This is a flowchart illustrating the first embodiment of the defect detection method of the present invention;

[0043] Figure 3 This is a flowchart illustrating the second embodiment of the defect detection method of the present invention;

[0044] Figure 4 This is a flowchart illustrating the third embodiment of the defect detection method of the present invention;

[0045] Figure 5 This is a schematic diagram illustrating the construction of a gradient descent algorithm according to an embodiment of the present invention;

[0046] Figure 6 This is a structural block diagram of the first embodiment of the defect detection device of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0049] Reference Figure 1 , Figure 1 This is a schematic diagram of the defect detection device structure of the hardware operating environment involved in the embodiment of the present invention.

[0050] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a defect detection program.

[0053] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the defect detection device. The electronic device calls the defect detection program stored in the memory 1005 through the processor 1001 and executes the defect detection method provided in the embodiment of the present invention.

[0054] This invention provides a defect detection method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of a defect detection method according to the present invention.

[0055] In this embodiment, the defect detection method includes the following steps:

[0056] Step S10: Perform mesh recognition on the product image of the mesh product, and construct a directed graph using the recognized mesh holes as nodes.

[0057] It should be noted that the execution subject of this embodiment can be the defect detection device. The defect detection device can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, personal computer, etc., or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the defect detection device is used as an example to describe the defect detection method of the present invention.

[0058] It should be noted that the network structure of a typical mesh product is generally quite large, and the directed graph contains a lot of data. Therefore, an adjacency list can be used to represent and store the relevant information of the directed graph. Of course, other similar data structures can also be used for representation, and this embodiment does not limit this.

[0059] In practical applications, mesh recognition of product images of mesh products, and the construction of a directed graph using the identified mesh holes as nodes, can be achieved by using a registration algorithm to identify and locate well-imaged mesh holes (such as quadrilateral mesh holes) in the mesh product, determine the mesh hole positions of the identified mesh holes, and construct a directed graph using the identified mesh holes as nodes and the mesh hole positions as node information.

[0060] Step S20: Calculate the dual of the directed graph to obtain the dual graph.

[0061] It should be noted that calculating the dual of a directed graph can be achieved by performing dual calculations on the areas of each edge partition in the directed graph, and then constructing the dual graph based on the calculation results. The dual graph can also be represented and stored using an adjacency list.

[0062] Step S30: Set the region of interest based on the edges in the dual graph, segment the product image, and obtain the edge image set.

[0063] It should be noted that setting a region of interest (ROI) based on the edges in the dual graph and segmenting the product image to obtain the edge image set can be done by setting a ROI based on each edge in the dual graph, then segmenting the product image according to the set ROI, obtaining the edge image corresponding to each edge in the dual graph, and then aggregating the obtained edge images into a set to obtain the edge image set.

[0064] Step S40: The pre-trained classification model scores each image in the edge image set, and the scoring results are pulled back to the dual graph to obtain a weighted directed graph.

[0065] It should be noted that the classification model can be a pre-trained model. For example, mesh localization can be performed on sample images, a directed graph can be constructed based on the localized meshes, and then the corresponding dual graph can be obtained through calculation. The Regions of Interest (ROIs) can be set based on the edges in the dual graph, and then the sample images can be segmented according to the ROIs to obtain training data. A model training set can then be constructed based on the training data (for example, setting corresponding defect scores or defect labels on the training data and using them as model training samples, and then constructing the model training set based on the model training samples). The classification model can be trained using the model training set. The classification model can be a convolutional neural network classification model or other models capable of classification; this embodiment does not impose any limitations on this.

[0066] In practical applications, scoring each image in the edge image set using a pre-trained classification model can be achieved by inputting each image in the edge image set into the pre-trained classification model, which then scores the images for defects. The scoring results can be then transferred back to the dual graph to obtain a weighted directed graph. This can be done by marking the scoring results corresponding to each edge image on the edges of the dual graph, and using the dual graph with the marked scoring results as the weighted directed graph.

[0067] Furthermore, to reduce the complexity of the classification model, the training data used when training the classification model can be rotated images. Therefore, to ensure that the classification model can correctly score defects, step S40 in this embodiment may include:

[0068] Rotate each edge image in the edge image set, and construct a sample set based on the processed edge images;

[0069] The pre-trained classification model is used to score each edge image in the sample set to obtain a score result set;

[0070] The scores corresponding to each edge image contained in the scoring result set are pulled back to the dual graph to obtain a weighted directed graph.

[0071] It should be noted that rotating each edge image in the edge image set and constructing a sample set based on the processed edge images can be done by rotating each image in the edge image set to a horizontal position, and then aggregating each rotated edge image as a sample to obtain the sample set.

[0072] It is understandable that if each edge image in the edge image set is processed in the same way as the input samples of the classification model, then the classification model can perform defect scoring normally without adjusting the edge images.

[0073] Of course, this rotation process can also be performed when constructing the edge image set. For example, each edge of the dual graph can be rotated to the horizontal, and then the two endpoints can be used as the ROI. After that, the product image can be segmented according to the ROI, and the edge image set can be constructed based on the segmented edge images.

[0074] Step S50: Perform defect classification based on the weighted directed graph.

[0075] It should be noted that defect classification can be used to classify the severity of defects in product images. When classifying defects, the higher the defect level of the product image, the higher the severity of the defect.

[0076] In practical applications, defect classification based on weighted directed graphs can be achieved by determining the defect elements in the weighted directed graph based on the scoring results, and classifying the product image based on the location and / or quantity of the defect elements.

[0077] This embodiment performs mesh recognition on product images of mesh products, constructing a directed graph using the identified mesh holes as nodes; calculates the dual graph to obtain a dual graph; sets regions of interest based on the edges in the dual graph, segments the product image, and obtains an edge image set; uses a pre-trained classification model to score defects in each image in the edge image set, and pulls the scoring results back into the dual graph to obtain a weighted directed graph; and performs defect classification based on the weighted directed graph. This invention indirectly locates the edges of mesh holes in mesh product images through the dual graph, generates an edge image set, scores defects in each edge image in the edge image set using a classification model to obtain a weighted directed graph, and then performs defect classification based on the weighted directed graph, ensuring automated defect detection of mesh products and improving the robustness and accuracy of defect detection for mesh products.

[0078] refer to Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of a defect detection method according to the present invention.

[0079] Based on the first embodiment described above, step S10 of the defect detection method in this embodiment includes:

[0080] Step S101: Obtain product images of the mesh product.

[0081] It should be noted that acquiring product images of the mesh product can be done by receiving product images sent by other devices, such as pre-collecting product images of the mesh product using industrial sensors (CCD, 3D laser scanner, etc.) and sending them to the defect detection equipment. Alternatively, product images of the mesh product can be acquired through an image acquisition device connected to the defect detection equipment; this embodiment does not impose any limitations on this method.

[0082] Step S102: Perform multi-level registration on the product image to determine the position of the calibration mesh corresponding to each mesh in the product image.

[0083] It should be noted that, generally speaking, when performing registration, it can be divided into two main categories: coarse registration and fine registration, depending on the registration algorithm used. Coarse registration is fast, but has lower detection accuracy. Fine registration uses a convex set as its support set. If it is iterated over the entire image (non-convex set), it cannot converge to the center of the mesh.

[0084] At this point, in order to balance recognition speed and recognition accuracy, coarse registration and fine registration can be combined to perform multi-level registration, and after multi-level registration, the position of the calibration mesh corresponding to each mesh in the product image can be obtained.

[0085] In practical implementation, the specific number of registration levels used in multi-level registration can be set by the management personnel of the defect detection equipment according to actual needs. However, if two-level registration is used, then step S102 in this embodiment may include:

[0086] The initial mesh position of each mesh in the product image is located by a preset template matching algorithm;

[0087] The initial mesh positions are calibrated by constructing a gradient using the central difference method and then performing gradient descent to obtain the calibrated mesh positions corresponding to each mesh.

[0088] It should be noted that locating the initial mesh positions of each mesh in a product image using a preset template matching algorithm can be achieved by coarsely registering the product image using the template matching algorithm to determine the image positions of each mesh in the product image, and then using the obtained image positions of each mesh as the initial mesh positions. The template used in the template matching algorithm can be set by the personnel managing the defect detection equipment according to the specific scenario.

[0089] In practical applications, the initial mesh positions are calibrated by constructing a gradient using the central difference method and then performing gradient descent to obtain the calibrated mesh positions corresponding to each mesh. This can be achieved by constructing a gradient using the central difference method based on the initial mesh positions and mesh template, and then performing fine registration of each mesh using gradient descent. Position calibration is then performed based on the initial mesh positions to obtain the calibrated mesh positions corresponding to each mesh. This improves the positioning speed while ensuring the accuracy of mesh positioning.

[0090] Step S103: Construct a directed graph based on the calibration mesh positions corresponding to each mesh.

[0091] In a practical implementation, constructing a directed graph based on the calibration mesh positions corresponding to each mesh can be achieved by constructing directed graph nodes based on the calibration mesh positions corresponding to each mesh, and then constructing a directed graph based on the directed graph nodes.

[0092] In a specific implementation, step S103 of this embodiment may include:

[0093] The center coordinates of each mesh are determined based on the calibration mesh position of each mesh.

[0094] A directed graph corresponding to the product image is constructed based on the center coordinates of each mesh and a preset side length threshold.

[0095] It should be noted that determining the center coordinates of each mesh based on the calibration mesh position can be achieved by calculating the center coordinates of each mesh based on the position coordinates of each calibration mesh, thus obtaining the center coordinates of each mesh.

[0096] In practical applications, constructing a directed graph corresponding to the product image based on the center coordinates of each mesh and a preset edge length threshold can be achieved by using each mesh as a directed graph node, connecting the center coordinates of each mesh, and comparing the edge length of the connected edge with the preset edge length threshold. If the edge length is greater than the preset edge length threshold, the corresponding edge is discarded; if the edge length is less than or equal to the preset edge length threshold, the corresponding edge is retained. The final retained edge and directed graph nodes form the directed graph corresponding to the product image.

[0097] This embodiment acquires a product image of a mesh product; performs multi-level registration on the product image to determine the calibration mesh positions corresponding to each mesh in the product image; and constructs a directed graph based on the calibration mesh positions corresponding to each mesh. Because mesh recognition is not performed directly using either coarse or fine registration, but rather through a combination of coarse and fine registration, multi-level registration of the product image is performed, ensuring both the speed and accuracy of mesh recognition.

[0098] refer to Figure 4 , Figure 4 This is a flowchart illustrating a second embodiment of a defect detection method according to the present invention.

[0099] Based on the first embodiment described above, step S50 of the defect detection method in this embodiment includes:

[0100] Step S501: Determine the defect elements based on the defect scores corresponding to each edge in the weighted directed graph.

[0101] It should be noted that determining defect elements based on the defect scores corresponding to each edge in the weighted directed graph can be achieved by using edges in the weighted directed graph whose corresponding defect scores are greater than a preset defect judgment threshold as defect elements. The preset defect judgment threshold can be pre-set by the administrator of the defect detection equipment.

[0102] For example: Suppose the preset defect judgment threshold is set to 7. At this time, there are four edges A, B, C and D in the weighted directed graph, and their corresponding defect scores are 8, 9, 2 and 8 respectively. Then A, B and D are defect elements.

[0103] Step S502: Determine the adjacency relationship between each defect element based on the topology of the weighted directed graph.

[0104] It should be noted that, based on the topology of the weighted directed graph, determining the adjacency relationship between each defect element can be achieved by obtaining the adjacency list corresponding to the weighted directed graph, constructing the topology based on the association relationships between each node recorded in the adjacency list, and then determining the adjacency relationship between each defect element based on the position of the defect element in the topology.

[0105] For example, a topology can be constructed based on the adjacency list corresponding to a weighted directed graph. This topology can contain the position and number of each edge. Defective elements are marked in the topology, and the adjacency relationship between each defective element can be obtained based on the position and number of the edge corresponding to the defective element.

[0106] Step S503: Classify defects according to the adjacency relationship.

[0107] It should be noted that after obtaining the adjacency relationship of the defective elements, the defective elements can be discretely measured based on the adjacency relationship to classify the defects and thus obtain the defect level of the mesh product.

[0108] Furthermore, since there may be many defective elements, and the mesh sizes to which different defective elements belong also differ, in order to ensure the accuracy of defect classification, step S503 in this embodiment may include:

[0109] Based on the adjacency relationships between each defect element, cluster analysis is performed on each defect element to construct a defect subgraph set;

[0110] The defect subgraph set is classified using a graph classification algorithm;

[0111] The defects of the mesh product are classified according to the defect subgraph set after classification.

[0112] It should be noted that clustering analysis of each defect element based on the adjacency relationship between each defect element and constructing a defect subgraph set can be performed by clustering analysis of each defect element based on the adjacency relationship between each defect element, grouping multiple defect elements that are close to each other into the same category, constructing a defect subgraph based on the defect elements in the same category, thereby obtaining at least one defect subgraph, and aggregating the obtained defect subgraphs to obtain a defect subgraph set.

[0113] In practical applications, graph classification algorithms can be used to classify the defective subgraphs in the defective subgraph set by defect level, determine the defect level corresponding to each defective subgraph in the defective subgraph set, and then classify the overall defects of the mesh product based on the defect level corresponding to each defective subgraph in the defective subgraph set.

[0114] For example: Suppose that the defect subgraphs in the defect subgraph set include three types, A, B, and C, and their corresponding defect levels are C3, C4, and C3, respectively. Then, the highest defect level C4 of the defect subgraphs can be used as the overall defect classification of the mesh product.

[0115] Alternatively, the overall defect score of the mesh product can be determined based on the number of defect subgraphs and the defect level of each defect subgraph. The defect classification of the mesh product can then be determined based on the defect score (e.g., if there is one C3 defect in the mesh product, the defect score is increased by 10 points; if there is one C4 defect, the defect score is increased by 20 points. If the overall defect score is less than 20, the defect classification is extremely low; if the overall defect score is greater than 40, the defect classification is medium; if the overall defect score is greater than 60, the defect classification is high).

[0116] To facilitate understanding, an example is provided below, but it does not limit the scope of this solution. For example: an industrial sensor (e.g., CCD camera, 3D laser scanner, etc.) connected to defect detection equipment acquires an image of the mesh product. Based on the mesh template, a template matching algorithm is used to perform coarse registration of each mesh in the product image, determining the initial mesh position of each mesh. Based on the initial mesh position, the initial mesh center point is determined. The difference between the neighborhood features of the initial mesh center point and the mesh template is quantified. A gradient descent algorithm is then used to complete the fine registration of each mesh image, thereby determining the calibration mesh position of each mesh. The construction of the gradient descent algorithm can be referenced... Figure 5 First, a difference metric function is constructed to measure the neighborhood features between the mesh template and the initial mesh center point. This difference metric function can calculate the average gray level and gray level variance within the region. The value calculated by the difference metric function in the neighborhood of the initial mesh center point is denoted as I_middle. The initial mesh center point is shifted one unit to the left, and the neighborhood changes accordingly. The value calculated by the difference metric function in the neighborhood of the initial mesh center point after the left shift is denoted as I_left, the value after the right shift is I_right, the value after the upward shift is I_top, and the value after the downward shift is I_bottom. Thus, we obtain:

[0117] dx = (I_right - I_left) / 2

[0118] dy = (I_bottom - I_top) / 2

[0119] The gradient descent algorithm can then be expressed as:

[0120]

[0121] The point with the smallest gradient value is selected, and the point corresponding to the smallest gradient value is determined as the center point of the mesh. Each mesh image is used as a node, and the lines connecting the mesh center points are used as edges. Edges longer than a preset edge length threshold are removed to obtain a directed graph of the product image to be inspected. Considering the large mesh structure of the mesh product, an adjacency list is used to represent the directed graph structure. The dual graph of the directed graph is calculated to obtain the dual graph corresponding to the product image to be inspected. Each edge in the dual graph is rotated to a horizontal state, and then the endpoints of the two edges are used as regions of interest to form an edge image set. A pre-trained classification model is used to score defects in the images in the edge image set. The scoring results are then pulled back to the dual graph to obtain a weighted directed graph. Based on the weighted directed graph... Edge scores determine defective elements, and the defect level of each mesh is determined based on the adjacency relationship of the defective elements. For example, if the defect score of an edge in a weighted directed graph is greater than 7, the corresponding edge is considered defective and marked as a defective element. Suppose the defect scores of edges A, B, C, and D in the weighted directed graph are 8, 9, 7, and 8, respectively. Then, the defective element set contains defective elements A, B, C, and D. At this time, cluster analysis is performed on the defective elements based on the adjacency relationship between them. A, B, C, and D belong to the same defective subgraph, and the defect level of this defective subgraph is determined to be C3. At this time, the overall defect classification of the mesh product can be further performed based on the defect level of the defective subgraph.

[0122] This embodiment determines defect elements based on the defect scores corresponding to each edge in the weighted directed graph; determines the adjacency relationships between each defect element based on the topology of the weighted directed graph; and performs defect classification based on the adjacency relationships. Since the adjacency relationships between each defect element are obtained after determining the defect elements, and defect classification is performed based on these relationships, it ensures that overall defect analysis can be performed based on the actual distribution location of each defect element, which is more consistent with the actual mesh defect scenarios and meets practical defect detection needs.

[0123] Furthermore, embodiments of the present invention also propose a storage medium storing a defect detection program, wherein the defect detection program, when executed by a processor, implements the steps of the defect detection method described above.

[0124] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the defect detection device of the present invention.

[0125] like Figure 6 As shown, the defect detection device proposed in this embodiment of the invention includes:

[0126] Construction module 10 is used to perform mesh recognition on the product image of the mesh product, and construct a directed graph with the recognized mesh as nodes. The directed graph is represented by an adjacency list.

[0127] Calculation module 20 is used to calculate the dual of the directed graph to obtain the dual graph;

[0128] The segmentation module 30 is used to set the region of interest based on the edges in the dual graph, segment the product image, and obtain an edge image set;

[0129] The scoring module 40 is used to score each image in the edge image set using a pre-trained classification model, and pull the scoring results back to the dual graph to obtain a weighted directed graph.

[0130] The grading module 50 is used to grade defects based on the weighted directed graph.

[0131] This embodiment performs mesh recognition on product images of mesh products, constructing a directed graph using the identified mesh holes as nodes; calculates the dual graph to obtain a dual graph; sets regions of interest based on the edges in the dual graph, segments the product image, and obtains an edge image set; uses a pre-trained classification model to score defects in each image in the edge image set, and pulls the scoring results back into the dual graph to obtain a weighted directed graph; and performs defect classification based on the weighted directed graph. This invention indirectly locates the edges of mesh holes in mesh product images through the dual graph, generates an edge image set, scores defects in each edge image in the edge image set using a classification model to obtain a weighted directed graph, and then performs defect classification based on the weighted directed graph, ensuring automated defect detection of mesh products and improving the robustness and accuracy of defect detection for mesh products.

[0132] Furthermore, the construction module 10 is also used to acquire a product image of the mesh product; perform multi-level registration on the product image to determine the calibration mesh position corresponding to each mesh in the product image; and construct a directed graph based on the calibration mesh position corresponding to each mesh.

[0133] Furthermore, the construction module 10 is also used to locate the initial mesh position of each mesh in the product image through a preset template matching algorithm; and to calibrate the initial mesh position by constructing a gradient through the central difference method and then performing gradient descent to obtain the calibrated mesh position corresponding to each mesh.

[0134] Furthermore, the construction module 10 is also used to determine the center coordinates of each mesh based on the calibration mesh position of each mesh; and to construct a directed graph corresponding to the product image based on the center coordinates of each mesh and a preset side length threshold.

[0135] Furthermore, the scoring module 40 is also used to rotate each edge image in the edge image set and construct a sample set based on the processed edge images; score each edge image in the sample set using a pre-trained classification model to obtain a scoring result set; and pull the scores corresponding to each edge image in the scoring result set back to the dual graph to obtain a weighted directed graph.

[0136] Furthermore, the grading module 50 is also used to determine defect elements based on the defect scores corresponding to each edge in the weighted directed graph; determine the adjacency relationship between each defect element based on the topology of the weighted directed graph; and perform defect grading based on the adjacency relationship.

[0137] Furthermore, the grading module 50 is also used to perform cluster analysis on each defect element according to the adjacency relationship between each defect element to construct a defect subgraph set; classify the defect subgraph set using a graph classification algorithm; and grade the defects of the mesh product according to the classified defect subgraph set.

[0138] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0139] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0140] In addition, for technical details not described in detail in this embodiment, please refer to the defect detection method provided in any embodiment of the present invention, which will not be repeated here.

[0141] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0142] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0144] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A defect detection method, characterized in that, The method includes: Mesh recognition is performed on the product image of the mesh product, and a directed graph is constructed using the recognized mesh holes as nodes. The directed graph is represented by an adjacency list. Calculate the dual of the directed graph to obtain the dual graph; Using the edges in the dual graph as a reference, a region of interest is set, and the product image is segmented to obtain an edge image set; The pre-trained classification model scores each image in the edge image set, and the scoring results are then pulled back to the dual graph to obtain a weighted directed graph. Defect classification is performed based on the weighted directed graph; The step of performing mesh hole recognition on the product image of the mesh product, and constructing a directed graph using the recognized mesh holes as nodes, includes: Obtain product images of mesh products; The product image is registered at multiple levels to determine the position of the calibration mesh corresponding to each mesh in the product image. A directed graph is constructed based on the calibration mesh positions corresponding to each mesh. The construction of a directed graph based on the calibration mesh positions corresponding to each mesh includes: The center coordinates of each mesh are determined based on the calibration mesh position of each mesh. A directed graph corresponding to the product image is constructed based on the center coordinates of each mesh and a preset side length threshold.

2. The defect detection method as described in claim 1, characterized in that, The step of performing multi-level registration on the product image to determine the position of the calibration mesh corresponding to each mesh in the product image includes: The initial mesh position of each mesh in the product image is located by a preset template matching algorithm; The initial mesh positions are calibrated by constructing a gradient using the central difference method and then performing gradient descent to obtain the calibrated mesh positions corresponding to each mesh.

3. The defect detection method as described in claim 1, characterized in that, The classification model, obtained through pre-training, scores each image in the edge image set, and then pulls the scoring results back to the dual graph to obtain a weighted directed graph, including: Rotate each edge image in the edge image set, and construct a sample set based on the processed edge images; The pre-trained classification model is used to score each edge image in the sample set to obtain a score result set; The scores corresponding to each edge image contained in the scoring result set are pulled back to the dual graph to obtain a weighted directed graph.

4. The defect detection method as described in claim 1, characterized in that, The defect classification based on the weighted directed graph includes: Defect elements are determined based on the defect scores corresponding to each edge in the weighted directed graph. Based on the topology of the weighted directed graph, determine the adjacency relationships between each defective element; Defects are classified based on the aforementioned adjacency relationships.

5. The defect detection method as described in claim 4, characterized in that, The defect classification based on the adjacency relationship includes: Based on the adjacency relationships between each defect element, cluster analysis is performed on each defect element to construct a defect subgraph set; The defect subgraph set is classified using a graph classification algorithm; The defects of the mesh product are classified according to the defect subgraph set after classification.

6. A defect detection device, characterized in that, The device includes: A construction module is used to perform mesh recognition on product images of mesh products, and to construct a directed graph using the recognized meshes as nodes. The directed graph is represented by an adjacency list. The calculation module is used to calculate the dual of the directed graph to obtain the dual graph; The segmentation module is used to set the region of interest based on the edges in the dual graph, segment the product image, and obtain an edge image set; The scoring module is used to score each image in the edge image set using a pre-trained classification model, and pull the scoring results back to the dual graph to obtain a weighted directed graph. A grading module is used to grade defects based on the weighted directed graph; The construction module is also used to acquire a product image of the mesh product; perform multi-level registration on the product image to determine the calibration mesh position corresponding to each mesh in the product image; and construct a directed graph based on the calibration mesh position corresponding to each mesh. The construction module is further configured to determine the center coordinates of each mesh based on the calibration mesh position of each mesh; and to construct a directed graph corresponding to the product image based on the center coordinates of each mesh and a preset side length threshold.

7. A defect detection device, characterized in that, The device includes: a memory, a processor, and a defect detection program stored in the memory and executable on the processor, the defect detection program being configured to implement the steps of the defect detection method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a defect detection program, which, when executed by a processor, implements the steps of the defect detection method as described in any one of claims 1 to 5.

Images