Defect detection method and apparatus, and electronic device and storage medium
By using pre-trained deep learning models and semantic processing techniques, and utilizing feature distance thresholds to detect image defects, this method solves the problem of low efficiency in traditional detection methods, achieving efficient and accurate defect detection applicable to a variety of industrial products.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- RUIJIE NETWORKS CO LTD
- Filing Date
- 2025-09-29
- Publication Date
- 2026-07-30
AI Technical Summary
Traditional defect detection relies on manual inspection, which is inefficient. Existing automated inspection methods require a large amount of sample data annotation and are difficult to effectively detect both simple and complex product surface defects simultaneously.
It employs a pre-trained deep learning model to extract image features through a semantic processing model, and uses feature distance thresholds to determine whether there are defects in the image, reducing the need for sample data and making it suitable for both simple and complex surface detection.
It improves defect detection efficiency, reduces sample data annotation time, and enables efficient and accurate defect detection with minimal data, making it suitable for a variety of industrial products.
Smart Images
Figure CN2025125484_30072026_PF_FP_ABST
Abstract
Description
Defect detection methods, devices, electronic equipment and storage media
[0001] Cross-references to related applications
[0002] This application claims priority to Chinese Patent Application No. 202510120776.5, filed on January 24, 2025, entitled "Defect Detection Method, Apparatus, Electronic Device, Storage Medium and Product", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to a defect detection method, apparatus, electronic device, and storage medium, belonging to the field of computer technology applications. Background Technology
[0004] Defect detection refers to the process of inspecting the surface of products to screen out those with surface quality defects and prevent them from entering the market. Traditionally, this inspection is mainly done manually, but this method suffers from low efficiency. Currently, it often requires labeling a large amount of sample data for a specific product, using this labeled sample data to train a defect detection model, and then using the trained defect detection model to inspect the product surface. However, this method still requires a large amount of manual labeling of sample data and still suffers from low inspection efficiency. Summary of the Invention
[0005] This application provides a defect detection method, apparatus, electronic device, storage medium, and product.
[0006] In a first aspect, exemplary embodiments of this application provide a defect detection method, including:
[0007] Acquire the image to be detected, which is collected for the object to be detected; and
[0008] Determining whether the image to be detected has defects includes the presence of defects at at least one location in the feature space corresponding to the first semantic. Whether the object to be detected has defects at the location corresponding to the feature space of the first semantic is based on whether the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic. The feature distance threshold is the maximum feature distance of multiple normal images at the first semantic.
[0009] In some implementations, it also includes:
[0010] The reference depth semantic features of the reference image are obtained, and the first depth semantic features of the image to be detected are determined. The reference image is one of multiple normal images. The normal image is collected for defect-free objects. The object to be detected and the defect-free object belong to the same type of object. The reference depth semantic features include the depth features and semantic segmentation features of the reference image. The first depth semantic features include the depth features and semantic segmentation features of the image to be detected.
[0011] Determine the target feature distances of the first deep semantic features and reference deep semantic features in multiple first semantic features in the image to be detected, with each target feature distance corresponding to a first semantic feature.
[0012] In some implementations, determining whether an image to be detected has defects includes:
[0013] For each of the multiple first semantics, if the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic, it is determined that there is a defect in the image to be detected at the corresponding position in the feature space of the first semantic.
[0014] In some implementations, determining the first deep semantic features of the image to be detected includes: extracting semantic features from the image to be detected using a semantic processing model to obtain the first deep semantic features.
[0015] In some implementations, the semantic processing model includes a semantic matching and segmentation unit, a feature extraction unit, and a combination unit. The semantic processing model extracts semantic features from the image to be detected to obtain first deep semantic features, including:
[0016] Obtain the reference semantic segmentation features of the reference image;
[0017] The semantic segmentation unit matches the reference semantic segmentation features, performs semantic segmentation on the image to be detected, and obtains the first semantic segmentation features.
[0018] The feature extraction unit extracts features from the image to be detected, thereby obtaining the first depth features of the image to be detected.
[0019] The first deep semantic feature is obtained by combining the first semantic feature and the first deep feature by combining the unit.
[0020] In some implementations, it also includes:
[0021] Acquire multiple normal images;
[0022] Determine the maximum feature distance between multiple normal images and their different first semantics in multiple first semantics;
[0023] The maximum feature distance is determined as the feature distance threshold corresponding to the first semantic.
[0024] In some implementations, determining the maximum feature distance between multiple normal images across multiple first semantics includes:
[0025] Based on the semantic processing model, semantic features are extracted from multiple normal images to obtain the second deep semantic features of each normal image.
[0026] For every two normal images, the distance between the first features under different first semantics is determined based on the second deep semantic features of the two normal images.
[0027] For the same semantic meaning, the largest first feature distance among multiple first feature distances is determined as the maximum feature distance.
[0028] In some implementations, the multiple normal images include: a reference image and other normal images. The semantic processing model includes: a segmentation unit, a semantic matching segmentation unit, a feature extraction unit, and a combination unit. Semantic features are extracted from the multiple normal images according to the semantic processing model to obtain the second deep semantic features of each normal image, including:
[0029] The reference image is semantically segmented by the segmentation unit to obtain reference semantic segmentation features, which are the second semantic segmentation features of the reference image.
[0030] By matching the reference semantic segmentation features through the semantic matching segmentation unit, semantic segmentation processing is performed on other normal images to obtain the second semantic segmentation features.
[0031] The feature extraction unit extracts features from multiple normal images to obtain the second depth features of each normal image.
[0032] For each normal image, the second semantic feature and the second depth feature of the normal image are combined by the combination unit to obtain the second depth semantic feature of the normal image.
[0033] In some implementations, the second depth feature includes the positional features of each pixel in the normal image.
[0034] Firstly, this application also provides a defect detection method, comprising:
[0035] Acquire an image to be detected, wherein the image to be detected is acquired for the object to be detected; and
[0036] If the target feature distance of the object to be detected at at least one first semantic is greater than the feature distance threshold corresponding to the at least one first semantic, then the image to be detected is determined to have a defect, wherein the feature distance threshold is the maximum feature distance of multiple normal images of the object to be detected at the first semantic of the object to be detected.
[0037] Thirdly, this application also provides a defect detection device, comprising:
[0038] The acquisition module is used to acquire the image to be detected, which is collected from the object to be detected;
[0039] The determination module is used to determine whether the image to be detected has defects. The defects in the image to be detected include the presence of defects at at least one location in the feature space corresponding to the first semantic. Whether the object to be detected has defects at the location in the feature space corresponding to the first semantic is determined based on whether the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic. The feature distance threshold is the maximum feature distance of multiple normal images at the first semantic.
[0040] Fourthly, this application provides an electronic device, including: a processor and a memory;
[0041] The memory stores the instructions that the computer executes;
[0042] The processor executes computer execution instructions stored in memory, causing the processor to perform any of the defect detection methods in the first aspect.
[0043] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the defect detection method as described in any of the first aspects.
[0044] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements a defect detection method as described in any of the first aspects.
[0045] The defect detection method provided in this application includes: acquiring an image to be detected, wherein the image to be detected is collected for the object to be detected; determining whether the image to be detected has a defect, wherein the presence of a defect in the image to be detected includes the presence of a defect at at least one location corresponding to a feature space of a first semantic, and whether the presence of a defect at the location corresponding to the feature space of the first semantic is based on whether the target feature distance of the first semantic is greater than a feature distance threshold corresponding to the first semantic, wherein the feature distance threshold is the maximum feature distance of multiple normal images at the first semantic, and this application can realize defect detection of the object to be detected based on the feature distance threshold determined by multiple normal images, thereby improving the efficiency of defect detection. Attached Figure Description
[0046] The above and other objects, features, and advantages of embodiments of this application will become more readily understood through the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application will be described by way of example and non-limitation, wherein:
[0047] Figure 1 is an application scenario diagram of a defect detection method according to an embodiment of this application;
[0048] Figure 2 is a flowchart of a defect detection method according to an embodiment of this application;
[0049] Figure 3 is a flowchart of another defect detection method according to an embodiment of this application;
[0050] Figure 4 is a schematic diagram of a semantic processing model processing a normal image according to an embodiment of this application;
[0051] Figure 5 is a schematic diagram of a semantic processing model processing an image to be detected according to an embodiment of this application;
[0052] Figure 6 is a structural block diagram of a defect detection device according to an embodiment of this application;
[0053] Figure 7 is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0054] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0055] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0056] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0057] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between components; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0058] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0059] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0060] Defect detection refers to the process of inspecting the surface of products to screen out those with surface quality defects and prevent them from entering the market. Traditionally, this inspection is mainly done manually, but this method suffers from low efficiency. Currently, it often requires labeling a large amount of sample data for a specific product, using this labeled sample data to train a defect detection model, and then using the trained defect detection model to inspect the product surface. However, this method still requires a large amount of manual labeling of sample data and still suffers from low inspection efficiency.
[0061] Currently known automated defect detection methods mainly suffer from two problems: 1. They require significant time and manual effort to label and collect large amounts of sample data; 2. They perform well on simple product surfaces but poorly on complex ones. However, in real-world scenarios, due to the rapid delivery of industrial products, spending excessive time on sample data collection and labeling can impact product delivery. For industrial products where time allows for data collection and labeling, users also desire to quickly implement automated detection methods. Furthermore, industrial enterprises may produce both simple and complex surface products; therefore, simply being able to detect defects on simple surfaces is insufficient to meet the needs of industrial users.
[0062] This application proposes a pre-trained, general-purpose deep learning model that can detect defects in objects with very limited data on normal objects. Furthermore, the method provided in this application can detect defects on both simple and complex surfaces.
[0063] The application scenario of this application is as follows: for objects of the same type, wherein the object is an object that needs to be surface inspected, such as the fabric shown in Figure 1, or industrial products such as plastic products, metal workpieces, and printed circuit boards (PCBs). The object includes: a defect-free object and an object to be inspected. First, multiple normal images can be acquired for the defect-free object (image 10 of the defect-free fabric shown in Figure 1). Then, an image to be inspected is acquired for the object to be inspected. A feature distance threshold is determined based on the multiple normal images. This feature distance threshold can be used to determine the defect detection of the image to be inspected.
[0064] The defect detection method provided in this application will be described in detail below with reference to specific embodiments.
[0065] Referring to Figure 2, this application provides a flowchart of a defect detection method, which specifically includes the following steps.
[0066] S201, Obtain the image to be detected.
[0067] The image to be detected is acquired from the object to be detected. This image is obtained through image collection methods. Understandably, image collection methods include photographing the surface of the object to be detected, scanning the surface of the object to be detected, etc. This application does not limit the method of obtaining the image to be detected.
[0068] S202, determine whether there are defects in the image to be detected.
[0069] Among them, the defects in the image to be detected include the presence of defects in the object to be detected at at least one location of the first semantic. Whether the object to be detected has defects at the location of the first semantic is based on whether the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic. The feature distance threshold is the maximum feature distance of multiple normal images at the first semantic.
[0070] Understandably, the location of the first semantic refers to the location corresponding to the feature space of the first semantic.
[0071] In this embodiment, the first semantic meaning represents the meaning of the content of the image to be detected. The first semantic meaning can be represented by natural language or symbolic language. For example, if the image to be detected includes a flower pattern, the first semantic meaning can be the name, location, etc. of the flower pattern. It can also be represented by the numerical symbol "①".
[0072] The image to be detected may have multiple first semantics. For example, for the fabric image to be detected in Figure 1, there are multiple categories of flowers, as well as tree trunks and blank areas. Each flower can be used as the semantics of the area where the flower is located, the tree trunk as the semantics of the area where the tree trunk is located, and the blank area as the semantics of the area where the blank area is located. In the embodiments of this application, semantics can be represented by semantic features, and there is no limitation on this.
[0073] In this embodiment, the feature distance threshold is the maximum feature distance between multiple normal images at the semantic level.
[0074] The feature distance threshold is predetermined based on normal images. The feature distance between normal images is less than or equal to the feature distance threshold. This means that features with the same semantic meaning are clustered together in normal images. If the target feature distance is greater than the feature distance threshold, it can be understood that the image to be detected is significantly different from the normal image, and thus it can be determined that the image to be detected has defects.
[0075] In summary, the exemplary embodiments of this application perform defect detection based on feature distance thresholds, which can avoid both missed and false detections. Furthermore, by determining the feature distance threshold using normal images, defect detection of the target object can be performed even with very limited data on normal objects. Additionally, the method of this application can perform defect detection on both simple and complex surfaces.
[0076] Referring to Figure 3, a flowchart of another defect detection method provided in this application embodiment is shown, which specifically includes the following steps.
[0077] S301, acquire multiple normal images.
[0078] In this embodiment, the multiple normal images are obtained from defect-free objects. The number of normal images can be determined as needed, such as 2 to 8.
[0079] S302, determine the maximum feature distance between multiple normal images in multiple first semantics.
[0080] For example, in an embodiment of this application, a normal image includes 10 first semantics, and a maximum feature distance can be determined for each of the 10 first semantics.
[0081] The determination of the maximum feature distance of multiple normal images in any one of multiple first semantics includes: extracting semantic features from multiple normal images according to a semantic processing model to obtain the second deep semantic features of each normal image, wherein the second deep semantic features of each normal image may include the depth features and semantic segmentation features of the corresponding normal image; for every two normal images, determining the first feature distances included in different first semantics in the two normal images according to the second deep semantic features of the two normal images; and for the same semantic, determining the largest first feature distance among the multiple first feature distances corresponding to the multiple normal images as the maximum feature distance.
[0082] In the embodiments of this application, the semantic processing model can be a general neural network model, which can extract the deep semantic features of each normal image.
[0083] For example, multiple normal images include three normal images: normal image A, normal image B, and normal image C. A second deep semantic feature 'a' is obtained for normal image A, a second deep semantic feature 'b' is obtained for normal image B, and a second deep semantic feature 'c' is obtained for normal image C. Multiple first semantic features include: first semantic y1, first semantic y2, first semantic y3, and first semantic y4.
[0084] In one embodiment, firstly, based on the second deep semantic feature a and the second deep semantic feature b, the first feature distances for different first semantics (first semantic y1 to first semantic y4) are determined. For example, the first feature distance L1 for the first semantic y1 includes x 11 The first feature distance L2 of the first semantic y2 includes x. 12 The first feature distance L3 of the first semantic y3 includes x13 The first feature distance L4 of the first semantic y4 includes x 14 indivual.
[0085] Then, based on the second deep semantic feature a and the second deep semantic feature c, the first feature distances for different first semantics (semantics y1 to semantics y4) are determined. For example, the first feature distance L1 for the first semantic y1 includes x. 21 The first feature distance L2 of the first semantic y2 includes x. 22 The first feature distance L3 of the first semantic y3 includes x 23 The first feature distance L4 of the first semantic y4 includes x 24 indivual.
[0086] Furthermore, based on the second deep semantic feature b and the second deep semantic feature c, the first feature distances for different first semantics (first semantics y1 to first semantics y4) are determined. For example, the first feature distance L1 for first semantic y1 includes x 31 The first feature distance L2 of the first semantic y2 includes x. 32 The first feature distance L3 of the first semantic y3 includes x 33 The first feature distance L4 of the first semantic y4 includes x 34 indivual.
[0087] Finally, the number of first feature distances L1 for the first semantic y1 is determined to be (x 11 +x 21 +x 31 ), then in (x 11 +x 21 +x 31 The largest first feature distance among the first feature distances L1 is determined as the largest feature distance of the first semantic y1, such as the largest feature distance T1. The number of first feature distances L2 for determining the first semantic y2 is (x 12 +x 22 +x 32 ), then in (x 12 +x 22 +x 32 The largest first feature distance among the first feature distances L2 is determined as the largest feature distance of the first semantic y2, such as the largest feature distance T2. The number of first feature distances L3 for determining the first semantic y3 is (x 13 +x 23 +x 33 ), then in (x 13 +x 23 +x 33The largest first feature distance among the first feature distances L3 is determined as the largest feature distance of the first semantic y3, such as the largest feature distance T3. The number of first feature distances L4 used to determine the first semantic y4 is (x... 14 +x 24 +x 34 ), then in (x 14 +x 24 +x 34 The largest first feature distance among the first feature distances L4 is determined as the largest feature distance of the first semantic y4, such as the largest feature distance T4.
[0088] It is evident that this application can determine the first feature distance between any two normal images under different first semantics, and determine the feature distance threshold for each first semantic based on multiple first feature distances. Specifically, the first feature distance between normal images is always less than or equal to this feature distance threshold.
[0089] Further, the multiple normal images include: a reference image and other normal images. Referring to Figure 4, the semantic processing model 40 includes: a segmentation unit 41, a semantic matching segmentation unit 42, a feature extraction unit 43, and a combination unit 44. The semantic processing model extracts semantic features from the multiple normal images to obtain the second deep semantic features of each normal image. This includes: performing semantic segmentation on the reference image through the segmentation unit to obtain reference semantic segmentation features, which are the second semantic segmentation features of the reference image; matching the reference semantic segmentation features with the semantic matching segmentation unit to perform semantic segmentation on the other normal images to obtain second semantic segmentation features; extracting features from each normal image through the feature extraction unit to obtain the second deep features of each normal image; and combining the second semantic features and the second deep features of each normal image through the combination unit to obtain the second deep semantic features of the normal image.
[0090] It is understood that terms such as "semantic segmentation," "semantics," and "feature extraction" in this application can be understood by those skilled in the art and will not be elaborated upon here.
[0091] In semantic segmentation of images, "semantics" refers to the machine's "category-level interpretation" of image content. By annotating pixels one by one, low-level visual signals are transformed into high-level concepts that humans can understand (such as "this is a cat" or "that is the background"). Its core goal is to enable computers to "understand" the "meaning" of each part of an image, providing structured semantic information for subsequent intelligent decision-making (such as navigation, diagnosis, and quality inspection).
[0092] The "feature space" in "semantic feature space" is the mathematical carrier of semantic information. It is a mathematical space composed of high-dimensional vectors, where each dimension corresponds to a feature with semantic meaning, and each point (i.e., vector) in the space uniquely represents the high-level semantic information of an image, image region, or target. Its core is to transform raw visual data (such as image pixels) into a quantifiable and computable vector form through a feature extraction model, enabling machines to understand semantic concepts through mathematical relationships between vectors (such as distance and similarity).
[0093] It is understood that the reference image in this application is one of a plurality of normal images randomly sampled from among multiple normal images. For example, the plurality of normal images includes: normal image A, normal image B, and normal image C. The reference image is normal image A. Among the plurality of normal images, the other normal images are those other than the reference image; for example, the other normal images include normal image B and normal image C.
[0094] In one embodiment, the segmentation unit may include: a pre-trained Segment Anything Model (SAM) network; the pre-trained SAM network performs image segmentation on the reference image to obtain reference semantic segmentation features (second semantic segmentation features of the reference image).
[0095] It is understood that this application performs semantic segmentation on a random normal image from multiple normal images by pre-training a general segmentation unit 41, and the obtained reference semantic segmentation features can be used as a reference to provide a benchmark for subsequent semantic matching.
[0096] In this embodiment, the semantic matching segmentation unit can employ the DeAOT (Descriptive Asynchronous Object) algorithm. The semantic matching segmentation unit can match reference semantic segmentation features to perform semantic segmentation on other normal images, obtaining a second semantic segmentation feature for each of the other normal images.
[0097] It is understood that the semantic matching segmentation unit 42 of this application can ensure that targets with the same semantic meaning remain consistent at the pixel level through a pre-trained multi-target tracking algorithm, thereby completing target semantic segmentation and semantic matching for other normal images.
[0098] In addition, the feature extraction unit 43 uses feature engineering techniques and a pre-trained deep feature extraction network to construct a series of features related to key representations, providing high-quality comparative data for defect detection.
[0099] It is understandable that the feature extraction unit 43 can use a deep self-supervised pre-trained DINOv2 (a self-supervised model for computer vision) network to extract deep features from each of the multiple normal images to obtain the second deep features of each normal image.
[0100] The second depth feature includes the positional features of each pixel in the normal image.
[0101] Finally, for each normal image, the unit combines the second depth feature with location features and the second semantic segmentation feature to obtain the second depth semantic feature.
[0102] It is understandable that combining the second deep features with the second semantic segmentation features can ensure that the feature comparison is not interfered with by different first semantics, thereby improving the defect detection capability of complex images.
[0103] For example, the multiple normal images include: normal image A, normal image B, and normal image C. Normal image A is a reference image, and the other normal images include: normal image B and normal image C. The reference image (normal image A) is processed by segmentation unit 41 to obtain reference semantic segmentation feature a1. Reference semantic segmentation feature a1 and normal image B are processed by semantic matching segmentation unit 42 to obtain second semantic segmentation feature b1. Reference semantic segmentation feature a1 and normal image C are processed by semantic matching segmentation unit 42 to obtain second semantic segmentation feature c1. Normal image A is processed by feature extraction unit 43 to obtain second depth feature a2, normal image B is processed by feature extraction unit 43 to obtain second depth feature b2, normal image C is processed by feature extraction unit 43 to obtain second depth feature c2. Then, referring to semantic segmentation feature a1 and second depth feature a2, they are combined by combination unit to obtain second depth semantic feature a, second semantic segmentation feature b1 and second depth feature b2 are combined by combination unit to obtain second depth semantic feature b, and second semantic segmentation feature c1 and second depth feature c2 are combined by combination unit to obtain second depth semantic feature c.
[0104] S303, determine the maximum feature distance as the feature distance threshold corresponding to the first semantic.
[0105] It is understood that in this application, the maximum feature distance that can accurately reflect the degree of abnormality of a normal image is found through cross-validation selection in the constructed training feature set, and is used as the feature distance threshold, so that the defect detection method of this application has good generalization ability and applicability on various complex surfaces.
[0106] For example, the maximum feature distance T1 is the feature distance threshold for semantic y1. The maximum feature distance T2 is the feature distance threshold for semantic y2. The maximum feature distance T3 is the feature distance threshold for semantic y3. The maximum feature distance T4 is the feature distance threshold for semantic y4.
[0107] S304, Obtain the reference depth semantic features of the reference image and determine the first depth semantic features of the image to be detected.
[0108] The reference image is one of several normal images, which are acquired for defect-free objects. The object to be detected and the defect-free object belong to the same type of object. The reference deep semantic features include the depth features and semantic segmentation features of the reference image, and the first deep semantic features include the depth features and semantic segmentation features of the image to be detected.
[0109] It can be understood that the reference deep semantic features include: the depth features and semantic segmentation features of the reference image. The first deep semantic features include: the depth features and semantic segmentation features of the image to be detected.
[0110] The reference image is any one of multiple normal images. The normal images are acquired for defect-free objects. The objects to be detected and the defect-free objects are of the same type.
[0111] In the embodiments of this application, objects of the same type can be understood as product objects of the same batch on the production line.
[0112] This process involves capturing multiple normal images of a defect-free object, and then selecting one of these normal images as a reference image. Furthermore, the reference depth semantic features of the reference image can be predetermined; in this embodiment, these features can be directly obtained.
[0113] In one embodiment, the first depth semantic feature of the image to be detected includes: depth features and semantic features of the image to be detected.
[0114] In one embodiment, determining the first deep semantic features of the image to be detected includes: extracting semantic features from the image to be detected using a semantic processing model to obtain the first deep semantic features.
[0115] Referring to Figure 5, the semantic processing model 40 includes a semantic matching and segmentation unit 42, a feature extraction unit 43, and a combination unit 44. The semantic processing model extracts semantic features from the image to be detected to obtain first deep semantic features, including: obtaining reference semantic segmentation features of a reference image; matching the reference semantic segmentation features with the semantic matching and segmentation unit to perform semantic segmentation processing on the image to be detected to obtain first semantic segmentation features; extracting features from the image to be detected by the feature extraction unit to obtain first deep features of the image to be detected; and combining the first semantic features and the first deep features by the combination unit to obtain first deep semantic features.
[0116] It can be understood that the semantic processing model for processing the image to be detected and the semantic processing model for processing normal images are the same model.
[0117] Furthermore, in the above steps, reference semantic segmentation features are obtained by segmenting the reference image according to the segmentation unit 41, and these reference semantic segmentation features are used to determine the first semantic segmentation features.
[0118] The determination of the first deep semantic feature is based on the second deep semantic feature mentioned above, and will not be repeated here.
[0119] S305, determine the target feature distance between the first deep semantic feature and the reference deep semantic feature in multiple first semantic features in the image to be detected.
[0120] In the embodiments of this application, multiple target feature distances can be determined for each first semantic, and each target feature distance can refer to the target feature distance of a pixel in the first semantic.
[0121] It is understood that the target feature distance in this application can be Mahalanobis distance, Euclidean distance, or other distances, and there is no limitation on this.
[0122] For example, the first semantics in the image to be detected include: semantic y1, semantic y2, semantic y3, and semantic y4. Among them, semantic y1 corresponds to 50 pixels, and a target feature distance can be determined for each pixel.
[0123] In one embodiment, the target feature distance corresponding to the i-th pixel of semantic y1 is Xi.
[0124] S306, for each of the multiple first semantics, if the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic, it is determined that there is a defect at the semantic location in the image to be detected.
[0125] In this embodiment, the feature distance threshold is the maximum feature distance between multiple normal images at the first semantic location.
[0126] For example, referring to the above, if semantic y1 corresponds to the feature distance threshold T1, then if the target feature distance Xi is greater than the feature distance threshold T1, it can be determined that semantic y1 has a defect at the i-th pixel.
[0127] In summary, this application achieves defect detection with only a small number of normal images, significantly reducing the time required for sample data annotation and collection, and is suitable for the cold start phase of object detection when data is scarce. Furthermore, through semantic matching segmentation, we achieve high-precision defect detection and localization at the pixel level. Compared to existing technologies that use large models to segment each image, semantic matching segmentation improves the efficiency of determining feature distance thresholds, enabling it to solve defect detection problems with strong generalization and robustness. Moreover, this application requires only a monocular camera. Through a defect detection method with generalization capabilities, it can detect defects not only in standard industrial products such as textiles and PCB boards, but also in everyday items such as lunch boxes and beverage bottles. It is understood that this application integrates adaptive thresholding, enabling the entire defect detection method to achieve end-to-end defect detection without any manual intervention. Finally, this application achieves defect detection from both uniform and complex patterns, and is compatible with both, greatly expanding the detection range of the defect detection method.
[0128] Figure 6 is a structural block diagram of a defect detection device provided in an embodiment of this application. The defect detection device 60 may include the following modules:
[0129] The acquisition module 61 is used to acquire the image to be detected, which is collected for the object to be detected;
[0130] The determination module 62 is used to determine whether the image to be detected has defects. The defects in the image to be detected include the presence of defects at at least one location in the feature space corresponding to the first semantic. Whether the object to be detected has defects at the location in the feature space corresponding to the first semantic is based on whether the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic. The feature distance threshold is the maximum feature distance of multiple normal images at the first semantic.
[0131] In an optional embodiment, the determining module 62 is further configured to:
[0132] The reference depth semantic features of the reference image are obtained, and the first depth semantic features of the image to be detected are determined. The reference image is one of multiple normal images. The normal image is collected for defect-free objects. The object to be detected and the defect-free object belong to the same type of object. The reference depth semantic features include the depth features and semantic segmentation features of the reference image. The first depth semantic features include the depth features and semantic segmentation features of the image to be detected.
[0133] Determine the target feature distances of the first deep semantic features and reference deep semantic features in multiple first semantic features in the image to be detected, with each target feature distance corresponding to a first semantic feature.
[0134] In one optional embodiment, the determining module 62 is specifically used to determine that the image to be detected has a defect at the semantic location if the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic for each of the plurality of first semantics, and the feature distance threshold is the maximum feature distance of the plurality of normal images at the semantic location.
[0135] In an optional embodiment, when determining the first deep semantic feature of the image to be detected, the determining module 62 is specifically used for:
[0136] The semantic processing model is used to extract semantic features from the image to be detected, thus obtaining the first deep semantic features.
[0137] In one optional embodiment, the semantic processing model includes: a semantic matching and segmentation unit, a feature extraction unit, and a combination unit. When the determination module 62 extracts semantic features from the image to be detected using the semantic processing model to obtain first deep semantic features, it is specifically used for:
[0138] Obtain the reference semantic segmentation features of the reference image;
[0139] The semantic segmentation unit matches the reference semantic segmentation features, performs semantic segmentation on the image to be detected, and obtains the first semantic segmentation features.
[0140] The feature extraction unit extracts features from the image to be detected, thereby obtaining the first depth features of the image to be detected.
[0141] The first deep semantic feature is obtained by combining the first semantic feature and the first deep feature by combining the unit.
[0142] In an optional embodiment, the determining module 62 is further configured to:
[0143] Acquire multiple normal images;
[0144] Determine the maximum feature distance between multiple normal images and their different first semantics in multiple first semantics;
[0145] The maximum feature distance is determined as the feature distance threshold corresponding to the first semantic.
[0146] In an optional embodiment, when determining the maximum feature distance between multiple normal images and different first semantics in multiple first semantics, the determining module 62 is specifically used for:
[0147] Based on the semantic processing model, semantic features are extracted from multiple normal images to obtain the second deep semantic features of each normal image.
[0148] For every two normal images, the distance between the first features under different first semantics is determined based on the second deep semantic features of the two normal images.
[0149] For the same semantic meaning, the largest first feature distance among multiple first feature distances is determined as the maximum feature distance.
[0150] In one optional embodiment, the multiple normal images include: a reference image and other normal images. The semantic processing model includes: a segmentation unit, a semantic matching segmentation unit, a feature extraction unit, and a combination unit. When the determination module 62 extracts semantic features from the multiple normal images according to the semantic processing model to obtain the second deep semantic features of each normal image, it is specifically used to: perform semantic segmentation on the reference image through the segmentation unit to obtain reference semantic segmentation features, wherein the reference semantic segmentation features are the second semantic segmentation features of the reference image.
[0151] By matching the reference semantic segmentation features through the semantic matching segmentation unit, semantic segmentation processing is performed on other normal images to obtain the second semantic segmentation features.
[0152] The feature extraction unit extracts features from multiple normal images to obtain the second depth features of each normal image.
[0153] For each normal image, the second semantic feature and the second depth feature of the normal image are combined by the combination unit to obtain the second depth semantic feature of the normal image.
[0154] In one optional embodiment, the second depth feature includes: the positional features of each pixel in the normal image within the normal image.
[0155] The specific implementation process of the defect detection device provided in this application embodiment can be referred to the defect detection method embodiment, and will not be repeated here.
[0156] To implement the above embodiments, this application also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the defect detection method as described in any of the above embodiments.
[0157] To implement the above embodiments, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the defect detection method as described in any of the above embodiments.
[0158] To implement the above embodiments, this application also provides an electronic device, including: a processor and a memory;
[0159] The memory stores the instructions that the computer executes;
[0160] The processor executes computer execution instructions stored in memory, causing the processor to perform the defect detection method as described in any of the above embodiments.
[0161] Figure 7 is a schematic diagram of the structure of an electronic device provided in an example embodiment of this application. As shown in Figure 7, the electronic device 70 includes: a processor 71, and a memory 72 communicatively connected to the processor 71, the memory 72 storing computer execution instructions.
[0162] The processor executes computer execution instructions stored in the memory to implement the defect detection method provided in any of the above method embodiments. The specific functions and technical effects to be achieved will not be elaborated here.
[0163] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the defect detection method provided in any of the above method embodiments.
[0164] This application also provides a computer program product, which includes a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the defect detection method provided in any of the above method embodiments.
[0165] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.
[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.
[0168] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0169] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0170] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0171] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A defect detection method, comprising: Acquire an image to be detected, wherein the image to be detected is acquired for the object to be detected; and If the target feature distance of the object to be detected at at least one first semantic is greater than the feature distance threshold corresponding to the at least one first semantic, then the image to be detected is determined to have a defect, wherein the feature distance threshold is the maximum feature distance of multiple normal images of the object to be detected at the first semantic of the object to be detected.
2. The defect detection method according to claim 1, wherein, The plurality of normal images includes a reference image and other normal images. Before determining that the image to be detected has a defect if the target feature distance of the object to be detected is greater than the feature distance threshold corresponding to at least one first semantic, the method further includes: Based on the reference image, a reference depth semantic feature of the reference image is obtained, and based on the image to be detected, a first depth semantic feature of the image to be detected is determined, wherein the reference depth semantic feature includes the depth feature and semantic segmentation feature of the reference image, and the first depth semantic feature includes the depth feature and semantic segmentation feature of the image to be detected; and Determine the target feature distance between the first deep semantic feature and the reference deep semantic feature at at least one first semantic location in the image to be detected, wherein the target feature distance corresponds to the first semantic location.
3. The defect detection method according to claim 2, wherein, The determination that the image to be detected has defects includes: For each of the at least one first semantics, if the target feature distance of the first semantic is greater than the feature distance threshold corresponding to the first semantic, it is determined that the image to be detected has a defect at the location of the first semantic.
4. The defect detection method according to claim 2, wherein, The step of determining the first deep semantic feature of the image to be detected based on the image to be detected includes: The semantic features of the image to be detected are extracted by a semantic processing model to obtain the first deep semantic features.
5. The defect detection method according to claim 4, wherein, The semantic processing model includes a semantic matching and segmentation unit, a feature extraction unit, and a combination unit. The step of extracting semantic features from the image to be detected using the semantic processing model to obtain the first deep semantic features includes: Obtain the reference semantic segmentation features of the reference image; The semantic matching segmentation unit matches the reference semantic segmentation features, and performs semantic segmentation processing on the image to be detected to obtain the first semantic segmentation features. The feature extraction unit extracts features from the image to be detected, obtaining the first depth feature of the image to be detected; and The first deep semantic feature is obtained by combining the first semantic feature and the first deep feature by combining the combination unit.
6. The defect detection method according to any one of claims 1 to 5, wherein, Before determining that the image to be detected has a defect if the target feature distance of the object to be detected in at least one first semantic is greater than the feature distance threshold corresponding to the at least one first semantic, the method further includes: Acquire the multiple normal images; Determine the maximum feature distance between the plurality of normal images in the at least one first semantic; and The feature distance threshold corresponding to the first semantic is determined based on the maximum feature distance.
7. The defect detection method according to claim 6, wherein, Determining the maximum feature distance between the multiple normal images in the at least one first semantic meaning includes: Semantic features are extracted from the multiple normal images according to the semantic processing model to obtain the second deep semantic features of each normal image. Based on the second deep semantic features of two normal images among the plurality of normal images, determine the distances of multiple first features under different first semantics; and For the same first semantic, the largest first feature distance among the plurality of first feature distances is determined as the maximum feature distance.
8. The defect detection method according to claim 7, wherein, The plurality of normal images includes: a reference image and other normal images. The semantic processing model includes: a segmentation unit, a semantic matching segmentation unit, a feature extraction unit, and a combination unit. The step of extracting semantic features from the plurality of normal images according to the semantic processing model to obtain the second deep semantic features of each normal image includes: The reference image is semantically segmented by the segmentation unit to obtain the reference semantic segmentation features, wherein the reference semantic segmentation features are the second semantic segmentation features of the reference image; The semantic matching segmentation unit matches the reference semantic segmentation features, performs semantic segmentation processing on the other normal images, and obtains the second semantic segmentation features of the other normal images. The feature extraction unit extracts features from the multiple normal images respectively, obtaining the second depth feature of each normal image; and For each normal image, the second semantic feature and the second depth feature of the normal image are combined by the combination unit to obtain the second depth semantic feature of the normal image.
9. The defect detection method according to claim 8, wherein, The second depth feature includes: the positional features of each pixel in the plurality of normal images within the normal images.
10. A defect detection device, comprising: An acquisition module is used to acquire an image to be detected, wherein the image to be detected is acquired for the object to be detected; The determination module is configured to determine that the image to be detected has a defect if the target feature distance of the object to be detected at at least one first semantic is greater than the feature distance threshold corresponding to the at least one first semantic, wherein the feature distance threshold is the maximum feature distance of multiple normal images of the object to be detected at the first semantic of the object to be detected.
11. An electronic device, comprising: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the defect detection method as described in any one of claims 1 to 9.
12. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the defect detection method as described in any one of claims 1 to 9.