Defect detection method for preventing over-detection and missing detection and electronic equipment

Abstract

The embodiment of the invention discloses a defect detection method for preventing over-detection and missing detection and electronic equipment, and the method comprises the steps: obtaining industrial image samples which are obtained through the image collection of a plurality of stations, carrying out the classification of the industrial image samples, and obtaining an image classification result, the image classification result is used for indicating that the industrial image sample is a defective image sample or a defect-free image sample, training a supervised model based on the defective image sample, training an unsupervised model based on the defect-free image sample, and obtaining a to-be-detected industrial image, and performing defect detection on the to-be-detected industrial image according to the trained supervised model and unsupervised model to obtain a defect detection result, thereby effectively reducing the over-detection rate and the omission rate, and improving the accuracy of defect detection.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a defect detection method and an electronic device for preventing over-detection and under-detection. Background Art

[0002] In practical applications, defect detection is a key link in product quality control. In related technologies, defect detection methods often rely on preset geometric features or gray-scale features to identify defect types. However, such traditional defect detection methods are difficult to adapt to diverse objects to be detected, and are prone to over-detection and under-detection, affecting the accuracy of defect detection. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail in the present disclosure. This overview is not intended to limit the scope of protection of the claims.

[0004] Embodiments of the present disclosure provide a defect detection method for preventing over-detection and under-detection, which can effectively reduce the over-detection rate and the under-detection rate, and improve the accuracy of defect detection.

[0005] On the one hand, embodiments of the present disclosure provide a defect detection method for preventing over-detection and under-detection, including:

[0006] Obtaining an industrial image sample, where the industrial image sample is obtained by image acquisition at multiple workstations;

[0007] Classifying the industrial image sample to obtain an image classification result, where the image classification result is used to indicate that the industrial image sample is a defective image sample or a non-defective image sample;

[0008] Training a supervised model based on the defective image sample, and training an unsupervised model based on the non-defective image sample;

[0009] Obtaining an industrial image to be detected, and performing defect detection on the industrial image to be detected according to the trained supervised model and the unsupervised model to obtain a defect detection result.

[0010] On the other hand, embodiments of the present disclosure further provide an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor implements the above-mentioned defect detection method for preventing over-detection and under-detection when executing the computer program.

[0011] The embodiments of the present disclosure at least include the following beneficial effects: obtaining industrial image samples, and the industrial image samples are obtained by image acquisition at multiple workstations, which can obtain rich industrial image samples, classifying the industrial image samples to obtain an image classification result, and the image classification result is used to indicate whether the industrial image sample is a defective image sample or a non-defective image sample, and training data for subsequent model training can be obtained based on the image classification result. At this time, a supervised model is trained based on the defective image samples, and an unsupervised model is trained based on the non-defective image samples. By combining supervised training and unsupervised training, the detection and recognition capabilities of the supervised model and the unsupervised model for diverse defects can be improved, thereby balancing over-inspection and missed inspection. Therefore, when obtaining an industrial image to be detected and performing defect detection on the industrial image to be detected according to the trained supervised model and unsupervised model to obtain a defect detection result, the over-inspection rate and the missed inspection rate can be effectively reduced, and the accuracy of defect detection can be improved.

[0012] Other features and advantages of the present disclosure will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings are used to provide a further understanding of the technical solutions of the present disclosure, and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solutions of the present disclosure, and do not constitute a limitation to the technical solutions of the present disclosure.

[0014] Figure 1 It is a schematic diagram of an optional implementation environment provided by the embodiments of the present disclosure;

[0015] Figure 2 It is an optional flowchart of a defect detection method for preventing over-inspection and missed inspection provided by the embodiments of the present disclosure;

[0016] Figure 3 It is an optional overall process of the defect detection method for preventing over-inspection and missed inspection provided by the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and are not used to limit the present disclosure.

[0018] It should be noted that in each specific embodiment of the present disclosure, when it comes to performing relevant processing based on data related to the characteristics of the target object, such as target object attribute information or a set of attribute information, the permission or consent of the target object will be obtained first. Moreover, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. Among them, the target object can be a user. In addition, when the embodiments of the present disclosure need to obtain the target object attribute information, a separate permission or separate consent of the target object will be obtained through methods such as a pop-up window or jumping to a confirmation page. After clearly obtaining the separate permission or separate consent of the target object, the necessary data related to the target object for the normal operation of the embodiments of the present disclosure will be obtained.

[0019] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other relevant parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of that module or unit.

[0020] To facilitate the understanding of the technical solutions provided by the embodiments of the present disclosure, some key terms used in the embodiments of the present disclosure are explained here first:

[0021] Over-inspection: Refers to over-detection, which is a behavior of identifying normal (defect-free) parts as defects during the defect detection process. For example, identifying the surface texture of the object to be detected as a defect, resulting in over-inspection.

[0022] Under-inspection: It is a behavior of failing to identify actual existing defects during the defect detection process. For example, the model cannot identify defect types that have not been learned and trained, resulting in under-inspection.

[0023] General type of defect: Refers to defects that are commonly present in the detected products and have similar properties or characteristics, and have universality, typicality, and predictability.

[0024] Non-general type of defect: Refers to defects that are specific to a certain detected product and have unique properties or characteristics. These characteristics can stem from specific business requirements, technical implementations, or environmental factors, etc., and have uniqueness, unpredictability, and complexity.

[0025] In practical applications, defect detection is a key link in product quality control. In related technologies, traditional defect detection methods often rely on preset geometric features or grayscale features to identify defect types, or combine traditional defect detection methods with deep learning to build a detection model for defect detection based on preset features. However, when there are texture regions on the surface of the object to be detected that meet the description of the preset features during the detection process, over-detection is likely to occur, and defect types that do not appear in the training set cannot be identified, resulting in missed detection. Especially when performing batch defect detection, the defects are often complex and diverse, with irregular shapes and unpredictable, and the situations of over-detection and missed detection are particularly obvious, affecting the accuracy of defect detection.

[0026] Based on this, the embodiments of the present disclosure provide a defect detection method for preventing over-detection and missed detection, which can effectively reduce the over-detection rate and missed detection rate and improve the accuracy of defect detection.

[0027] Refer to Figure 1 , Figure 1 which is a schematic diagram of an optional implementation environment provided by the embodiments of the present disclosure. Industrial image samples obtained based on multiple workstations are acquired in the terminal 101, and the industrial image samples are sent to the server 102. The industrial image samples are classified in the server 102 to obtain an image classification result, where the image classification result is used to indicate whether the industrial image sample is a defective image sample or a non-defective image sample. A supervised model is trained based on the defective image samples, and an unsupervised model is trained based on the non-defective image samples. After the training is completed, the industrial image to be detected is acquired from the terminal 101, and the industrial image to be detected is subjected to defect detection according to the trained supervised model and unsupervised model to obtain a defect detection result, and the defect detection result is sent to the terminal 101 for the terminal 101 to make a decision based on the defect detection result.

[0028] It can be understood that the defect detection method for preventing over-detection and missed detection provided by the embodiments of the present disclosure can also be executed alone in the terminal 101.

[0029] Refer to Figure 2 , Figure 2 which is a schematic flow chart of an optional defect detection method for preventing over-detection and missed detection provided by the embodiments of the present disclosure, and specifically may include but is not limited to the following steps S201 to S204:

[0030] Step S201: Acquire industrial image samples.

[0031] Among them, the industrial image samples, as training data for training the model, are obtained by image acquisition at multiple workstations, and the workstations include a stripe workstation, a matrix workstation, and a line scan workstation.

[0032] In a possible implementation, during the process of obtaining industrial image samples, specifically, multiple workstations can be set up around the industrial inspection table, and industrial image samples corresponding to the object to be detected on the industrial inspection table can be obtained based on the stripe workstation, the area array workstation, and the line scan workstation. Among them, the industrial inspection table is used to place the object to be detected, and the object to be detected can be industrial products, such as mechanical parts, electronic product casings, and so on.

[0033] Specifically, the stripe workstation projects a specific stripe pattern onto the surface of the object to be detected using stripe projection technology, the area array workstation obtains the complete image information of the object to be detected through an area array camera, and the line scan workstation obtains the high-precision image information of the object to be detected through a line scan camera. When the detection task is relatively simple, that is, when only the image information on a specific surface or in a specific direction of the object to be detected needs to be obtained, a stripe workstation, an area array workstation, and a line scan workstation can be set up around the industrial inspection table to quickly obtain the local surface information of the object to be detected; when the overall image information of the object to be detected needs to be obtained, multiple stripe workstations, multiple area array workstations, and multiple line scan workstations can be set up around the industrial inspection table to obtain more comprehensive surface information of the object to be detected. Then, based on the stripe workstation, the area array workstation, and the line scan workstation, industrial image samples corresponding to the object to be detected on the industrial inspection table can be obtained, and industrial image samples with rich image information and high quality can be obtained, enabling the sample detection image samples to more accurately capture the surface information of the object to be detected, which helps to improve the accuracy of defect detection.

[0034] In a possible implementation, during the process of obtaining industrial image samples corresponding to the object to be detected on the industrial inspection table based on the stripe workstation, the area array workstation, and the line scan workstation, specifically, a stripe pattern can be projected onto the object to be detected on the industrial inspection table based on the stripe workstation, and a stripe projection image can be obtained according to the change situation of the stripe pattern. The object to be detected can be comprehensively collected based on the area array workstation to obtain a global planar image. The object to be detected can be scanned row by row based on the line scan workstation to obtain a line image. Multiple line images are spliced according to the acquisition order to obtain a line spliced image. An industrial image sample is obtained based on the stripe projection image and the line spliced image. Among them, the stripe pattern is a specific stripe pattern designed for different surfaces of the object to be detected, such as a sine stripe; the stripe projection image is an image recording the change situation when the stripe pattern is projected onto the object to be detected; the global planar image is an image containing the overall information of the object to be detected, and the overall information includes the appearance, shape, surface defects, etc. of the object to be detected; the line image is an image of one or more rows of pixel information collected when scanning the object to be detected row by row; the line spliced image is an image obtained after splicing multiple line images according to the acquisition order, and the line spliced image can be a two-dimensional image or a three-dimensional image.

[0035] Specifically, based on the stripe station, a stripe pattern is projected onto the object to be detected on the industrial inspection table. The change of the stripe pattern on the surface of the object to be detected is recorded by the acquisition camera configured in the stripe station, and a stripe projection image is obtained. Based on the acquisition camera configured in the area array station, a comprehensive acquisition of the object to be detected on the industrial inspection table is performed, and the overall plane information of the object to be detected is obtained at one time to obtain a global plane image. The acquisition camera configured in the moving line scan station is moved, and a plurality of high-precision line images are obtained in a row-by-row acquisition manner from the first edge of the object to be detected until it moves to the second edge of the object to be detected, and the acquisition of the line images is completed. Here, the first edge and the second edge are opposite. For example, if the first edge is the leftmost side of the object to be detected, then the second edge is the rightmost side of the object to be detected. Then, the line images are spliced according to the acquisition order to obtain a spliced image. The stripe projection image, the global plane image, and the line spliced image are used as industrial image samples. It should also be noted that the positions of the above-mentioned multiple stations around the industrial inspection table and the acquisition order of the images acquired by the multiple stations are set based on the ability to achieve image acquisition, and the present application does not make specific limitations.

[0036] Step S202: Classify the industrial image samples to obtain an image classification result.

[0037] Among them, the image classification result includes categories, classification confidence levels, etc. The categories include defective and non-defective, which are used to indicate whether the industrial image sample is a defective image sample or a non-defective image sample. A defective image sample is an image sample with defects on the object to be detected, and a non-defective image sample is an image sample without defects on the object to be detected, or it can also be a defective image sample that cannot be recognized.

[0038] Specifically, the industrial image samples are input into the industrial image analysis model for classification to obtain an image classification result. The industrial image analysis model is used to identify whether there are defects in the object to be detected in the industrial image samples and classify them based on this. In order to further determine the image classification result, the image classification result can be manually verified, and the accuracy of the classification can be checked based on the true state of the object to be detected in the industrial image samples, and the image samples with low classification confidence levels in the defective image samples can be reconfirmed to avoid misjudgment caused by the industrial image analysis model's fuzzy recognition of defect features, so as to obtain accurate and reliable model training data.

[0039] Step S203: Train the supervised model based on the defective image samples and train the unsupervised model based on the non-defective image samples.

[0040] Among them, the supervised model is a model for supervised training, and the unsupervised model is a model for unsupervised training.

[0041] In a possible implementation, the supervised model includes a first detection model and a second detection model. In the process of training the supervised model based on defective image samples, defect type labels may be configured for the defective image samples, universal defect images may be determined in the defective image samples according to universal labels, non-universal defect images may be determined in the defective image samples according to non-universal labels, the first detection model may be trained based on universal defect images, and the second detection model may be trained based on non-universal defect images. Among them, the defect type labels include universal labels and non-universal labels; the first detection model is a large model with more model parameters or network layers, and is trained based on universal defect images; the second detection model is a small model with fewer model parameters or network layers, and is trained based on non-universal defect images.

[0042] Specifically, defect type labels are configured for defective image samples according to the universality and typicality of the defects, universal labels are configured for universal and typical defects, and non-universal labels are configured for defects that only exist under certain specific conditions. Defective image samples configured with universal labels are determined as universal defect images, and defective image samples configured with non-universal labels are determined as non-universal defect images. By using universal defect images and non-universal defect images to train the first detection model and the second detection model respectively, universal defects and non-universal defects can be detected at the same time without switching the detection model according to the defect type, and targeted detection of non-universal defects is achieved on the basis of accurate identification of universal defects, making the defect detection process more flexible and fast, allowing the second detection model to deeply explore specific defect features for non-universal defects for learning, making the defect detection process more flexible, and effectively reducing the problems of over-detection and missed detection in the defect detection process, thereby improving the accuracy and efficiency of defect detection.

[0043] In a possible implementation, during the process of training the first detection model based on general defect images and training the second detection model based on non-general defect images, specifically, the general defect images can be input into the first detection model for classification prediction to obtain the predicted general defect types. The first loss is determined according to the difference between the true general defect types and the predicted general defect types, and the first detection model is trained based on the first loss. The detection parameters learned by the first detection model during the training process are transmitted to the second detection model through knowledge distillation technology, and the second detection model with the obtained detection parameters is trained based on non-general defect images. Among them, the true general defect type is the true label obtained based on the general defect images, which can be the general defect type determined after manual verification and is used for comparison with the predicted general defect types; the detection parameters are the content learned by the first detection model during training, including network weights, general defect feature representations, etc.; the general defect types include at least one defect type, and classifying and predicting the general defect types can be regarded as a multi-classification task.

[0044] Specifically, the general defect images are input into the first detection model for classification prediction to obtain the predicted general defect types, the true general defect types corresponding to the general defect images are obtained, the first loss is determined according to the difference between the predicted general defect types and the true general defect types, and the first loss can be a cross-entropy loss function. The first detection model is trained based on the first loss. Then, the detection parameters learned by the first detection model during the training process are transmitted to the second detection model through knowledge distillation technology, so that the second detection model can be fine-tuned based on non-general defect images on the basis of having the ability to express general defect features. By transmitting the detection parameters, the second detection model is enabled to have the ability to detect general defects. On this basis, the detection of non-general defects is trained based on the detection logic of general defects, and the features of non-general defects are specifically learned, realizing the fine-tuning of the second detection model and enhancing the detection ability of the second detection model for non-general defects.

[0045] In a possible implementation manner, during the process of training a second detection model for obtaining detection parameters based on non-general defect images, specifically, the non-general defect images may be input into the second detection model for obtaining detection parameters for classification prediction to obtain a predicted non-general defect type. The second loss is determined according to the difference between the true non-general defect type and the predicted non-general defect type, and the second detection model is trained in small rounds based on the second loss. Among them, the true non-general defect type is the true label obtained based on the non-general defect images and is used for comparison with the predicted non-general defect type. The non-general defect type includes at least one defect type, and classifying and predicting the non-general defect type can be regarded as a multi-classification task.

[0046] Specifically, the non-general defect images are input into the second detection model for obtaining detection parameters for classification prediction to obtain a predicted non-general defect type, the true non-general defect type corresponding to the non-general defect images is obtained, and the second loss is determined according to the difference between the predicted general defect type and the true general defect type. The second loss may be a cross-entropy loss function, and the second detection model is trained in small rounds based on the second loss. Since the number of non-general defect types is relatively small compared to general defect types, and non-general defect images are relatively difficult to obtain and are also relatively small in quantity compared to general defect images, therefore, performing small-round training and fine-tuning on the second detection model based on non-general defect images can train the second detection model with less computational resources, and can quickly fine-tune the second detection model according to the actual defect conditions of different products and scenarios, enabling the second detection model to have the ability to detect specific non-general defects in a relatively short time, reducing the over-detection rate of defect detection, improving the training speed of the second detection model, and effectively enhancing the generalization ability of the second detection model.

[0047] In a possible implementation manner, during the process of training an unsupervised model based on defect-free image samples, specifically, the defect-free image samples may be input into the unsupervised model to extract the image sample features of the defect-free image samples. Several initial sample centers are randomly initialized according to the feature distribution of the image sample features, the first distances between each image sample feature and all the initial sample centers are determined, the image sample features are assigned to the initial sample clusters corresponding to the initial sample centers based on the first distances, new sample centers are determined according to the image sample features in the initial sample clusters, and the second distances are determined based on the new sample centers and the image sample features until the target sample centers are obtained, and the unsupervised model is trained based on the target sample clusters corresponding to the target sample centers.

[0048] Specifically, the defect-free sample images are input into the unsupervised model to extract the image sample features of the defect-free sample images, and the target sample center is obtained through multiple rounds of iteration based on the image sample features. For the first round, several initial sample centers are randomly initialized according to the feature distribution of the image sample features, and the first distances between each image sample feature and all the initial sample centers are calculated. For any one image sample feature, determine the first distance between the image sample feature and all the initial sample centers, and assign the image sample feature to the initial sample cluster corresponding to the initial sample center with the shortest first distance until all the image sample features are assigned to the corresponding initial sample clusters. In each initial sample cluster, average the image sample features in the initial sample cluster, and use the averaged features as the new sample center, which can represent the average feature of the initial sample cluster. For the remaining rounds, determine the second distance between the image sample feature and the sample center to be updated obtained in the previous round, assign the image sample feature to the updated sample cluster corresponding to the sample center to be updated based on the second distance, and continue to update the sample center according to the image sample features in the updated sample cluster until the target sample center is obtained. The unsupervised model is trained based on the target sample cluster corresponding to the target sample center. Clustering the defect-free sample images based on the image features to obtain the target sample center can classify different types of sample images into target sample clusters without any labels. At this time, there may be undetected defects in the target sample clusters. Therefore, training the unsupervised model based on the target sample center can effectively detect the defects that the industrial image analysis model fails to identify, thereby reducing the missed detection rate of defect detection.

[0049] It should be noted that when there are undetected defects in the target sample cluster, assign a defect type to the defect. The defect type can be the type learned by the industrial image analysis model or the supervised model, or a new defect type. Input the image corresponding to the undetected defect as an industrial image sample into the industrial image analysis model and the supervised model for training and optimization, so that the industrial image analysis model and the supervised model can learn the feature representation of the defect type. Based on this, after continuous optimization, the industrial image analysis model and the supervised model can analyze and identify various types of defects, which helps to improve the accuracy of defect detection.

[0050] In a possible implementation, during the process of training an unsupervised model based on a target sample cluster corresponding to a target sample center, specifically, for any target sample cluster, determine the third distance between each image sample feature in the target sample cluster and the target sample center of the target sample cluster, and determine the third loss based on multiple third distances. For any two target sample clusters, determine the fourth distance between the target sample centers of any two target sample clusters and the difference in sample center features, and determine the fourth loss based on multiple fourth distances and the difference in sample center features. Train the unsupervised model based on the third loss and the fourth loss.

[0051] Specifically, for any target sample cluster, determine the third distance between each image feature in the target sample cluster and the target sample center of the target sample cluster, set a maximum distance threshold, calculate the first difference value between multiple third distances and the maximum distance threshold, determine the intra-cluster sub-loss of the target sample cluster based on the mean value of multiple first difference values, and determine the third loss based on the mean value of the intra-cluster sub-losses of all target sample clusters. The third loss can be an intra-cluster similarity loss function.

[0052] For any two target sample clusters, determine the fourth distance between the target sample centers of any two target sample clusters and the difference in sample center features, set a minimum distance threshold, calculate the second difference value between the fourth distance and the minimum distance threshold, perform weighted summation based on the second difference value and the difference in sample center features to obtain the inter-cluster sub-loss of the two target sample clusters. Determine the fourth loss based on the mean value of multiple inter-cluster sub-losses. The fourth loss can be an inter-cluster separation loss function. Finally, train the unsupervised model based on the third loss and the fourth loss. Train the unsupervised model based on intra-cluster similarity and inter-cluster separation, enabling the unsupervised model to learn the subtle differences in intra-cluster features and the obvious differences in inter-cluster features, thereby improving the clustering accuracy of the unsupervised model and reducing the missed detection rate of defect detection.

[0053] Step S204: Obtain the industrial image to be detected, and perform defect detection on the industrial image to be detected according to the trained supervised model and unsupervised model to obtain a defect detection result.

[0054] Among them, the industrial image to be detected is an image similar to the industrial image sample and is also based on multiple workstations; the defect detection result is a combined result output by the supervised model and the unsupervised model, including the identified defect types output by the supervised model and the undetected defects output by the unsupervised model.

[0055] Specifically, industrial images to be detected are obtained based on multiple stations such as a stripe station, an area array station, and a line scan station. The industrial images to be detected are input into an industrial image analysis model for classification to obtain defective images and non-defective images. The defective images are input into a supervised model for defect detection to obtain the detection results of the supervised model. The non-defective images are input into an unsupervised model for defect detection to obtain the detection results of the unsupervised model. The detection results of the supervised model and the detection results of the unsupervised model are combined to obtain the defect detection results.

[0056] It should also be noted that in addition to inputting non-defective images into the unsupervised model for defect detection, it is also possible to separately obtain the areas of the object to be detected that are prone to abnormal conditions, and input the industrial images containing these areas into the unsupervised model for defect detection, so as to reduce the missed detection rate of defect detection.

[0057] In a possible implementation manner, the defect detection method for preventing over-missed detection provided by the embodiments of the present disclosure can be applied to the fan blade detection scenario. Refer to Figure 3 , Figure 3 which is an optional overall process of the defect detection method for preventing over-missed detection provided by the embodiments of the present disclosure.

[0058] First, multiple stations such as a stripe station, an area array station, and a line scan station are set around the industrial detection table. Based on the stripe station, a stripe pattern is projected onto the fan blade on the industrial detection table. Based on the area array camera of the area array station, the change of the stripe pattern on the fan blade is obtained to get a stripe projection image. Based on the line scan camera of the line scan station, the line images of the stripe pattern on the fan blade are obtained. The multiple line images are spliced in the acquisition order to get a line spliced image. An industrial image sample is obtained according to the line projection image and the line spliced image.

[0059] Next, the industrial image samples are input into the industrial image analysis model for classification to obtain defective image samples and non-defective image samples. Defect type labels are configured for the defective samples, and based on the defect type labels, the industrial image samples can be divided into general defect images and non-general defect images. The general defect images are input into the first detection model for supervised training, and the non-general defect images are input into the second detection model for supervised training. When performing supervised training on the first detection model and the second detection model, the loss value is determined according to the difference between the prediction result and the true label, and the first detection model and the second detection model are trained and optimized based on the loss value, ensuring that the gap between manual judgment and the detection of the first detection model and the second detection model can be narrowed on the premise of defect detection. The first detection model and the second detection model can be trained independently, or they can be jointly trained after integrating the first detection model and the second detection model. The first detection model and the second detection model can be collectively referred to as the supervised model. After training is completed, the first detection model and the second detection model are deployed. Then, the non-defective images are input into the unsupervised model for unsupervised training, and after training is completed, the unsupervised model is deployed. It should be noted that if the model training is carried out on the terminal, the supervised model and the unsupervised model can be directly deployed on the terminal; if the model training is carried out on the server, the supervised model and the unsupervised model can be deployed on the server side or the terminal.

[0060] After the supervised model and the unsupervised model are trained, industrial images to be detected are obtained based on multiple workstations such as the stripe workstation, the area array workstation, and the line scan workstation. The industrial images to be detected are input into the industrial image analysis model for classification to obtain defective images and non-defective images. The defective images are input into the supervised model for defect detection to obtain the detection results of the supervised model, the non-defective images are input into the unsupervised model for defect detection to obtain the detection results of the unsupervised model, and the detection results of the supervised model and the detection results of the unsupervised model are combined to obtain the defect detection results of the fan blade. Quality assessment is carried out based on the defect detection results of the fan blade to improve the quality consistency of the fan blade.

[0061] In summary, the defect detection method for preventing over-detection and under-detection provided by the embodiments of the present disclosure obtains industrial images for detection by configuring multi-station cameras on an industrial detection platform, classifies the defect types of industrial images into general defects and non-general defects according to an industrial image analysis model, and trains and fine-tunes a second detection model in a small-round iteration manner based on the industrial images of non-general defect categories, which can enhance the detection ability of the second detection model for non-general defect categories and reduce the over-detection rate of defect detection. In addition, unsupervised detection is performed on the defect-free image samples classified by the industrial image analysis model and the areas prone to abnormal conditions in the object to be detected, which can effectively reduce the under-detection rate of defect detection, thereby improving the accuracy of defect detection.

[0062] Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification and the above-mentioned drawings of the present disclosure are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so as to describe the embodiments of the present disclosure. For example, it can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0063] It should be understood that in the present disclosure, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally means that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or its similar expression means any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0064] It should be understood that in the description of the embodiments of the present disclosure, the meaning of "a plurality (or multiple)" is more than two. Understandings such as greater than, less than, exceeding, etc. do not include the present number, and understandings such as above, below, within, etc. include the present number.

[0065] In several embodiments provided by the present disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0066] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0067] In addition, in each embodiment of the present disclosure, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0068] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0069] It should also be understood that the various embodiments provided by the embodiments of the present disclosure can be combined arbitrarily to achieve different technical effects.

[0070] The above is a specific description of the preferred embodiments of the present disclosure. However, the present disclosure is not limited to the above-mentioned embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present disclosure. These equivalent deformations or substitutions are all included within the scope defined by the claims of the present disclosure.

Claims

1. A defect detection method for preventing over - omission and under - detection, characterized in that, Including: Obtain industrial image samples, where the industrial image samples are obtained by image acquisition at multiple workstations; Classify the industrial image samples to obtain an image classification result, where the image classification result is used to indicate whether the industrial image samples are defective image samples or non-defective image samples; Train a supervised model based on the defective image samples and train an unsupervised model based on the non-defective image samples; Obtain an industrial image to be detected, and perform defect detection on the industrial image to be detected according to the trained supervised model and the unsupervised model to obtain a defect detection result.

2. The defect detection method for preventing over-missing detection according to claim 1, characterized in that The supervised model includes a first detection model and a second detection model. The training of the supervised model based on the defective image samples includes: Configure defect type labels for the defective image samples, where the defect type labels include general type labels and non-general type labels; Determine general type defective images in the defective image samples according to the general type labels, and determine non-general type defective images in the defective image samples according to the non-general type labels; Train the first detection model based on the general type defective images and train the second detection model based on the non-general type defective images.

3. The defect detection method for preventing over-missing detection according to claim 2, wherein, The training of the first detection model based on the general type defective images and the training of the second detection model based on the non-general type defective images include: Input the general type defective images into the first detection model for classification prediction to obtain a predicted general type defect type, determine a first loss based on the difference between the true general type defect type and the predicted general type defect type, and train the first detection model based on the first loss; Transfer the detection parameters learned by the first detection model during training to the second detection model through knowledge distillation technology, and train the second detection model that obtains the detection parameters based on the non-general type defective images.

4. The defect detection method for preventing over-missing detection according to claim 3, characterized in that The training of the second detection model that obtains the detection parameters based on the non-general type defective images includes: Input the non-general type defective images into the second detection model that obtains the detection parameters for classification prediction to obtain a predicted non-general type defect type, determine a second loss based on the difference between the true non-general type defect type and the predicted non-general type defect type, and perform a small number of rounds of training on the second detection model based on the second loss.

5. The defect detection method for preventing over-missing detection according to claim 1, characterized in that The training of the unsupervised model based on the non-defective image samples includes: Input the defect-free image samples into an unsupervised model, extract the image sample features of the defect-free image samples, randomly initialize several initial sample centers according to the feature distribution of the image sample features, determine the first distances between each of the image sample features and all the initial sample centers, assign the image sample features to the initial sample clusters corresponding to the initial sample centers based on the first distances, determine new sample centers according to the image sample features in the initial sample clusters, and determine second distances based on the new sample centers and the image sample features until the target sample centers are obtained; Train the unsupervised model based on the target sample clusters corresponding to the target sample centers.

6. The defect detection method for preventing over-missing detection according to claim 5, characterized in that The training of the unsupervised model based on the target sample clusters corresponding to the target sample centers includes: For any one of the target sample clusters, determine the third distances between each of the image sample features in the target sample cluster and the target sample center of the target sample cluster, and determine a third loss based on the multiple third distances; For any two of the target sample clusters, determine the fourth distances between the target sample centers of any two of the target sample clusters and the sample center feature differences, and determine a fourth loss based on the multiple fourth distances and the sample center feature differences; Train the unsupervised model based on the third loss and the fourth loss.

7. The defect detection method for preventing over-missing detection according to claim 1, characterized in that, The obtaining of the industrial image samples includes: Set up multiple workstations around the industrial inspection table, where the workstations include a stripe workstation, a planar array workstation, and a line scan workstation; Obtain the industrial image samples corresponding to the object to be detected on the industrial inspection table based on the stripe workstation, the planar array workstation, and the line scan workstation.

8. The defect detection method for preventing over-missing detection according to claim 7, characterized in that, The obtaining of the industrial image samples corresponding to the object to be detected on the industrial inspection table based on the stripe workstation, the planar array workstation, and the line scan workstation includes: Project a stripe pattern onto the object to be detected on the industrial inspection table based on the stripe workstation, obtain a stripe projection image according to the change of the stripe pattern, conduct a comprehensive acquisition of the object to be detected based on the planar array workstation to obtain a global planar image, conduct a line-by-line acquisition of the object to be detected based on the line scan workstation to obtain line images, splice the multiple line images in the acquisition order to obtain a line spliced image; Obtain the industrial image samples based on the stripe projection image, the global planar image, and the line spliced image.

9. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the defect detection method for preventing over-missing detection according to any one of claims 1 to 8.

Images