Defect detection method and device, storage medium and electronic device

Abstract

The application relates to a defect detection method and device, a storage medium and an electronic device. The method comprises the following steps: in the case that a to-be-detected image is acquired, an intermediate layer vector of the to-be-detected image is acquired; a processing strategy of the to-be-detected image is determined according to the intermediate layer vector and a historical processing strategy of a historical image, wherein the processing strategy comprises a model used for processing the to-be-detected image and a processing area of the model; a feature map is obtained by processing a processing area of the to-be-detected image by using a model corresponding to the processing strategy; and an image defect of the to-be-detected image is recognized according to the feature map. The application solves the technical problem of low defect recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of defect detection, and in particular to a defect detection method and device, a storage medium, and an electronic device. BACKGROUND

[0002] In the process of detecting defects on the surface of an object, a model can be used to identify an image obtained by photographing the surface of the object to detect defects exhibited by the object.

[0003] Some models in the prior art can be used to detect defects on the surface of an object. For example, a method for detecting small visual defects on the surface of an object involves inputting a picture into a neural network model for detection, and the neural network model is a pre-trained model. The attention mechanism of the model is used to retain fine features, thereby detecting small defects on the surface of the object.

[0004] However, because the types of objects photographed are diverse and the pictures photographed are different, if the same model is used for unified processing, the recognition effect is not accurate. SUMMARY

[0005] The present application provides a defect detection method and device, a storage medium, and an electronic device to solve the technical problem of low defect recognition accuracy.

[0006] In a first aspect, the present application provides a defect detection method, including: in the case of obtaining a to-be-detected image, obtaining an intermediate layer vector of the to-be-detected image; determining a processing strategy of the to-be-detected image according to the intermediate layer vector and a historical processing strategy of a historical image, wherein the processing strategy includes a model used to process the to-be-detected image and a processing area of the model; processing the processing area of the to-be-detected image using the model corresponding to the processing strategy to obtain a feature map; and identifying an image defect of the to-be-detected image according to the feature map.

[0007] In a second aspect, the present application provides a defect detection device, including: an obtaining module configured to, in the case of obtaining a to-be-detected image, obtain an intermediate layer vector of the to-be-detected image; a determining module configured to determine a processing strategy of the to-be-detected image according to the intermediate layer vector and a historical processing strategy of a historical image, wherein the processing strategy includes a model used to process the to-be-detected image and a processing area of the model; a processing module configured to process the processing area of the to-be-detected image using the model corresponding to the processing strategy to obtain a feature map; and an identifying module configured to identify an image defect of the to-be-detected image according to the feature map.

[0008] As an optional example, the processing module comprises: a first processing unit, configured to, when the processing strategy is to process a first processing region of the to-be-detected image using a global model, input the first processing region of the to-be-detected image into the global model to obtain the feature map, wherein the model weights of the global model are converted from first floating-point numbers to first integers.

[0009] As an optional example, the processing module comprises: a second processing unit, configured to, when the processing strategy is to process a second processing region of the to-be-detected image using a local attention model, input the second processing region of the to-be-detected image into the local attention model to obtain the feature map, wherein the model weights of the local attention model are converted from second floating-point numbers to half-precision floating-point numbers.

[0010] As an optional example, the processing module comprises: a third processing unit, configured to, when the processing strategy is to process a third processing region of the to-be-detected image using a global model and to process a fourth processing region of the to-be-detected image using a local attention model, input the third processing region of the to-be-detected image into the global model and input the fourth processing region of the to-be-detected image into the local attention model; combine a global feature output by the global model and a local feature output by the local attention model to obtain the feature map.

[0011] As an optional example, the third processing unit comprises: a processing subunit, configured to, after spatially aligning the global feature and the local feature after up-sampling and spatial mapping operations on the global feature and the local feature, splice the aligned global feature and the local feature into a multi-scale combined feature; input the multi-scale combined feature into a deformable convolution layer, and perform pixel misregistration compensation on the multi-scale combined feature by the deformable convolution layer; and perform adaptive sampling on the compensated multi-scale combined feature to obtain the feature map.

[0012] As an optional example, the determination module comprises: a determination unit, configured to determine a historical intermediate layer output vector of the historical image according to the historical image; determine a target historical image from the historical images according to a similarity between the intermediate layer vector of the to-be-detected image and the historical intermediate layer output vector of the historical image; and determine the processing strategy of the target historical image as the processing strategy of the to-be-detected image.

[0013] As an optional example, the determination module comprises: an input unit, configured to, when the processing strategy is not determined, input the to-be-detected image into a strategy network, extract an intermediate layer vector of the to-be-detected image by the strategy network, and predict the processing strategy according to the intermediate layer vector.

[0014] In a third aspect, the present application provides an electronic device, comprising: at least one communication interface; at least one bus connected with the at least one communication interface; at least one processor connected with the at least one bus; and at least one memory connected with the at least one bus, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement any of the above defect detection methods.

[0015] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute any of the above defect detection methods.

[0016] Compared with the prior art, the above technical solution provided by the embodiments of the present application has the following advantages: in the scheme provided by the embodiments of the present application, after obtaining a to-be-detected image, a middle layer vector of the to-be-detected image is obtained; a processing strategy of the to-be-detected image is determined according to the middle layer vector and a historical processing strategy of a historical image, wherein the processing strategy includes a model used for processing the to-be-detected image and a processing region of the model; a feature map is obtained by processing the processing region of the to-be-detected image using the model corresponding to the processing strategy; and an image defect of the to-be-detected image is identified according to the feature map, so that the middle layer vector of the to-be-detected image can be assisted according to the historical processing strategy of the historical image to determine a suitable model of the to-be-detected image and a processing region of the to-be-detected image, the processing region is processed by the corresponding model to obtain a feature map, and the image defect of the to-be-detected image is identified according to the feature map, thereby achieving the effect of processing the processing region of the to-be-detected image using a suitable model and improving the accuracy of defect identification of the to-be-detected image. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor.

[0019] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings, and these exemplary illustrations do not constitute a limitation on the embodiments, and elements with the same reference numerals in the drawings represent similar elements, unless otherwise specified, and the drawings do not constitute a proportional limitation.

[0020] Figure 1 A flow chart of a defect detection method provided by an embodiment of the present application is shown in FIG. 1.

[0021] Figure 2 A processing strategy determination diagram of a defect detection method provided by an embodiment of the present application is shown in FIG. 2.

[0022] Figure 3 A feature combination diagram of a defect detection method provided by an embodiment of the present application is shown in FIG. 3.

[0023] Figure 4 A structural schematic diagram of a defect detection device provided by an embodiment of the present application is shown in FIG. 4.

[0024] Figure 5 An electronic device provided by an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0026] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplifying the present application, the components and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeatedly refer to numbers and / or letters in different examples. Such repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0027] In order to solve the technical problem of low defect recognition accuracy in the prior art, the present application provides a defect detection method, which can improve the accuracy of defect recognition of the to-be-detected image.

[0028] Figure 1 A flow chart of a defect detection method provided by an embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the defect detection method includes:

[0029] S101, in the case of obtaining a to-be-detected image, obtaining a middle layer vector of the to-be-detected image;

[0030] S102, determining a processing strategy of the to-be-detected image according to the intermediate layer vector and the historical processing strategy of the historical image, wherein the processing strategy comprises a model used for processing the to-be-detected image and a processing region of the model;

[0031] S103, processing the processing region of the to-be-detected image using the model corresponding to the processing strategy, to obtain a feature map;

[0032] S104, identifying an image defect of the to-be-detected image according to the feature map.

[0033] The defect detection method can be applied in a process of detecting defects of an object in an image. For example, detecting defects on a workpiece surface, detecting cracks on a road surface, etc.

[0034] The to-be-detected image can be an image obtained by photographing an object to be detected, for example, photographing a workpiece to obtain an image containing the workpiece, or photographing a road surface to obtain an image containing the road surface, which is used for defect detection.

[0035] The embodiment includes a plurality of models. In the specific process of image defect detection, one or more models can be used for defect detection. Which model or models are used for defect detection is mainly determined according to the to-be-detected image and the historical image.

[0036] The idea of the embodiment is to determine the historical processing strategy of the historical image, and the historical image corresponds to a historical intermediate layer vector of the historical image. By comparing the intermediate layer vector of the to-be-detected image with the historical intermediate layer vector of the historical image, it can be determined that the to-be-detected image belongs to which type or is similar to which historical image, and further the historical processing strategy of the historical image can be used as the processing strategy of the to-be-detected image.

[0037] The processing strategy in the embodiment includes which model is used to detect the to-be-detected image, and which processing region in the to-be-detected image is detected. Therefore, for the to-be-detected image, when the model to be used and the processing region in the to-be-detected image to be detected are determined by comparison, the corresponding model can be used. The corresponding processing region is processed to obtain a feature map. The feature map is used for final defect identification.

[0038] For example, as shown in Figure 2 For the historical image, the corresponding processing strategy has been determined and processed, and the to-be-detected image can be compared with the historical image in the intermediate layer feature to determine which historical image processing strategy is used to process the to-be-detected image.

[0039] In this embodiment, the model used to process the image to be detected is divided into a global model and a local attention model. The global model can be used to extract the global features of the image to be detected. The local attention model can focus more on the local features of the image to be detected. The two models can be used alternatively or simultaneously. In addition, it can be determined which part of the image to be detected is processed using the model during the use of the model.

[0040] The scheme provided by the embodiments of the present application can obtain the intermediate layer vector of the image to be detected when the image to be detected is obtained, determine the processing strategy of the image to be detected according to the intermediate layer vector and the historical processing strategy of the historical image, wherein the processing strategy includes a model used to process the image to be detected and a processing area of the model, process the processing area of the image to be detected using the model corresponding to the processing strategy to obtain a feature map, and identify the image defects of the image to be detected according to the feature map. Therefore, the intermediate layer vector of the image to be detected can be assisted according to the historical processing strategy of the historical image to determine the appropriate model of the image to be detected and the processing area of the image to be detected, the processing area is processed by the corresponding model to obtain a feature map, and the image defects of the image to be detected are identified according to the feature map, thereby achieving the effect of processing the processing area of the image to be detected using the appropriate model and improving the accuracy of defect identification of the image to be detected.

[0041] As an optional example, processing the processing area of the image to be detected using the model corresponding to the processing strategy to obtain a feature map includes: when the processing strategy is to process the first processing area of the image to be detected using the global model, inputting the first processing area of the image to be detected into the global model to obtain the feature map, wherein the model weight of the global model is converted from the first floating point number to the first integer.

[0042] The first processing area of the embodiment can be the entire area of the image to be detected, or a partial area, which can be a key area of the image to be detected.

[0043] The global model in the embodiment can be a convolutional neural network model (CNN), which can also be referred to as a large-scale expert model. The model is used to extract the global context features of the image to be detected or the processing area of the image to be detected. In order to make the global model focus more on the integrity of the image to be detected, the weight of the global model can be adjusted in the embodiment. The weight of the global model is adjusted according to the degree of focus on the integrity of the image to be detected. For example, the weight can be a floating point number, and the number of digits of the decimal part of the floating point number can be adjusted according to the degree of focus on the integrity of the image to be detected. The more focused on the integrity, the more digits of the decimal part. If the focus is on the local, the number of digits of the decimal part can be less.

[0044] In this embodiment, in order to make the global model focus on the whole of the to-be-detected image, the weight of the global model can be determined, and if the weight is a floating-point number, the floating-point number is converted into an integer. According to different degrees of focus on the whole, the number of bits of the converted integer can be determined, and the more the focus on the whole, the fewer the number of bits of the integer. For example, a single-precision floating-point number of 32 bits is converted into an integer of 8 bits, or a double-precision floating-point number of 64 bits is converted into an integer of 16 bits.

[0045] In this embodiment, the variation range of the weight can also be determined according to the image complexity of the to-be-detected image, and in order to retain more overall features, the higher the image complexity, the greater the variation range of the weight.

[0046] As an optional example, the processing region of the to-be-detected image is processed using the model corresponding to the processing strategy to obtain the feature map, including: when the processing strategy is to process the second processing region of the to-be-detected image using the local attention model, inputting the second processing region of the to-be-detected image into the local attention model to obtain the feature map, wherein the model weight of the local attention model is converted from the second floating-point number to the half-precision floating-point number.

[0047] The second processing region in this embodiment can be all or part of the to-be-detected image, and if it is part of the to-be-detected image, it can be a key region of the to-be-detected image.

[0048] The local attention model in this embodiment can be a model based on the Transformer architecture, which is used to identify local features of the to-be-detected image. In order to make the local attention model focus more on the local features of the to-be-detected image, the weight of the model can be adjusted, such as increasing the number of bits of the weight of the model. For example, if the weight of the model is a floating-point number, the number of bits of the decimal part of the floating-point number can be increased.

[0049] In this embodiment, if the number of bits of the weight is blindly increased, it will put a large burden on the calculation and memory. After calculation, it is found that when the number of bits of the weight of the local attention model is reduced, the extraction of the local features can still be met, and the calculation process is accelerated and the memory optimization effect is better. For example, the weight of a single-precision floating-point number of 32 bits is converted into a half-precision floating-point number of 16 bits, and the model can still extract the local features of the to-be-detected image, and the calculation efficiency is higher and the memory optimization effect is better.

[0050] In this embodiment, the variation range of the weight can also be determined according to the image complexity of the to-be-detected image, and in order to retain more local features, the more complex the image, the smaller the variation range of the weight.

[0051] As an optional example, the processing region of the image to be detected is processed using the model corresponding to the processing strategy to obtain the feature map. This includes: when the processing strategy is to use a global model to process the third processing region of the image to be detected and to use a local attention model to process the fourth processing region of the image to be detected, the third processing region of the image to be detected is input into the global model and the fourth processing region of the image to be detected is input into the local attention model; the global features output by the global model and the local features output by the local attention model are combined to obtain the feature map.

[0052] In this embodiment, if the processing strategy includes processing the image to be detected using both a global model and a local attention model, then the image to be processed is input into both the global model and the local attention model. The third processing region of the image to be detected involved in the global model can be the entire region or a part of the image to be detected. The fourth processing region of the image to be detected involved in the local attention model can be the entire region or a local region of the image to be detected. The third and fourth processing regions can be the same or different.

[0053] The features obtained by the global model processing the third processing region of the image to be detected can be considered global features, while the features obtained by the local attention model processing the fourth processing region of the image to be detected can be considered local features. The global and local features are combined to obtain a feature map, which is used to determine whether the image to be detected contains defects.

[0054] For example, such as Figure 3 As shown, the image to be detected is processed by both a global model and a local attention model. The global features obtained from the global model and the local features obtained from the local attention model are combined to obtain a feature map.

[0055] As an optional example, combining the global features output by the global model with the local features output by the local attention model to obtain a feature map includes: performing upsampling and spatial mapping operations on the global and local features, followed by spatial alignment; concatenating the aligned global and local features into a multi-scale combined feature; inputting the multi-scale combined feature into a deformable convolutional layer to perform pixel misalignment compensation on the multi-scale combined feature; and adaptively sampling the compensated multi-scale combined feature to obtain the feature map.

[0056] In the embodiment, for the case that the to-be-detected image is processed by the global model and the local attention model together, when the feature combination is performed, multiple cases are divided. First, since the regions of the to-be-detected image processed by the global model and the local attention model can be different, the global features extracted by the global model and the local features extracted by the local attention model are up-sampled and mapped into the same space, so that the features are spatially aligned, and the aligned features are spliced. Since the regions of the to-be-detected image processed by the global model and the local attention model can be the same, can have overlapping parts, or can be different. For the case that they are the same or have overlapping parts, pixel misalignment compensation is needed. The global features and the local features can be spliced first, and then a plurality of scale combined features are obtained after splicing. The plurality of scale combined features are input into a deformable convolution layer. The pixel misalignment compensation of the plurality of scale combined features is performed through the deformable convolution layer. The compensated combined features are adaptively sampled to obtain a feature map.

[0057] As an optional example, according to the intermediate layer vector and the historical processing strategy of the historical image, the processing strategy of the to-be-detected image is determined, including: determining the historical intermediate layer output vector of the historical image according to the historical image; determining a target historical image from the historical images according to the similarity between the intermediate layer vector of the to-be-detected image and the historical intermediate layer output vector of the historical image; and determining the processing strategy of the target historical image as the processing strategy of the to-be-detected image.

[0058] In the embodiment, when the processing strategy of the to-be-detected image is determined, the intermediate layer vector of the to-be-detected image is obtained, so that the intermediate layer vector of the to-be-detected image can be compared with the historical intermediate layer vector of the historical image. Since the corresponding processing strategy of the historical image has been determined and processed according to the corresponding processing strategy, it can be known whether the processing strategy of the historical image is appropriate. The historical image with an appropriate strategy can be marked. After comparing the intermediate layer vector of the to-be-detected image with the historical intermediate layer vector of the historical image, it can be determined which historical image or which historical image has a higher similarity with the to-be-detected image. Therefore, the processing strategy of the historical image with the mark is used as the processing strategy of the to-be-detected image. If the corresponding historical image does not have a mark, it means that no appropriate processing strategy is found. If no appropriate processing strategy is found, the strategy of the to-be-detected image can be determined, so that the appropriate processing strategy of the to-be-detected image is determined.

[0059] In one scheme, the difference between adjacent features in the extracted features can be analyzed. If the difference between adjacent features is high, it can be indicated that the detail difference of the to-be-detected image is larger, and the local attention model can be used in the processing strategy. If the difference between adjacent features is low, the global model can be used. The high and low of the difference can be constrained by a threshold.

[0060] As an optional example, according to the intermediate layer vector and the historical processing strategy of the historical image, determining the processing strategy of the to-be-detected image includes: in the case where the processing strategy is not determined, inputting the to-be-detected image into the strategy network, extracting the intermediate layer vector of the to-be-detected image by the strategy network, and predicting the processing strategy according to the intermediate layer vector.

[0061] In the present example, when the to-be-detected image is judged by the strategy, the to-be-detected image can be input into the strategy network, and the intermediate layer vector of the to-be-detected image can be extracted by the strategy network. Based on the intermediate layer vector, the processing strategy is predicted. The predicted processing strategy can be used for processing the to-be-detected image, and the processing result is fed back to the strategy network for training.

[0062] In the process of predicting the processing strategy, the intermediate layer vector of the entire region of the to-be-detected image can be used for prediction, or a prediction region can be determined from the to-be-detected image, and then the intermediate layer vector of the prediction region is used for prediction.

[0063] In the prediction, a layer-by-layer progressive method can be used, such as selecting a prediction region, predicting a first prediction result, then expanding the prediction region to the surrounding, and continuing to predict, predicting a second prediction result, and determining whether the prediction results after the two predictions are the final prediction result according to the difference between the second prediction result and the first prediction result. If it is not the final prediction result, the prediction region can be further expanded until the final prediction result is determined. If the final prediction result is not determined after expanding to the entire to-be-detected image, the final prediction result is determined by comprehensively considering the prediction results of each prediction.

[0064] The present application solves the accurate fusion of features of different scales and different sources by providing deformable convolution for feature fusion, and the obtained feature map is used for defect detection, thereby improving the accuracy of defect detection.

[0065] Taking the application of the method in the present application in the scene of workpiece defect detection as an example, the to-be-detected image can be an image of a workpiece taken.

[0066] First, the model is constructed.

[0067] Initialize the global model (or large-scale expert model): build a large-scale expert model based on a convolutional neural network. This model is designed to extract the global context features of the image to identify the overall integrity of the product and large-scale anomalies (such as deformation, large-area stains). During the model training or fine-tuning stage, introduce a simulated quantization operation (pseudo-quantization node) to convert the model's weights from standard 32-bit floating-point numbers (FP32) to 8-bit integers (INT8) through Quantization-Aware Training (QAT), allowing the model to learn to adapt to the precision loss caused by quantization during training, ensuring that the precision loss is less than 1% when deployed.

[0068] Initialize the local attention model (also known as the small-scale expert model): build a small-scale expert model based on the Transformer architecture. This model uses local attention mechanisms and is designed for high-precision detection and positioning of small local defects (such as fine scratches and pits). Use the TensorRT framework to convert the Transformer model's weights from FP32 to 16-bit half-precision floating-point numbers (FP16) mode, ensuring that the detail detection accuracy is maintained while achieving computational acceleration and memory optimization.

[0069] When receiving the workpiece image, a dynamic routing decision is made.

[0070] After the system receives the industrial product image to be detected, it performs a historical routing decision reuse query: feature extraction: input the workpiece image or key region (patch) of the image into the large-scale expert model, and use the intermediate layer output as the visual feature vector. Similarity comparison: compare the extracted feature vector with the historical feature vectors of historical images stored in the historical decision library, using cosine similarity measurement, and compare with the preset threshold. The historical decision library stores recent routing decisions. Decision reuse: if a historical region with visual content that meets the threshold condition is found, directly reuse its associated routing decision, use the same processing strategy as the routing decision to process the workpiece image, and skip the strategy network inference. Reinforcement learning strategy routing: if no reusable historical decision is found or the system needs to be re-evaluated, input the workpiece image or key region of the image as the state into the pre-trained reinforcement learning (RL) strategy network. The reinforcement learning strategy network performs inference and outputs an action that specifies the best strategy for processing the current region, such as: only call the large-scale expert, only call the small-scale expert, call both, or specify a specific region range for processing. Historical library update: store the routing decision obtained by this inference and the corresponding region feature vector in the historical decision library for future reuse.

[0071] After the processing strategy is determined according to the historical routing, the image data can be distributed to the corresponding expert model. If the decision contains large-scale expert model processing, the large-scale expert model reasons the input data, extracts global feature maps, and identifies overall abnormalities. If the decision contains small-scale expert model processing, the small-scale expert model reasons the specified input data and locates minor defects.

[0072] If the processing strategy is the joint processing of large-scale and small-scale expert models, feature combination needs to be performed. The outputs from the large-scale expert (low-resolution global features) and the small-scale expert (high-resolution local features) are collected. Through upsampling and spatial mapping operations, preliminary spatial alignment is performed, and they are spliced in the channel dimension to form multi-scale combined feature maps. The combined feature maps are input into a deformable convolution layer. The use of DCN enables the learning of the spatial offset of the convolution kernel sampling points, which can adaptively compensate for pixel misalignment, effectively overcome pixel-level deviations caused by scale differences, different model structures, and imperfect preliminary alignment, and accurately extract information from corresponding positions in different source feature maps; optimize edge representation, better focus on information complex areas such as defect boundaries through data-driven adaptive sampling, integrate multi-scale evidence, and significantly improve the accuracy of defect edge segmentation and positioning; generate fine features, and output fine feature maps that contain rich multi-scale information after effective fusion and optimization by DCN.

[0073] The feature maps are input into the final prediction head, which outputs the final high-precision industrial defect detection results for subsequent quality control or process. The detection results can include whether there is a defect and the location of the defect.

[0074] The above image and detection results can be used to train the model in reverse. The evaluation results are used as reward signals to update the RL policy network offline or online to continuously improve the efficiency and accuracy of routing decisions. According to the feedback data, fine-tune or incrementally train the large and small scale expert models to improve their recognition ability for specific defects or scenarios. At the same time, monitor the accuracy of the INT8 model, and if necessary, re-perform QAT to maintain model accuracy.

[0075] Figure 4 A structural schematic diagram of a defect detection device provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the defect detection device includes: Figure 4 An acquisition module 401 is configured to acquire an intermediate layer vector of a to-be-detected image when the to-be-detected image is acquired.

[0076]

[0077] ​The determination module 402 is used to determine the processing strategy of the image to be detected based on the intermediate layer vector and the historical processing strategy of the historical image. The processing strategy includes the model for processing the image to be detected and the processing region of the model.

[0078] The processing module 403 is used to process the processing region of the image to be detected using the model corresponding to the processing strategy, and obtain the feature map;

[0079] The recognition module 404 is used to identify image defects in the image to be detected based on the feature map.

[0080] The aforementioned defect detection device can be used to detect defects in objects within images. For example, it can be used to detect defects on the surface of workpieces or cracks on the surface of highways.

[0081] The image to be detected can be an image of the object to be defect detected, such as an image of a workpiece containing the workpiece, or an image of a road surface containing the road surface. The image is used for defect detection.

[0082] This embodiment includes multiple models. When performing image defect detection, one or more of these models can be used. The specific model used depends on the image to be detected and historical images.

[0083] The approach of this embodiment is to determine the historical processing strategy of historical images, and each historical image corresponds to a historical intermediate layer vector. By comparing the intermediate layer vector of the image to be detected with the historical intermediate layer vector of the historical images, the type of the image to be detected or which historical image it is similar to can be determined. Furthermore, the historical processing strategy of the historical images can be used as the processing strategy for the image to be detected.

[0084] The processing strategy in this embodiment includes both which model to use for detection in the image and which processing regions within the image to detect. Therefore, for the image to be detected, once the model and the processing regions in the image to be detected are determined through comparison, the corresponding model can be used. The corresponding processing regions are then processed to obtain feature maps. These feature maps are used for final defect identification.

[0085] For example, such as Figure 2 As shown, for historical images, the corresponding processing strategies have been determined and processed. For the image to be detected, intermediate layer features can be compared with those of historical images to determine which historical image processing strategy to use for processing the image to be detected.

[0086] In this embodiment, the model used to process the image to be detected is divided into a global model and a local attention model. The global model can be used to extract the global features of the image to be detected. The local attention model can focus more on the local features of the image to be detected. The two models can be used alternatively or simultaneously. In addition, it can be determined which part of the image to be detected is processed using the model during the use of the model.

[0087] The scheme provided by the embodiment of the application comprises the following steps: obtaining an intermediate layer vector of an image to be detected in the case that the image to be detected is obtained; determining a processing strategy of the image to be detected according to the intermediate layer vector and a historical processing strategy of a historical image, wherein the processing strategy comprises a model used to process the image to be detected and a processing region of the model; processing a processing region of the image to be detected using the model corresponding to the processing strategy to obtain a feature map; and identifying an image defect of the image to be detected according to the feature map. Therefore, the intermediate layer vector of the image to be detected can be assisted according to the historical processing strategy of the historical image to determine the appropriate model of the image to be detected and the processing region of the image to be detected. The processing region is processed by the corresponding model to obtain the feature map, so that the image defect of the image to be detected is identified according to the feature map. The effect of using the appropriate model to process the processing region of the image to be detected is achieved, and the accuracy of defect identification of the image to be detected is improved.

[0088] Other examples of the embodiment are described in the above examples, which will not be described here.

[0089] The embodiment also provides a defect detection system, which comprises the device or module, and the above defect detection method is realized by the combination of the device or module. Specific examples are described in the above examples, which will not be described here.

[0090] As Figure 5 shown, the embodiment of the application provides an electronic device, which comprises a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114,

[0091] The memory 113 is used to store a computer program.

[0092] In an embodiment of the application, the processor 111 is used to execute the program stored in the memory 113, so as to realize the defect detection method provided by any one of the above method embodiments.

[0093] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to realize the defect detection method provided by any one of the above method embodiments.

[0094] The apparatus embodiments described above are only illustrative, and the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the embodiments or some parts of the embodiments.

[0096] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order in which they are described, unless specifically indicated as such. It is also to be understood that additional or alternative steps can be employed.

[0097] The above description is merely illustrative of the application and should not be taken as limiting. Numerous modifications and variations underlying the general principles of the applications can be made by those of ordinary skill in the art without departing from the spirit or scope of the application. Therefore, the application is not to be limited to the embodiments described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A defect detection method characterized by, Comprise: In the case of obtaining a to-be-detected image, obtaining an intermediate layer vector of the to-be-detected image; According to the intermediate layer vector and the historical processing strategy of the historical image, the processing strategy of the to-be-detected image is determined, wherein the processing strategy includes the model for processing the to-be-detected image and the processing area of the model determined from a plurality of models; Using the model corresponding to the processing strategy to process the processing area of the to-be-detected image, a feature map is obtained; According to the feature map, the image defect of the to-be-detected image is identified; Wherein, the using the model corresponding to the processing strategy to process the processing area of the to-be-detected image, a feature map is obtained, including: when the processing strategy is to use a global model to process a first processing area of the to-be-detected image, inputting the first processing area of the to-be-detected image into the global model to obtain the feature map, wherein the model weight of the global model is converted from a first floating point number to a first integer; when the processing strategy is to use a local attention model to process a second processing area of the to-be-detected image, inputting the second processing area of the to-be-detected image into the local attention model to obtain the feature map, wherein the model weight of the local attention model is converted from a second floating point number to a half-precision floating point number.

2. The method of claim 1, wherein, The using the model corresponding to the processing strategy to process the processing area of the to-be-detected image, a feature map is obtained, including: When the processing strategy is to use a global model to process a third processing area of the to-be-detected image and to use a local attention model to process a fourth processing area of the to-be-detected image, inputting the third processing area of the to-be-detected image into the global model and inputting the fourth processing area of the to-be-detected image into the local attention model; Combining the global feature output by the global model with the local feature output by the local attention model to obtain the feature map.

3. The method of claim 2, wherein, The combining the global feature output by the global model with the local feature output by the local attention model to obtain the feature map includes: After upsampling and spatial mapping operations are performed on the global feature and the local feature, spatial alignment is performed; The aligned global feature and the local feature are spliced into a multi-scale combined feature; The multi-scale combined feature is input into a deformable convolution layer, and pixel dislocation compensation is performed on the multi-scale combined feature by the deformable convolution layer; Adaptive sampling is performed on the compensated multi-scale combined feature to obtain the feature map.

4. The method of claim 1, wherein, According to the intermediate layer vector and the historical processing strategy of the historical image, the processing strategy of the to-be-detected image is determined, including: According to the historical image, a historical intermediate layer output vector of the historical image is determined; According to the similarity between the intermediate layer vector of the to-be-detected image and the historical intermediate layer output vector of the historical image, a target historical image is determined from the historical images; The processing strategy of the target historical image is determined as the processing strategy of the to-be-detected image.

5. The method of claim 1, wherein, According to the intermediate layer vector and the historical processing strategy of the historical image, the processing strategy of the to-be-detected image is determined, including: In a case where the processing strategy is not determined, the image to be detected is input into a strategy network, an intermediate layer vector of the image to be detected is extracted by the strategy network, and the processing strategy is predicted according to the intermediate layer vector.

6. A defect detection apparatus characterized by comprising: The method comprises the following steps: An acquisition module is configured to acquire an intermediate layer vector of the image to be detected in a case where the image to be detected is acquired. A determination module is configured to determine a processing strategy of the image to be detected according to the intermediate layer vector and a historical processing strategy of a historical image, wherein the processing strategy comprises a model used for processing the image to be detected and a processing region of the model. A processing module is configured to process the processing region of the image to be detected by using the model corresponding to the processing strategy, to obtain a feature map. An identification module is configured to identify an image defect of the image to be detected according to the feature map. The processing of the processing region of the image to be detected by using the model corresponding to the processing strategy to obtain the feature map comprises: in a case where the processing strategy is to process a first processing region of the image to be detected by using a global model, inputting the first processing region of the image to be detected into the global model to obtain the feature map, wherein a model weight of the global model is converted from a first floating-point number to a first integer; and in a case where the processing strategy is to process a second processing region of the image to be detected by using a local attention model, inputting the second processing region of the image to be detected into the local attention model to obtain the feature map, wherein a model weight of the local attention model is converted from a second floating-point number to a half-precision floating-point number.

7. An electronic device, comprising: The method comprises the following steps: At least one communication interface; At least one bus connected with the at least one communication interface; At least one processor connected with the at least one bus; At least one memory connected with the at least one bus, wherein the memory stores a computer program, and the processor executes the computer program to implement the defect detection method in any one of claims 1 to 5.

8. A computer readable storage medium, characterized in that, The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the defect detection method in any one of claims 1 to 5. The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the defect detection method in any one of claims 1 to 5.

Images