Predetermined position-based industrial defect detection method, device, equipment and storage medium

Abstract

The application relates to the technical field of visual detection, and discloses an industrial defect detection method, device and equipment based on pre-positioning and a storage medium. The method comprises the following steps: inputting an industrial product image to be detected into a pre-trained defect pre-positioning model to obtain a defect segmentation heat map, each pixel point of the defect segmentation heat map has a corresponding defect confidence; based on the defect segmentation heat map, a pre-positioned defect region is intercepted; and the defect region is input into a pre-trained defect detection model to obtain a defect category and a detection frame position. According to the scheme, a two-stage detection process of pre-positioning and fine detection is adopted, high-confidence candidate regions are screened through a segmentation heat map, and then high-precision target detection is performed in a local window, so that invalid calculation is greatly reduced, and the detection accuracy and inference efficiency are considered.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and more specifically, to a method, apparatus, equipment, and storage medium for industrial defect detection based on pre-positioning. Background Technology

[0002] In modern industrial production systems, product quality is a key factor determining a company's core competitiveness and ensuring the safety of end users. This is especially true in strategic emerging industries such as photovoltaic energy, where the manufacturing quality of critical components like solar panels is paramount. Microscopic defects on the product surface, such as black cores, broken grids, and short-circuit spots, not only significantly reduce photoelectric conversion efficiency but can also trigger safety hazards like hot spot effects, leading to component failure or even safety accidents.

[0003] Currently, the industry still relies heavily on manual visual inspection for the detection of these defects. However, this method has inherent drawbacks such as high detection costs, low efficiency, and susceptibility to the subjective state and visual fatigue of the inspectors, making it completely unsuitable for the large-scale, high-speed, zero-defect intelligent quality control requirements of modern manufacturing.

[0004] Despite the progress made in deep learning-based visual inspection technology, industrial defect detection still faces significant challenges: defect areas are typically very small, sparsely distributed, and vary greatly in scale. To capture minute defects, industrial cameras usually acquire extremely high-resolution images, resulting in data volumes far exceeding the processing capabilities of general-purpose object detection models. Directly performing intensive computations on the entire image leads to enormous computational overhead and memory consumption. These issues severely restrict the practical application of industrial defect detection in terms of accuracy and efficiency. Summary of the Invention

[0005] This application provides a pre-positioning-based industrial defect detection method, apparatus, equipment, and storage medium that significantly improves detection efficiency while ensuring detection accuracy.

[0006] According to one aspect of the embodiments of this application, a pre-positioning-based industrial defect detection method is provided, comprising: The image of the industrial product to be detected is input into a pre-trained defect pre-location model to obtain a defect segmentation heatmap, where each pixel of the defect segmentation heatmap has a corresponding defect confidence level. Based on the defect segmentation heatmap, the pre-positioned defect region is extracted; The defect region is input into a pre-trained defect detection model to obtain the defect category and the location of the detection box.

[0007] In one implementation, the step of extracting a pre-positioned defect region based on the defect segmentation heatmap includes: The defect segmentation heatmap is cropped using a sliding window method to obtain multiple cropped windows; Calculate the sum of confidence scores for each pixel within each window; Window with a total confidence score greater than or equal to a preset threshold is retained, and window with a total confidence score less than the preset threshold is discarded; The reserved window is mapped to the corresponding position in the initial industrial product image, and the image is cropped based on the position to obtain the pre-positioned defect area.

[0008] In one implementation, before inputting the image of the industrial product to be detected into a pre-trained defect pre-location model, the method further includes: For a defect sample image, an anomaly segmentation mask is generated based on the region enclosed by its corresponding defect detection box, thus obtaining the segmentation mask dataset of the defect sample. Randomly crop several regions from defect-free images and set their corresponding masks to zero to obtain a segmentation mask dataset of normal samples. A CLIP-based multimodal defect segmentation model is selected and trained using a segmentation mask dataset of defect samples and normal samples to obtain the trained defect pre-location model.

[0009] In one implementation, it further includes: The loss function is based on minimizing the pixel-wise binary cross-entropy loss. The defect pre-location model is trained based on the loss function until the model converges.

[0010] In one implementation, before inputting the defect region into a pre-trained defect detection model, the method further includes: Randomly initialize the parameters of the object detection model; The industrial defect image dataset is acquired and preprocessed, and a training dataset is generated using a preset sample selection strategy. The training dataset is input into the object detection model for training.

[0011] In one implementation, the step of acquiring an industrial defect image dataset and preprocessing it to generate a training dataset using a preset sample selection strategy includes: Load the annotation information of the industrial defect image dataset to obtain the coordinates of the upper left and lower right corner points of the defect bounding box; Based on the coordinates of the upper left and lower right corners of the defect bounding box, the image is cropped using a rectangle of a preset size. If the number of defects within the cropping box is greater than 1, the cropping region is discarded. If the number of defects within the cropping box is less than or equal to 1, the cropped image and its corresponding coordinates, category label, and path information are retained and added to the training dataset. No defective samples are randomly cropped from the normal region of the defective image. Multiple no-defect samples and defective samples are mixed and stitched together to obtain reconstructed samples. The reconstructed samples are then added to the training dataset.

[0012] In one implementation, it further includes: Based on the classification loss function and the regression loss function, a weighted sum is performed to obtain the constructed multi-task loss function; The model is optimized based on the multi-task loss function, and the model parameters are updated using stochastic gradient descent until the model converges, thus obtaining the trained defect detection model.

[0013] According to another aspect of the embodiments of this application, a pre-positioning-based industrial defect detection device is also provided, comprising: The defect pre-location module is used to input the image of the industrial product to be detected into the pre-trained defect pre-location model to obtain a defect segmentation heatmap, wherein each pixel of the defect segmentation heatmap has a corresponding defect confidence level. The interception module is used to intercept the pre-positioned defect region based on the defect segmentation heatmap; The defect detection module is used to input the defect region into a pre-trained defect detection model to obtain the defect category and the location of the detection box.

[0014] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described pre-position-based industrial defect detection method through the computer program.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, which is configured to execute the above-described industrial defect detection method based on pre-positioning when it is run.

[0016] The technical solutions provided in this application embodiment may include the following beneficial effects: This application provides a pre-location-based industrial defect detection method. First, an image of the industrial product to be detected is input into a pre-trained defect pre-location model to obtain a defect segmentation heatmap. Based on the defect segmentation heatmap, pre-located defect regions are extracted. These defect regions are then input into a pre-trained defect detection model to obtain the defect category and detection box position. This application innovatively adopts a two-stage detection process of "pre-location + fine detection." First, the defect pre-location model quickly generates a defect segmentation heatmap, filtering out high-confidence candidate regions. This concentrates subsequent computational resources on a very small number of suspicious regions, avoiding the huge computational overhead of global traversal analysis of the entire high-resolution image in existing technologies. This method can significantly reduce invalid computation, achieving an order-of-magnitude improvement in detection speed, effectively solving the bottleneck problem of slow processing speed for high-resolution industrial images, and meeting the real-time quality inspection needs of high-speed modern industrial production. Since the detection model only needs to process local windows containing potential defects, it can distinguish defects with higher resolution and more focused feature extraction capabilities. The focused analysis of local windows reduces interference from complex backgrounds, further ensuring the accuracy of defect classification and location. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of an optional pre-position-based industrial defect detection method according to an embodiment of this application; Figure 2 This is a schematic diagram of a training method for a defect pre-location model according to an embodiment of this application; Figure 3 This is a schematic diagram of a training method for a defect detection model according to an embodiment of this application; Figure 4 This is a flowchart of another defect detection method according to an embodiment of this application; Figure 5 This is a schematic diagram of an industrial defect detection device based on pre-positioning according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] To address the shortcomings of existing industrial defect detection methods when processing high-resolution images, such as low computational efficiency, inaccurate defect region localization, severe imbalance in positive and negative sample distribution, and lack of effective pre-localization capabilities using large-scale visual models, this paper proposes an industrial defect detection method that can significantly improve detection efficiency while maintaining detection accuracy. This method aims to alleviate the computational burden caused by excessively high image resolution, improve the detection of missed and false positives for small or sparse defects, and enhance the generalization and practicality of the model in real-world industrial scenarios.

[0021] The pre-position-based industrial defect detection method of this application, as described below with reference to the accompanying drawings, will be described in detail. Figure 1 As shown, the method mainly includes the following steps: S101 inputs the image of the industrial product to be detected into the pre-trained defect pre-location model to obtain a defect segmentation heatmap. Each pixel in the defect segmentation heatmap has a corresponding defect confidence level.

[0022] First, images of the industrial products to be inspected are acquired, such as images of photovoltaic solar panels, printed circuit boards, metal stampings, etc. This application does not impose specific limitations on the applicable products.

[0023] In one implementation, an industrial camera or webcam is used to capture images of the product at a fixed workstation on the production line. During the capture process, a high-brightness, uniform light source is used to illuminate the image to eliminate glare interference. The captured raw images are then preprocessed by format conversion and resolution standardization to obtain an image to be detected that meets the model input requirements.

[0024] Furthermore, the image to be detected is input into a pre-trained defect pre-location model to obtain a defect segmentation heatmap. Each pixel Each has a corresponding defect confidence level. .

[0025] The defect segmentation heatmap in this embodiment has the same spatial dimensions as the input image, meaning each pixel corresponds one-to-one, ensuring that the heatmap accurately reflects the probability of a defect at each location in the original image. The grayscale value of each pixel in the heatmap is typically normalized to [0,1], representing the confidence level that the location belongs to a defect. The closer the value is to 1, the higher the probability that the pixel belongs to a defect; the closer the value is to 0, the higher the probability that it belongs to the normal background.

[0026] As a dense, pixel-level defect probability distribution map, the defect segmentation heatmap provides key spatial attention guidance for the two-stage detection process of this application, and is the foundation for achieving efficient and accurate defect detection.

[0027] S102 extracts the pre-positioned defect area based on the defect segmentation heatmap.

[0028] In one implementation, based on a defect segmentation heatmap, pre-positioned defect regions are extracted, including: performing sliding window extraction on the defect segmentation heatmap to obtain multiple extracted windows; calculating the sum of confidence scores of pixels within each window; retaining windows with a sum of confidence scores greater than or equal to a preset threshold, and discarding windows with a sum of confidence scores less than the preset threshold; mapping the retained windows to corresponding positions in an initial industrial product image, and extracting the image based on the position to obtain the pre-positioned defect regions.

[0029] Specifically, the acquired defect segmentation heatmap is viewed through a sliding window. Capture the sliding window; the window size is... Step size is Get multiple windows .

[0030] Furthermore, the sum of confidence scores for each pixel within each window is calculated:

[0031] Set threshold Perform the screening, if Then keep the window Otherwise, discard; where, threshold The specific value can be set according to the actual situation, and this application embodiment does not impose any restrictions.

[0032] Based on the obtained reserved window Locate the initial image The corresponding position is then selected, and that area is extracted to obtain the pre-positioned defect window. .

[0033] S103 inputs the defect region into the pre-trained defect detection model to obtain the defect category and the location of the detection box.

[0034] In one embodiment of this application, the pre-positioned defect area is... Input the data into a pre-trained defect detection model to perform detection and obtain the defect category. and the position of the detection frame (Local coordinates relative to the clipping region).

[0035] Furthermore, the positions of the detection boxes output by the model are restored. For the k-th detection box, its relative coordinates in the local window are added to the absolute offset of the window in the original image to obtain its global coordinates in the original image.

[0036] At the same time, if overlapping windows appear, i.e., multiple detection boxes... The intersection-union ratio (IoU) between them exceeds the threshold. Then, the non-maximum suppression (NMS) algorithm is needed to filter the boxes, and the optimal detection boxes are selected based on the class confidence scores output by the model.

[0037] in, This represents the category confidence score output by the model. The final defect localization result is... .

[0038] When the input industrial product image does not contain any defects, the defect segmentation heatmap generated by the defect pre-location model will exhibit an overall low confidence profile. In the subsequent sliding window filtering step, the sum of the confidence scores of all windows is lower than a preset threshold, resulting in no candidate regions being retained. At this point, the defect detection model will not need to perform any calculations, and the system will directly output an empty detection result set, thereby efficiently and accurately determining that the product is defect-free. This mechanism ensures that the method of this application can quickly make judgments when dealing with normal products, avoiding unnecessary consumption of computational resources and further improving the overall detection efficiency.

[0039] The defect detection method of this application firstly utilizes a pre-location model to quickly generate a defect segmentation heatmap, filtering out high-probability candidate regions. This concentrates subsequent computational resources on a very small number of local windows, avoiding the high computational overhead of global traversal and significantly improving detection speed. Secondly, the refined detection model only processes the pre-located high-probability windows, enabling focused analysis of defect features at higher resolution. This effectively overcomes the problems of missed detection of small targets and background interference, ensuring the accuracy of defect classification and localization.

[0040] In one implementation, the defect pre-location model is trained before the image of the industrial product to be detected is input into the pre-trained defect pre-location model.

[0041] Specifically, for defect sample images, an abnormal segmentation mask is generated based on the region enclosed by the corresponding defect detection box to obtain the segmentation mask dataset of defect samples; several regions are randomly cropped from defect-free images and their corresponding masks are set to zero to obtain the segmentation mask dataset of normal samples; a CLIP-based multimodal defect segmentation model is selected and trained using the segmentation mask datasets of defect samples and normal samples to obtain a trained defect pre-location model.

[0042] This application employs a detection-based segmentation strategy to construct a segmentation dataset. For defect sample images, the region enclosed by the defect detection box is directly regarded as an anomaly segmentation mask to construct a defect segmentation mask dataset. That is, for the... Sample images If its corresponding detection box is Then the corresponding binary segmentation mask is generated. Where:

[0043] In this mask, all pixels inside the detection box are set to 1 (representing a defect), and pixels outside the detection box are set to 0, thus directly converting the detection box-level annotations into a pixel-level segmentation mask.

[0044] Simultaneously, defect-free negative samples are extracted and added to the segmentation dataset, i.e., from defect-free images... Randomly crop several regions And set its corresponding mask. Thus forming a complete training set. .

[0045] Furthermore, a CLIP-based multimodal defect segmentation model is selected as the framework, denoted as [model name missing]. Using the complete training set Training, in one implementation, also includes minimizing pixel-wise binary cross-entropy loss as the loss function:

[0046] A defect pre-location model is trained based on a loss function until the model converges. This is the defect probability map predicted by the model. After training, the defect prelocation model AnomalyCLIP is obtained.

[0047] To facilitate understanding of the training method of the defect pre-location model in this application, the following is in conjunction with the appendix. Figure 2 Further description. For example... Figure 2 As shown, a segmentation dataset is constructed based on defect data using a detection-based segmentation strategy. The regions enclosed by detection boxes are directly treated as anomaly segmentation masks to construct a defect segmentation mask dataset. Simultaneously, defect-free negative samples are truncated and added to the segmentation dataset. A CLIP-based multimodal defect segmentation model is selected as the framework, and the obtained segmentation mask dataset is used for training to obtain the trained defect segmentation pre-location model, AnomalyCLIP. The input of this model is an industrial image, and the output is the image's segmentation mask.

[0048] In one implementation, before inputting the defect region into the pre-trained defect detection model, the method further includes training the defect detection model. This includes randomly initializing the parameters of the target detection model; acquiring an industrial defect image dataset for preprocessing; generating a training dataset using a preset sample selection strategy; and inputting the training dataset into the target detection model for training.

[0049] Specifically, the target detection model parameters are initialized randomly. .

[0050] Currently, high-quality labeled samples are scarce, and the positive and negative samples are severely imbalanced, making it difficult for existing sampling strategies to effectively construct a balanced training set. This application addresses industrial defect image datasets. Preprocessing is performed, and the following sample selection strategy is used to generate training samples.

[0051] This includes loading the annotation information of the industrial defect image dataset to obtain the coordinates of the top left and bottom right corners of the defect bounding box; cropping the image using a rectangle of a preset size based on the coordinates of the top left and bottom right corners of the defect bounding box; discarding the cropped area if the number of defects within the cropping box is greater than 1, and retaining the cropped image and its corresponding coordinates, category label, and path information if the number of defects within the cropping box is less than or equal to 1, and adding it to the training dataset.

[0052] Obtain the top-left corner of the defect's bounding box by loading annotation information. and the bottom right corner The coordinates, using a size of The rectangular bounding box is used to trim defects. The specific sampling criteria are as follows: If the defect is located at one of the four vertices of the image, then the image is cropped using the defect as one of the vertices of the rectangle.

[0053] If the defect is not located at an image vertex, then first define the left boundaries as follows: After obtaining the top left boundary, from the interval... Uniform random sampling cut start point This results in the cropping frame.

[0054] Then determine whether the cropping area contains multiple defects, assuming the current cropping area is... If the following conditions are met:

[0055] If the number of defects is greater than 1, the cropped sample is discarded to avoid label confusion; if the number is 1, it indicates that the cropped box contains only a single defect, meeting the requirements for a positive sample, so the cropped sample is retained, and its corresponding image, coordinates, category label, and path information are added to the training dataset for subsequent model training. If the number is 0, it indicates that the cropped box does not contain any defects, meeting the requirements for a negative sample, so the cropped sample is retained as a negative sample.

[0056] At the same time, to ensure a balanced ratio of positive and negative samples, the number of positive samples should be controlled. With the number of negative samples satisfy ,in, , which is the preset proportional coefficient.

[0057] Furthermore, it also includes using an industrial defect detection algorithm based on semantic capture of reconstructed samples to construct training samples, thus solving the problem of uneven distribution of positive and negative samples after sampling. Defect-free samples are randomly cropped from the normal region of the defect image, and multiple defect-free samples and defect samples are combined and stitched together using an image stitching algorithm to form reconstructed samples, which are then added to the training dataset.

[0058] Randomly extract defect-free samples from the normal area of ​​the defect image. Multiple defect-free and defective samples are then stitched together using Mosaic to create a reconstructed image. The stitched image may contain only defect-free samples, or it may contain both. The stitched image appears more complex visually, containing multiple objects or textures, which can easily lead the model to misclassify it as containing defects. By allowing the model to learn to correctly classify these difficult reconstructed samples, the robustness of the model can be significantly improved.

[0059] For example, given four images and their corresponding tags The splicing operation is defined as follows:

[0060]

[0061] in, This indicates vertical splicing. Using the input size, these reconstructed samples are also added to the dataset to train the model, improving the model's robustness and generalization ability.

[0062] In one implementation, it also includes a classification loss function. and regression loss function We then perform a weighted summation to obtain the constructed multi-task loss function:

[0063] in, , To balance the hyperparameters, optimization is performed based on a multi-task loss function, and stochastic gradient descent is used to update the model parameters until the model converges, resulting in a well-trained defect detection model.

[0064]

[0065] in, The learning rate is used to iterate until convergence, resulting in a trained industrial defect detection model. .

[0066] To facilitate understanding of the training method of the defect detection model in this application, the following is in conjunction with the appendix. Figure 3 Further description, such as Figure 3 As shown, defect images are obtained, samples are constructed based on the defect images, and a target detection model is trained to obtain a trained defect detection model.

[0067] To facilitate understanding of the entire testing process of this application, the following is in conjunction with the appendix. Figure 4 Further description. For example... Figure 4 As shown, a high-resolution defect image to be detected is fed into a trained defect pre-location model. The model outputs a defect segmentation heatmap, where each pixel has a corresponding defect confidence score. A sliding window approach is used to extract multiple windows from the defect segmentation heatmap. The sum of the confidence scores of pixels within each window is calculated, and a threshold is set for filtering. Windows higher than the threshold are retained, while those lower are discarded. Based on the retained windows, the pre-located defect window is obtained. This region is then input into the trained defect detection model for detection, obtaining the defect category and the location of the detection box. The locations of the detection boxes output by the model are restored, and it is determined whether there is overlap. If so, the NMS algorithm is used for filtering, ultimately yielding the defect localization result.

[0068] The defect detection method provided in this application achieves a significant improvement in detection accuracy and efficiency, with the following specific benefits: (1) Reduce labeling costs and improve data utilization efficiency: By adopting the "inspection-based segmentation" strategy, the easily obtainable detection box labels are directly converted into pixel-level segmentation masks to build a high-quality segmentation dataset. This eliminates the need to rely on expensive and time-consuming fine manual segmentation labeling, significantly reducing data preparation costs.

[0069] (2) Enhance the ability to perceive and locate defects: AnomalyCLIP, a defect pre-location model based on CLIP multimodal prior, is constructed. By utilizing its powerful visual-semantic alignment capability, it effectively enhances the perception and location capabilities of weak defects and complex texture defects, and significantly improves the accuracy and robustness of the coarse location stage.

[0070] (3) Optimize training sample quality and model convergence stability: Design a non-uniform sampling and multi-defect filtering mechanism to dynamically adapt the defect location and remove regions containing multiple defects when cropping training samples, so as to ensure the semantic purity and localization consistency of training samples. (4) Effectively solve the problem of sample imbalance and enhance the model's discriminative power: A reconstruction sample generation method based on Mosaic splicing is proposed. Defect-free blocks are randomly extracted from normal areas and spliced ​​together, which effectively alleviates the problem of severe imbalance between positive and negative samples in industrial scenarios and enhances the model's ability to discriminate normal textures. (5) Achieving the best balance between detection accuracy and inference efficiency: The two-stage detection process of "pre-positioning + fine detection" is adopted. First, high-confidence candidate regions are screened by segmenting heatmaps, and then high-precision target detection is performed in local windows, which greatly reduces invalid calculations and takes into account both detection accuracy and inference efficiency.

[0071] According to another aspect of the embodiments of this application, a pre-position-based industrial defect detection apparatus for implementing the above-described pre-position-based industrial defect detection method is also provided. For example... Figure 5 As shown, the device includes: The defect pre-location module 501 is used to input the image of the industrial product to be detected into the pre-trained defect pre-location model to obtain a defect segmentation heatmap. Each pixel in the defect segmentation heatmap has a corresponding defect confidence level. The interception module 502 is used to intercept pre-positioned defect areas based on the defect segmentation heatmap; The defect detection module 503 is used to input the defect region into the pre-trained defect detection model to obtain the defect category and the location of the detection box.

[0072] It should be noted that the industrial defect detection device based on pre-positioning provided in the above embodiments is only illustrated by the division of the above functional modules when executing the industrial defect detection method based on pre-positioning. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the equipment can be divided into different functional modules to complete all or part of the functions described above. In addition, the industrial defect detection device based on pre-positioning provided in the above embodiments and the industrial defect detection method based on pre-positioning are based on the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0073] According to another aspect of the embodiments of this application, an electronic device corresponding to the pre-position-based industrial defect detection method provided in the foregoing embodiments is also provided, for executing the pre-position-based industrial defect detection method described above.

[0074] Please refer to Figure 6 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 6 As shown, the electronic device includes: a processor 600, a memory 601, a bus 602, and a communication interface 603. The processor 600, the communication interface 603, and the memory 601 are connected via the bus 602. The memory 601 stores a computer program that can run on the processor 600. When the processor 600 runs the computer program, it executes the pre-positioning-based industrial defect detection method provided in any of the foregoing embodiments of this application.

[0075] The memory 601 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 603 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0076] Bus 602 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 601 is used to store programs. After receiving execution instructions, processor 600 executes the program. The pre-positioning-based industrial defect detection method disclosed in any of the foregoing embodiments of this application can be applied to processor 600, or implemented by processor 600.

[0077] The processor 600 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 600 or by instructions in software form. The processor 600 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 601. Processor 600 reads the information in memory 601 and, in conjunction with its hardware, completes the steps of the above method.

[0078] The electronic device provided in this application embodiment and the industrial defect detection method based on pre-positioning provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0079] According to another aspect of the embodiments of this application, a computer-readable storage medium corresponding to the pre-position-based industrial defect detection method provided in the foregoing embodiments is also provided, wherein a computer program (i.e., a program product) is stored thereon, and when the computer program is run by a processor, it executes the pre-position-based industrial defect detection method provided in any of the foregoing embodiments.

[0080] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0081] The computer-readable storage medium provided in the above embodiments of this application and the industrial defect detection method based on prepositioning provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0082] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0083] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A pre-positioning-based industrial defect detection method, characterized in that, include: The image of the industrial product to be detected is input into a pre-trained defect pre-location model to obtain a defect segmentation heatmap, where each pixel of the defect segmentation heatmap has a corresponding defect confidence level. Based on the defect segmentation heatmap, the pre-positioned defect region is extracted; The defect region is input into a pre-trained defect detection model to obtain the defect category and the location of the detection box.

2. The method according to claim 1, characterized in that, The step of extracting pre-positioned defect regions based on the defect segmentation heatmap includes: The defect segmentation heatmap is cropped using a sliding window method to obtain multiple cropped windows; Calculate the sum of confidence scores for each pixel within each window; Window with a total confidence score greater than or equal to a preset threshold is retained, and window with a total confidence score less than the preset threshold is discarded; The reserved window is mapped to the corresponding position in the initial industrial product image, and the image is cropped based on the position to obtain the pre-positioned defect area.

3. The method according to claim 1, characterized in that, Before inputting the image of the industrial product to be detected into the pre-trained defect pre-location model, the following steps are also included: For a defect sample image, an anomaly segmentation mask is generated based on the region enclosed by its corresponding defect detection box, thus obtaining the segmentation mask dataset of the defect sample. Randomly crop several regions from defect-free images and set their corresponding masks to zero to obtain a segmentation mask dataset of normal samples. A CLIP-based multimodal defect segmentation model is selected and trained using a segmentation mask dataset of defect samples and normal samples to obtain the trained defect pre-location model.

4. The method according to claim 3, characterized in that, Also includes: The loss function is based on minimizing the pixel-wise binary cross-entropy loss. The defect pre-location model is trained based on the loss function until the model converges.

5. The method according to claim 1, characterized in that, Before inputting the defect region into the pre-trained defect detection model, the following steps are also included: Randomly initialize the parameters of the object detection model; The industrial defect image dataset is acquired and preprocessed, and a training dataset is generated using a preset sample selection strategy. The training dataset is input into the object detection model for training.

6. The method according to claim 5, characterized in that, The process of acquiring and preprocessing the industrial defect image dataset, and generating a training dataset using a preset sample selection strategy, includes: Load the annotation information of the industrial defect image dataset to obtain the coordinates of the upper left and lower right corner points of the defect bounding box; Based on the coordinates of the upper left and lower right corners of the defect bounding box, the image is cropped using a rectangle of a preset size. If the number of defects within the cropping box is greater than 1, the cropping region is discarded. If the number of defects within the cropping box is less than or equal to 1, the cropped image and its corresponding coordinates, category label, and path information are retained and added to the training dataset. No defective samples are randomly cropped from the normal region of the defective image. Multiple no-defect samples and defective samples are mixed and stitched together to obtain reconstructed samples. The reconstructed samples are then added to the training dataset.

7. The method according to claim 5, characterized in that, Also includes: Based on the classification loss function and the regression loss function, a weighted sum is performed to obtain the constructed multi-task loss function; The model is optimized based on the multi-task loss function, and the model parameters are updated using stochastic gradient descent until the model converges, thus obtaining the trained defect detection model.

8. An industrial defect detection device based on pre-positioning, characterized in that, include: The defect pre-location module is used to input the image of the industrial product to be detected into the pre-trained defect pre-location model to obtain a defect segmentation heatmap, wherein each pixel of the defect segmentation heatmap has a corresponding defect confidence level. The interception module is used to intercept the pre-positioned defect region based on the defect segmentation heatmap; The defect detection module is used to input the defect region into a pre-trained defect detection model to obtain the defect category and the location of the detection box.

9. An electronic device, characterized in that, It includes a processor and a memory storing program instructions, the processor being configured to, when executing the program instructions, perform the pre-position-based industrial defect detection method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, It stores computer-readable instructions that are executed by a processor to implement the pre-position-based industrial defect detection method as described in any one of claims 1 to 7.

Images