A defect detection method and device, electronic equipment and storage medium

Abstract

The application discloses a kind of defect detection method, device, electronic equipment and storage medium.The present application relates to the technical field of equipment surface defect detection, method includes: obtaining defect test picture;Defect test picture is input to the target defect detection model of pre-training, and defect test result is determined;Wherein, the target defect detection model is based on segmentation model and binary classification segmentation model construction, and the model determined by training with optimization rule as index.The technical scheme of the present application is based on segmentation model and binary classification segmentation model construction, and the target defect detection model determined by training with optimization rule as index, and then obtaining defect test picture;Defect test picture is input to the target defect detection model of pre-training, and defect test result is determined, aims at retraining to obtain the defect edge feature of industrial equipment, improve the accuracy of detection result, discover potential hidden danger of equipment in time, improve the safety of equipment work.

Description

Technical Field

[0001] This invention relates to the field of equipment surface defect detection technology, and in particular to a defect detection method, apparatus, electronic device and storage medium. Background Technology

[0002] In industrial production, large-scale production and processing equipment faces increasing safety risks over time due to wear, aging, corrosion, and other factors. The lifespan specified in the equipment manual does not accurately reflect the actual condition of the equipment, and relying solely on manual, periodic quality inspections is time-consuming, labor-intensive, and prone to overlooking minor non-standard issues.

[0003] Under current technological conditions, the task of detecting surface defects in large industrial equipment based on machine vision and artificial intelligence technologies faces the following problems: scarce samples, few similar devices, and difficulty in collecting a sufficient number of defect images to train the model; unclear distinction between foreground and background in defect images, and difficulty in extracting defect features due to diverse backgrounds; and the inability of ordinary "box selection" target detection models to extract defect edge features, resulting in low accuracy of detection results. Summary of the Invention

[0004] This invention provides a defect detection method, apparatus, electronic device, and storage medium, which aims to retrain and acquire defect edge features of industrial equipment, improve the accuracy of detection results, promptly detect potential hazards in equipment, and enhance the safety of equipment operation.

[0005] According to one aspect of the present invention, a defect detection method is provided, the method comprising:

[0006] Obtain defect test images;

[0007] The defect test image is input into a pre-trained target defect detection model to determine the defect test result; the target defect detection model is a model built based on a segmentation model and a binary classification segmentation model and trained using optimization rules as indicators.

[0008] Optionally, the method for training the target defect detection model includes: acquiring surface defect sample images and visual segmentation data images of industrial equipment; performing knowledge distillation on the segmentation model and the binary classification segmentation model based on the surface defect sample images to obtain the surface defect detection model; and training the surface defect detection model based on the surface defect sample images and the visual segmentation data images using optimization rules as indicators to determine the target defect detection model.

[0009] Optionally, based on the surface defect sample images and visual segmentation data images, knowledge distillation is performed on the segmentation model and the binary classification segmentation model to obtain a surface defect detection model. This includes: inputting the surface defect sample images and visual segmentation data images into the segmentation model for knowledge distillation to determine a first sample image mask and a first data image mask; inputting the surface defect sample images and visual segmentation data images into the binary classification segmentation model for training to determine a second sample image mask and a second data image mask; determining a first target based on the first data image mask and the second data image mask, and determining a second target based on the first sample image mask and the second sample image mask; determining a loss function value based on the first target and the second target; determining whether the loss function value is less than or equal to a preset function value; if the loss function value is greater than the preset function value, then returning to the steps of inputting the surface defect sample images and visual segmentation data images into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample images and visual segmentation data images into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; if the loss function value is less than or equal to the preset function value, then determining the current binary classification segmentation model as a surface defect detection model.

[0010] Optionally, determining the first target based on the first data image mask and the second data image mask includes: determining the number of pixels in the surface defect sample image, and determining the first pixel coordinates of the first data image mask corresponding to the number of pixels and the second pixel coordinates of the second data image mask corresponding to the number of pixels; determining the first target based on the number of pixels, the first pixel coordinates, and the second pixel coordinates.

[0011] Optionally, determining the second target based on the first sample image mask and the second sample image mask includes: determining the number of pixels in the surface defect sample image, and determining the third pixel coordinate of the first sample image mask corresponding to the number of pixels and the fourth pixel coordinate of the second sample image mask corresponding to the number of pixels; determining the second target based on the number of pixels, the third pixel coordinate, and the fourth pixel coordinate.

[0012] Optionally, determining the loss function value based on the first objective and the second objective includes: determining the loss weight; and determining the loss function value based on the loss weight, the first objective, and the second objective.

[0013] Optionally, based on surface defect sample images, the surface defect detection model is trained using optimization rules as indicators to determine the target defect detection model, including: obtaining the main architecture model of the surface defect detection model; determining the classification model based on the main architecture model, the classification output layer, and the network model; inputting the surface defect sample images into the classification model to determine the defect category labels; keeping the underlying parameters of the surface defect detection model unchanged, and inputting the surface defect sample images and defect category labels into the surface defect detection model for retraining to determine the target defect detection model.

[0014] According to another aspect of the present invention, a defect detection device is provided, the device comprising:

[0015] The image acquisition module is used to acquire defect test images;

[0016] The result determination module is used to input defect test images into a pre-trained target defect detection model to determine the defect test results. The target defect detection model is a model built based on a segmentation model and a binary classification segmentation model, and trained and determined using optimization rules as indicators.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and a memory communicatively connected to the at least one processor;

[0019] The memory stores a computer program that can be executed by at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the defect detection method of any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided that stores computer instructions for causing a processor to execute and implement the defect detection method of any embodiment of the present invention.

[0021] The technical solution of this invention involves acquiring defect test images; inputting these images into a pre-trained target defect detection model to determine the defect test results; wherein the target defect detection model is a model constructed based on a segmentation model and a binary classification segmentation model, and trained using optimization rules as indicators. This technical solution aims to retrain and acquire defect edge features of industrial equipment, improve the accuracy of detection results, promptly identify potential hazards in equipment, and enhance the safety of equipment operation.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a defect detection method provided in Embodiment 1.

[0025] Figure 2 This is a schematic diagram of the structure of a defect detection device provided in Embodiment 2;

[0026] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1This is a flowchart illustrating a defect detection method provided in Embodiment 1. This embodiment is applicable to the detection of surface defects in industrial equipment. The method can be executed by a defect detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0031] S101. Obtain defect test images.

[0032] Among them, defect test images are sample images of surface defects in industrial equipment, and are used to test the surface defects of industrial equipment.

[0033] Specifically, it involves collecting and acquiring defect test images of surface defects in industrial equipment.

[0034] S102. Input the defect test image into the pre-trained target defect detection model to determine the defect test result.

[0035] The target defect detection model is constructed based on a segmentation model and a binary classification segmentation model, and trained using optimization rules as indicators. The defect test results include defect images and defect features. The defect images reflect the defect category of the defective part of the industrial equipment in the defect test image. Defect categories can be, for example, wear, aging, etc., which are not limited in this embodiment. Defect features represent the degree of defect corresponding to the defect category of the industrial equipment in the defect image, such as a wear defect degree of 30%, which is also not limited in this embodiment.

[0036] Segmentation models are typically used to segment defect test images to obtain image masks. One example is the Segment Anything Model (SAM), a general-purpose model that can segment any seen or unseen object in an image or video, outputting a mask that matches the edge features of the foreground object, similar to automatic image matting. Examples include object instance, material, part, contour, and text segmentation. Experimental results across various tasks demonstrate its powerful capabilities. The advantages of the SAM segmentation model are its strong versatility, enabling semantic segmentation of images of arbitrary size and format, and uniformly converting the mask to an image format for subsequent processing—flexible, universal, and a one-step solution; and its enhanced capabilities: experimental results across various tasks demonstrate superior power and performance. Compared to other techniques, it offers significant advantages in terms of accuracy, speed, and efficiency; it also exhibits strong generalization ability, showing good adaptability to images like industrial equipment defects—images rarely seen in daily life and not frequently used for training—and accurately delineating defect locations.

[0037] Image masking refers to various bitwise operations between images used to partially or completely hide parts of an object or element. Applying an image mask to an image object is like painting a mask over the background, thus completely or partially obscuring different parts of the image object, while the image within the mask remains unchanged. The binary classification segmentation model is a pre-defined basic model used for network training.

[0038] Specifically, after obtaining the defect test image, the defect test image is input into the pre-trained target defect detection model to determine the defect test result.

[0039] In one specific embodiment, the method for training a target defect detection model includes: acquiring surface defect sample images and visual segmentation data images of industrial equipment; performing knowledge distillation on a segmentation model and a binary classification segmentation model based on the surface defect sample images to obtain a surface defect detection model; and training the surface defect detection model based on the surface defect sample images and visual segmentation data images using optimization rules as indicators to determine the target defect detection model.

[0040] Among them, the surface defect sample images are pre-collected images of surface defects in industrial equipment used for model training; the visual segmentation data images are visual segmentation datasets.

[0041] Specifically, surface defect sample images and visual segmentation data images of industrial equipment are acquired. Then, based on the surface defect sample images, knowledge distillation is performed on the segmentation model and the binary classification segmentation model to obtain the surface defect detection model. Based on the surface defect sample images and the visual segmentation data images, the surface defect detection model is trained using optimization rules as indicators to determine the target defect detection model.

[0042] In one specific implementation, a surface defect detection model is obtained by performing knowledge distillation on a segmentation model and a binary classification segmentation model based on surface defect sample images and visual segmentation data images. This includes: inputting surface defect sample images and visual segmentation data images into the segmentation model for knowledge distillation to determine a first sample image mask and a first data image mask; inputting the surface defect sample images and visual segmentation data images into the binary classification segmentation model for training to determine a second sample image mask and a second data image mask; determining a first target based on the first and second data image masks, and determining a second target based on the first and second sample image masks; determining a loss function value based on the first and second targets; determining whether the loss function value is less than or equal to a preset function value; if the loss function value is greater than the preset function value, then returning to the steps of inputting surface defect sample images and visual segmentation data images into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample images and visual segmentation data images into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; if the loss function value is less than or equal to the preset function value, then determining the current binary classification segmentation model as a surface defect detection model.

[0043] The first sample image mask is the image mask after the segmentation model occludes the surface defect sample image; the first data image mask is the image mask after the segmentation model occludes the visual segmentation data image; the second sample image mask is the image mask after the binary classification segmentation model occludes the surface defect sample image; and the second data image mask is the image mask after the binary classification segmentation model occludes the visual segmentation data image.

[0044] The first target, also known as the soft target, is determined by calculating the Euclidean distance between the first and second data image masks; the second target, also known as the hard target, is determined by calculating the Euclidean distance between the first and second sample image masks.

[0045] Wherein, the loss function value is loss = aloss soft +(1-a)loss hard 'a' represents a pre-defined weight. The preset function value is a pre-defined function value used to determine whether the loss function value meets the requirements.

[0046] Specifically, after acquiring surface defect sample images and visual segmentation data images, the surface defect sample images and visual segmentation data images are first input into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask. Simultaneously, the surface defect sample images and visual segmentation data images are input into a binary classification segmentation model for training to determine the second sample image mask and the second data image mask. Then, based on the first data image mask and the second data image mask, the first target is determined, and based on the first sample image mask and the second sample image mask, the second target is determined. Further, the first target... Where N is the number of pixels in the current input surface defect sample image, (x 11i -x 21i ) represents the pixel coordinates of the first data image mask, (y... 11i -y 21i ) represents the pixel coordinates of the second data image mask, where i is the number of pixels, and the second target... Where N is the number of pixels in the current input surface defect sample image, (x 12i -x 22i (y) represents the pixel coordinates of the mask for the first sample image. 12i -y 22i Let be the pixel coordinates of the mask for the second sample image, and i be the number of pixels. Then, determine the loss function value based on the first and second objectives. The loss function value is loss = aloss. soft +(1-a)loss hard Determine if the loss function value is greater than, less than, or equal to a preset function value. If the loss function value is greater than the preset function value, return to the steps of inputting the surface defect sample image and visual segmentation data image into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample image and visual segmentation data image into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask. Continue to obtain masks and continue to determine the loss function value. If the loss function value is less than or equal to the preset function value, determine that the binary classification segmentation model trained in the current iteration is the surface defect detection model.

[0047] In one specific embodiment, determining a first target based on a first data image mask and a second data image mask includes: determining the number of pixels in a surface defect sample image, and determining the first pixel coordinates of the first data image mask corresponding to the number of pixels and the second pixel coordinates of the second data image mask corresponding to the number of pixels; determining the first target based on the number of pixels, the first pixel coordinates, and the second pixel coordinates.

[0048] Wherein, the first pixel coordinates are the coordinates corresponding to each pixel point in the first data image mask, and the second pixel coordinates are the coordinates corresponding to each pixel point in the second data image mask.

[0049] Specifically, the number of pixels in the surface defect sample image is determined, and the first pixel coordinates of the first data image mask corresponding to the number of pixels and the second pixel coordinates of the second data image mask corresponding to the number of pixels are determined; based on the number of pixels, the first pixel coordinates, and the second pixel coordinates, the first target is determined. Where N is the number of pixels in the current input surface defect sample image, (x 11i -x 21i ) represents the pixel coordinates of the first data image mask, (y... 11i -y 21i ) represents the pixel coordinates of the second data image mask, and i represents the number of pixels.

[0050] The advantage of this setup is that by determining the pixel coordinates of the first data image mask and the pixel coordinates of the second data image mask, the first target can be determined based on the Euclidean distance, thereby improving the accuracy of model training.

[0051] In one specific embodiment, determining the second target based on the first sample image mask and the second sample image mask includes: determining the number of pixels in the surface defect sample image, and determining the third pixel coordinate of the first sample image mask corresponding to the number of pixels and the fourth pixel coordinate of the second sample image mask corresponding to the number of pixels; and determining the second target based on the number of pixels, the third pixel coordinate, and the fourth pixel coordinate.

[0052] Wherein, the coordinates of the third pixel are the coordinates of each pixel in the mask of the first sample image, and the coordinates of the fourth pixel are the coordinates of each pixel in the mask of the second sample image.

[0053] Specifically, the number of pixels in the surface defect sample image is determined, and the third pixel coordinate of the first sample image mask corresponding to the number of pixels and the fourth pixel coordinate of the second sample image mask corresponding to the number of pixels are determined; based on the number of pixels, the third pixel coordinate, and the fourth pixel coordinate, the second target is determined. Where N is the number of pixels in the current input surface defect sample image, (x 12i -x 22i (y) represents the pixel coordinates of the mask for the first sample image. 12i -y 22i ) represents the pixel coordinates of the mask for the second sample image, and i represents the number of pixels.

[0054] In one specific implementation, determining the loss function value based on the first objective and the second objective includes: determining the loss weight; and determining the loss function value based on the loss weight, the first objective, and the second objective.

[0055] The loss weights are pre-defined weights used to determine the loss function value based on the first and second objectives. The loss function value is a weighted sum of the first and second objectives. A larger weighting coefficient for the first objective indicates that the transfer induction relies more on the contributions of the segmentation model and the visual segmentation data image. This is necessary in the early stages of training, as it helps the binary classification segmentation model to more easily identify simple samples. However, in the later stages of training, the weight of the first objective needs to be appropriately reduced to allow the ground truth annotations to help identify real defective samples.

[0056] Specifically, determine the loss weights; based on the loss weights, the first objective, and the second objective, determine the loss function value, where loss = aloss. soft +(1-a)loss hard .

[0057] In one specific implementation, a surface defect detection model is trained based on surface defect sample images using optimization rules as indicators to determine a target defect detection model. This includes: obtaining the main architecture model of the surface defect detection model; determining a classification model based on the main architecture model, the classification output layer, and the network model; inputting surface defect sample images into the classification model to determine defect category labels; keeping the underlying parameters of the surface defect detection model unchanged, and inputting the surface defect sample images and defect category labels into the surface defect detection model for retraining to determine the target defect detection model.

[0058] Specifically, the main architecture model of the surface defect detection model is obtained and used as the basic architecture of the network model. The neural network ends of the main architecture model are modified, and a classification output layer is added to determine the classification model. The classification output layer can be a deep convolutional network structure. Surface defect sample images are input into the classification model to determine the defect category label. The underlying parameters of the surface defect detection model remain unchanged, and the original context understanding and semantic segmentation capabilities of the model are preserved. The surface defect sample images and defect category labels are input into the surface defect detection model for retraining to determine the target defect detection model.

[0059] The advantage of this setup is that it increases the efficiency and accuracy of model training, focuses on defect edge features, and improves the user experience.

[0060] Specifically, the process involves acquiring surface defect sample images and visual segmentation data images of industrial equipment; performing knowledge distillation on the segmentation model and binary classification segmentation model based on the surface defect sample images to obtain a surface defect detection model; inputting the surface defect sample images and visual segmentation data images into the segmentation model for knowledge distillation to determine a first sample image mask and a first data image mask, and then inputting the surface defect sample images and visual segmentation data images into the binary classification segmentation model for training to determine a second sample image mask and a second data image mask; determining a first target based on the first and second data image masks, and further determining the target based on the first and second sample image masks. The image mask determines the second target; the loss function value is determined based on the first and second targets; it is determined whether the loss function value is less than or equal to a preset function value; if the loss function value is greater than the preset function value, the process returns to the step of inputting the surface defect sample image and visual segmentation data image into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample image and visual segmentation data image into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; if the loss function value is less than or equal to the preset function value, the current binary classification segmentation model is determined to be the surface defect detection model. Based on surface defect sample images and visual segmentation data images, a surface defect detection model is trained using optimization rules as indicators to determine the target defect detection model, i.e., to obtain the main architecture model of the surface defect detection model. A classification model is determined based on the main architecture model, the classification output layer, and the network model. Surface defect sample images are input into the classification model to determine the defect category label. Keeping the underlying parameters of the surface defect detection model unchanged, the surface defect sample images and defect category labels are input into the surface defect detection model for retraining to determine the target defect detection model. Finally, defect test images are input into the pre-trained target defect detection model to determine the defect test results. The aim is to retrain and acquire defect edge features of industrial equipment, improve the accuracy of detection results, promptly identify potential hazards in equipment, enhance equipment operating safety, and simultaneously scan surface defects in industrial equipment to promptly identify potential hazards, facilitating local repairs and parts replacement, and extending the overall lifespan of the equipment.

[0061] The technical solution of this invention involves acquiring defect test images; inputting these images into a pre-trained target defect detection model to determine the defect test results; wherein the target defect detection model is a model constructed based on a segmentation model and a binary classification segmentation model, and trained using optimization rules as indicators. This technical solution aims to retrain and acquire defect edge features of industrial equipment, improve the effectiveness of small-sample detection, increase the accuracy of detection results, promptly identify potential equipment hazards, and enhance the safety of equipment operation.

[0062] Example 2

[0063] Figure 2 This is a schematic diagram of the structure of a defect detection device provided in Embodiment 2. Figure 2 As shown, the device includes: an image acquisition module 201 and a result determination module 202; wherein,

[0064] Image acquisition module 201 is used to acquire defect test images.

[0065] The result determination module 202 is used to input the defect test image into the pre-trained target defect detection model to determine the defect test result; wherein, the target defect detection model is a model built based on the segmentation model and the binary classification segmentation model and trained and determined using optimization rules as indicators.

[0066] Optionally, the device also includes a model training module for acquiring surface defect sample images and visual segmentation data images of industrial equipment; performing knowledge distillation on the segmentation model and binary classification segmentation model based on the surface defect sample images to obtain a surface defect detection model; and training the surface defect detection model based on the surface defect sample images and visual segmentation data images using optimization rules as indicators to determine the target defect detection model.

[0067] Optionally, the model training module performs knowledge distillation on the segmentation model and the binary classification segmentation model based on the surface defect sample images and visual segmentation data images to obtain a surface defect detection model. Specifically, it is used to: input the surface defect sample images and visual segmentation data images into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask; input the surface defect sample images and visual segmentation data images into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; determine the first target based on the first data image mask and the second data image mask; and determine the first target based on the first sample image mask and the second sample image mask. The mask determines the second target; the loss function value is determined based on the first and second targets; it is determined whether the loss function value is less than or equal to a preset function value; if the loss function value is greater than the preset function value, the process returns to the step of inputting the surface defect sample image and the visual segmentation data image into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample image and the visual segmentation data image into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; if the loss function value is less than or equal to the preset function value, the current binary classification segmentation model is determined to be the surface defect detection model.

[0068] Optionally, the model training module determines the first target based on the first data image mask and the second data image mask, specifically by: determining the number of pixels in the surface defect sample image, and determining the first pixel coordinates of the first data image mask corresponding to the number of pixels and the second pixel coordinates of the second data image mask corresponding to the number of pixels; and determining the first target based on the number of pixels, the first pixel coordinates, and the second pixel coordinates.

[0069] Optionally, the model training module determines the second target based on the first sample image mask and the second sample image mask. Specifically, it is used to: determine the number of pixels in the surface defect sample image, and determine the third pixel coordinate of the first sample image mask corresponding to the number of pixels and the fourth pixel coordinate of the second sample image mask corresponding to the number of pixels; and determine the second target based on the number of pixels, the third pixel coordinate, and the fourth pixel coordinate.

[0070] Optionally, the model training module determines the loss function value based on the first objective and the second objective, specifically for: determining the loss weights; and determining the loss function value based on the loss weights, the first objective, and the second objective.

[0071] Optionally, the model training module trains the surface defect detection model based on surface defect sample images and uses optimization rules as indicators to determine the target defect detection model. Specifically, it is used to: obtain the main architecture model of the surface defect detection model; determine the classification model based on the main architecture model, the classification output layer, and the network model; input the surface defect sample images into the classification model to determine the defect category labels; keep the underlying parameters of the surface defect detection model unchanged, and input the surface defect sample images and defect category labels into the surface defect detection model for retraining to determine the target defect detection model.

[0072] The defect detection device provided by the present invention can execute the defect detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0073] Example 3

[0074] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0075] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0076] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0077] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as defect detection methods.

[0078] In some embodiments, the defect detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the defect detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the defect detection method by any other suitable means (e.g., by means of firmware).

[0079] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0080] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0081] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0082] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0083] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0084] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0085] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0086] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A defect detection method, characterized in that, The method includes: Obtain defect test images; The defect test image is input into a pre-trained target defect detection model to determine the defect test result; wherein, the target defect detection model is a model built based on a segmentation model and a binary classification segmentation model and trained and determined using optimization rules as indicators; The method for training the target defect detection model includes: Acquire surface defect sample images and visual segmentation data images of industrial equipment; Based on the surface defect sample image and the visual segmentation data image, knowledge distillation is performed on the segmentation model and the binary classification segmentation model to obtain the surface defect detection model. Based on the surface defect sample images, the surface defect detection model is trained using the optimization rules as indicators to determine the target defect detection model; The step of performing knowledge distillation on the segmentation model and the binary classification segmentation model based on the surface defect sample image and the visual segmentation data image to obtain the surface defect detection model includes: The surface defect sample image and the visual segmentation data image are input into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask. The surface defect sample image and the visual segmentation data image are then input into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask. A first target is determined based on the first data image mask and the second data image mask, and a second target is determined based on the first sample image mask and the second sample image mask. Determine the loss function value based on the first objective and the second objective; Determine whether the loss function value is less than or equal to a preset function value; If the loss function value is greater than the preset function value, then return to the steps of inputting the surface defect sample image and the visual segmentation data image into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample image and the visual segmentation data image into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; If the loss function value is less than or equal to the preset function value, then the current binary classification segmentation model is determined to be the surface defect detection model; The step of training the surface defect detection model based on the surface defect sample image and using the optimization rule as an indicator to determine the target defect detection model includes: Obtain the main architecture model of the surface defect detection model; The main architecture model of the surface defect detection model is used as the basic architecture of the network model. The neural network ends of the main architecture model of the surface defect detection model are modified, and a classification output layer is added to determine the classification model. The surface defect sample image is input into the classification model to determine the defect category label; Keeping the underlying parameters, original contextual understanding, and semantic segmentation capabilities of the surface defect detection model unchanged, the surface defect sample images and defect category labels are input into the surface defect detection model for retraining to determine the target defect detection model.

2. The defect detection method according to claim 1, characterized in that, Determining the first target based on the first data image mask and the second data image mask includes: Determine the number of pixels in the surface defect sample image, and determine the first pixel coordinates of the first data image mask corresponding to the number of pixels and the second pixel coordinates of the second data image mask corresponding to the number of pixels; The first target is determined based on the number of pixels, the coordinates of the first pixel, and the coordinates of the second pixel.

3. The defect detection method according to claim 1, characterized in that, The step of determining the second target based on the first sample image mask and the second sample image mask includes: The number of pixels in the surface defect sample image is determined, and the third pixel coordinate of the first sample image mask corresponding to the number of pixels and the fourth pixel coordinate of the second sample image mask corresponding to the number of pixels are determined. The second target is determined based on the number of pixels, the coordinates of the third pixel, and the coordinates of the fourth pixel.

4. The defect detection method according to claim 1, characterized in that, Determining the loss function value based on the first objective and the second objective includes: Determine the loss weights; The loss function value is determined based on the loss weight, the first objective, and the second objective.

5. A defect detection device, characterized in that, The device includes: The image acquisition module is used to acquire defect test images; The result determination module is used to input the defect test image into a pre-trained target defect detection model to determine the defect test result; wherein, the target defect detection model is a model built based on a segmentation model and a binary classification segmentation model and trained and determined using optimization rules as indicators; The device further includes a model training module, used for: Acquire surface defect sample images and visual segmentation data images of industrial equipment; Based on the surface defect sample image and the visual segmentation data image, knowledge distillation is performed on the segmentation model and the binary classification segmentation model to obtain the surface defect detection model. Based on the surface defect sample images, the surface defect detection model is trained using the optimization rules as indicators to determine the target defect detection model; Specifically, the model training module is used for: The surface defect sample image and the visual segmentation data image are input into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask. The surface defect sample image and the visual segmentation data image are then input into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask. A first target is determined based on the first data image mask and the second data image mask, and a second target is determined based on the first sample image mask and the second sample image mask. Determine the loss function value based on the first objective and the second objective; Determine whether the loss function value is less than or equal to a preset function value; If the loss function value is greater than the preset function value, then return to the steps of inputting the surface defect sample image and the visual segmentation data image into the segmentation model for knowledge distillation to determine the first sample image mask and the first data image mask, and inputting the surface defect sample image and the visual segmentation data image into the binary classification segmentation model for training to determine the second sample image mask and the second data image mask; If the loss function value is less than or equal to the preset function value, then the current binary classification segmentation model is determined to be the surface defect detection model; The model training module is also specifically used for: Obtain the main architecture model of the surface defect detection model; The main architecture model of the surface defect detection model is used as the basic architecture of the network model. The neural network ends of the main architecture model of the surface defect detection model are modified, and a classification output layer is added to determine the classification model. The surface defect sample image is input into the classification model to determine the defect category label; Keeping the underlying parameters, original contextual understanding, and semantic segmentation capabilities of the surface defect detection model unchanged, the surface defect sample images and defect category labels are input into the surface defect detection model for retraining to determine the target defect detection model.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the defect detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the defect detection method according to any one of claims 1 to 4.

Images