Target object defect detection methods, devices, equipment and storage media
By constructing an attention encoder and replacing the residual block with a multi-head self-attention mechanism layer, the problems of insufficient computational performance and loss of small target information in the target defect detection network in the existing technology are solved, and higher detection accuracy and efficiency are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-06-15
- Publication Date
- 2026-05-26
AI Technical Summary
Existing target defect detection networks incur excessive memory and inference time overhead on machines with low computing performance, and the loss of small target information due to downsampling of the backbone network results in low detection accuracy.
An attention encoder is constructed, and a multi-head self-attention mechanism layer is used to replace the residual block. A parallel attention encoding network is constructed, and the feature representation capability is improved through feature fusion and enhancement processing to cover objects of all scales.
It improves the accuracy of defect detection for target objects, especially for small targets, and reduces detection speed and computational overhead.
Smart Images
Figure CN116664949B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of digital medical technology, and in particular to a method, apparatus, electronic device, and readable storage medium for detecting defects in a target object. Background Technology
[0002] With the development of artificial intelligence, vision-based industrial defect detection aims to discover visible defects in the appearance of various industrial products, such as fabrics, chips, pharmaceuticals, and even infrastructure materials. Although these defects are small, they can seriously impair the normal function of the products. For example, defects in the appearance of pharmaceuticals often indicate problems with the composition or content of the drug's ingredients.
[0003] In existing technologies, object detection networks are often used for object detection and defect inspection. For example, in industrial quality inspection, object detection networks are used to detect defects in drug images to determine whether there are defects in the drug's appearance. The feature encoder is a crucial component of the object detector, reprocessing and appropriately utilizing the important features extracted from the backbone network. The encoder's feature extraction capability directly affects the object detector's performance. While multiple-input multiple-output encoders have achieved great success, it's important to note that they make the network structure increasingly complex, resulting in excessive memory and inference time overhead, which is unacceptable on machines with low computational performance. Although some improvements have been proposed to simplify the network, such as YOLOF, YOLOF focuses too much on reducing network structure and ignores an important problem: the backbone network will lose information about small objects during multiple downsampling. This means that it is difficult to cover small objects when detecting drug defects, resulting in significant limitations and low accuracy in object defect detection. Summary of the Invention
[0004] This invention provides a method, apparatus, electronic device, and readable storage medium for detecting defects in target objects, the main purpose of which is to improve the accuracy of defect detection in target objects.
[0005] To achieve the above objectives, the present invention provides a method for detecting defects in a target object, comprising:
[0006] An attention encoder is constructed, and a target defect detection network is constructed based on the attention encoder. The target defect detection network is trained to obtain a target defect detection model.
[0007] Acquire an image of the appearance of the drug to be tested, and extract multi-level image features of the appearance image of the drug to be tested using the feature extraction layer in the target defect detection model;
[0008] The attention encoder in the target defect detection model is used to perform attention enhancement processing on the multi-level image features to obtain enhanced image features.
[0009] The enhanced image features are decoded using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
[0010] Optionally, constructing the attention encoder includes:
[0011] An improved residual block is obtained by replacing the convolutional layer in the pre-constructed residual block with a multi-head self-attention mechanism layer.
[0012] An attention encoder is constructed based on the improved residual block.
[0013] Optionally, constructing the attention encoder based on the improved residual block includes:
[0014] A convolutional layer is connected in series after the improved residual block, and the series network and the convolutional layer are connected in parallel to obtain a parallel attention coding network.
[0015] The attention encoder is obtained by concatenating a first preset number of convolutional layers and a second preset number of residual blocks after the parallel attention encoding network.
[0016] Optionally, the step of constructing a target defect detection network based on the attention encoder, and training the target defect detection network to obtain a target defect detection model includes:
[0017] Obtain a pre-built single-level feature detection network that includes a feature extraction network, an encoder, and a decoder;
[0018] The attention encoder is used to replace the encoder in the single-level feature detection network to obtain the replacement detection network;
[0019] The replacement detection network is trained using a pre-constructed set of business images to obtain a target defect detection model.
[0020] Optionally, the step of using the attention encoder in the target defect detection model to perform attention enhancement processing on the multi-level image features to obtain enhanced image features includes:
[0021] The parallel attention coding network in the attention encoder is used to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image.
[0022] The initial fused feature image is fused with channel features by using the convolutional layer after the parallel attention coding network to obtain a standard fused feature image.
[0023] The standard fused feature image is enhanced by using the residual block with a preset second number in the attention encoder to obtain enhanced image features.
[0024] Optionally, the step of using the parallel attention coding network in the attention encoder to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image includes:
[0025] The last level of image features in the multi-level image features is processed by the branch containing the improved residual block in the parallel attention coding network to obtain the first enhanced image, and the first enhanced image is then upsampled.
[0026] The penultimate level image features in the multi-level image features are processed using the branch of the parallel attention coding network that does not contain the improved residual block to obtain the second enhanced image;
[0027] The first enhanced image and the second enhanced image after upsampling are fused to obtain an initial fused feature image.
[0028] Optionally, the step of using the decoder in the target defect detection model to perform feature decoding on the enhanced image features to obtain the target defect detection result of the appearance image of the drug to be detected includes:
[0029] The enhanced image features are classified and regressed using the decoder in the target object defect detection model to obtain the detected target object and the corresponding defect detection result.
[0030] The detected target object and the corresponding defect detection result are determined as the target object defect detection result of the appearance image of the drug to be detected.
[0031] To address the above problems, the present invention also provides a target object defect detection device, the device comprising:
[0032] The defect detection model construction module is used to construct an attention encoder, construct a target object defect detection network based on the attention encoder, train the target object defect detection network, and obtain a target object defect detection model.
[0033] The feature enhancement module is used to acquire the appearance image of the drug to be detected, extract multi-level image features of the appearance image of the drug to be detected using the feature extraction layer in the target defect detection model, and perform attention enhancement processing on the multi-level image features using the attention encoder in the target defect detection model to obtain enhanced image features.
[0034] The target defect detection module is used to perform feature decoding on the enhanced image features using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
[0035] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:
[0036] Memory, storing at least one computer program; and
[0037] The processor executes the computer program stored in the memory to implement the target defect detection method described above.
[0038] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the target defect detection method described above.
[0039] This invention reconstructs the attention encoder through an attention mechanism. For multi-level image features extracted by the feature extraction layer, the attention encoder pays more attention to feature maps at different stages. It not only focuses more on local features, but also, due to the addition of the attention mechanism block, the global representation of features is richer, which is beneficial for covering objects at all scales and improving the accuracy of target defect detection. Therefore, the target defect detection method, device, electronic device, and computer-readable storage medium proposed in this invention can improve the accuracy of target defect detection. Attached Figure Description
[0040] Figure 1 This is a schematic flowchart of a target object defect detection method provided in an embodiment of the present invention;
[0041] Figure 2 This is a functional block diagram of a target object defect detection device provided in an embodiment of the present invention;
[0042] Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the target defect detection method according to an embodiment of the present invention.
[0043] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0044] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0045] This invention provides a method for detecting defects in a target object. The execution entity of the method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this invention: a server, a terminal, etc. In other words, the method for detecting defects in a target object can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0046] Reference Figure 1 The diagram shown is a schematic flowchart of a target object defect detection method according to an embodiment of the present invention. In this embodiment, the target object defect detection method includes the following steps S1-S4:
[0047] S1. Construct an attention encoder, build a target defect detection network based on the attention encoder, train the target defect detection network to obtain a target defect detection model.
[0048] In this embodiment of the invention, the attention encoder adopts a multi-input single-output structure, including: a parallel attention coding network, a convolutional layer, and a residual block. The parallel attention coding network consists of an improved residual block, a 1×1 convolutional layer connected in series, and then another 1×1 convolutional layer connected in parallel. The improved residual block uses an MHSA (Multi-Head Self-Attention) layer instead of a 3×3 convolution to improve the global representation of image features.
[0049] Specifically, the construction of the attention encoder includes:
[0050] An improved residual block is obtained by replacing the convolutional layer in the pre-constructed residual block with a multi-head self-attention mechanism layer.
[0051] An attention encoder is constructed based on the improved residual block.
[0052] Furthermore, the construction of the attention encoder based on the improved residual block includes:
[0053] A convolutional layer is connected in series after the improved residual block, and the series network and the convolutional layer are connected in parallel to obtain a parallel attention coding network.
[0054] The attention encoder is obtained by concatenating a first preset number of convolutional layers and a second preset number of residual blocks after the parallel attention encoding network.
[0055] In an optional embodiment of the present invention, for example, the residual block includes 1×1 convolution, 3×3 convolution and 1×1 convolution. An improved residual block is obtained by replacing the 3×3 convolution with an MHSA (Multi-Head Self-Attention) layer. At the same time, a 1×1 convolution is concatenated after the improved residual block. The concatenated network and the 1×1 convolution are connected in parallel to obtain a parallel attention encoding network. After the parallel attention encoding network, a 1×1 kernel convolution (used to reduce the feature channel) or a 3×3 kernel convolution with a stride of 2 is concatenated (used to fuse features). Finally, four consecutive residual blocks with extended convolutions with different expansion factors (used to cover object information at different scales) are concatenated to obtain the attention encoder.
[0056] In detail, the step of constructing a target defect detection network based on the attention encoder, and training the target defect detection network to obtain a target defect detection model includes:
[0057] Obtain a pre-built single-level feature detection network that includes a feature extraction network, an encoder, and a decoder;
[0058] The attention encoder is used to replace the encoder in the single-level feature detection network to obtain the replacement detection network;
[0059] The replacement detection network is trained using a pre-constructed set of business images to obtain a target defect detection model.
[0060] In an optional embodiment of the present invention, the single-level feature detection network refers to the YOLOF (You Only Look One-level Feature) network, which includes: a feature extraction network (Backbone), a dilated encoder, and a decoder. Because YOLOF focuses too much on reducing network structure and neglects an important issue: the Backbone network will lose information about small targets during multiple downsampling processes, by using an attention encoder to replace the encoder to construct a replacement detection network, image features can be better mined, improving the accuracy of target defect detection.
[0061] In an optional embodiment of the present invention, in the medical field, the business image set can be a set of labeled images of different types of drugs. By adding MHSA blocks, it is beneficial to cover all scale feature objects and better capture drug appearance defects.
[0062] S2. Obtain the appearance image of the drug to be tested, and use the feature extraction layer in the target defect detection model to extract multi-level image features of the appearance image of the drug to be tested.
[0063] In this embodiment of the invention, the image of the drug to be detected can be an image in the medical field to be used for appearance defect detection. The feature extraction layer can be a ResNet50 network, etc., used to extract multiple feature maps of different scales from the image to be detected, including C2, C3, C4, and C5.
[0064] S3. Use the attention encoder in the target defect detection model to perform attention enhancement processing on the multi-level image features to obtain enhanced image features.
[0065] Specifically, the step of using the attention encoder in the target defect detection model to perform attention enhancement processing on the multi-level image features to obtain enhanced image features includes:
[0066] The parallel attention coding network in the attention encoder is used to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image.
[0067] The initial fused feature image is fused with channel features by using the convolutional layer after the parallel attention coding network to obtain a standard fused feature image.
[0068] The standard fused feature image is enhanced by using the residual block with a preset second number in the attention encoder to obtain enhanced image features.
[0069] In an optional embodiment of the present invention, the input of the attention encoder comes from the last two levels of output image features C4 and C5 of the backbone. After passing through the parallel attention coding network, the output image features M4 and M5 are obtained. M5 is upsampled to be rescaled to the same scale as M4. Then, M5 and M4 are concatenated into the same feature map Ocat to obtain the initial fused feature image. A 1x1 kernel convolution is used to reduce the channels of Ocat, and a 3×3 kernel convolution with a stride of 2 is used to fuse the features and downsample Ocat to Ofuse, i.e., the standard fused feature image. Finally, four consecutive residual blocks are used to cover object information at different scales and output Oout, i.e., to enhance the image features.
[0070] Specifically, the step of using the parallel attention encoding network in the attention encoder to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image includes:
[0071] The last level of image features in the multi-level image features is processed by the branch containing the improved residual block in the parallel attention coding network to obtain the first enhanced image, and the first enhanced image is then upsampled.
[0072] The penultimate level image features in the multi-level image features are processed using the branch of the parallel attention coding network that does not contain the improved residual block to obtain the second enhanced image;
[0073] The first enhanced image and the second enhanced image after upsampling are fused to obtain an initial fused feature image.
[0074] In an optional embodiment of the present invention, the last-level image features (C5) of the parallel attention coding network containing improved residual blocks are processed to obtain a first enhanced image (M5). The penultimate-level image features (C4) of the parallel attention coding network without improved residual blocks are processed to obtain a second enhanced image (M4). M5 is upsampled to the same scale as M4. Then, M5 and M4 are concatenated into the same feature map Ocat to obtain an initial fused feature image.
[0075] In this embodiment of the invention, the last output of the Backbone network may contain a lot, but not all, of information. Through the feature fusion process of the attention encoder, C5 is enhanced by MHSA for global representation to obtain M5, while M4 is used to refine the features of M5 to enhance local details. Furthermore, unlike FPN, the attention encoder connects M5 and M4 with different numbers of channels; M4 has only half the number of channels as M5, but the size of M5 has been enlarged to the size of M4. Therefore, M5 can still receive more attention in subsequent encoding processes. Through the above processing, the attention encoder retains information about small objects by sacrificing less detection speed and general computation, and can better display the features of the attention encoder. M4 focuses more on local features, and due to the addition of the MHSA block, the global representation of M5 is richer, which is beneficial for covering objects of all scales. By fusing M4 and M5 into Ofuse, information about objects of different scales is concentrated in the same feature map, greatly improving the accuracy of detecting small targets such as drug appearance defects.
[0076] S4. Use the decoder in the target defect detection model to perform feature decoding on the enhanced image features to obtain the target defect detection result of the appearance image of the drug to be detected.
[0077] In detail, the step of using the decoder in the target defect detection model to perform feature decoding on the enhanced image features to obtain the target defect detection result of the appearance image of the drug to be detected includes:
[0078] The enhanced image features are classified and regressed using the decoder in the target object defect detection model to obtain the detected target object and the corresponding defect detection result.
[0079] The detected target object and the corresponding defect detection result are determined as the target object defect detection result of the appearance image of the drug to be detected.
[0080] In this embodiment of the invention, the decoder uses two parallel branches: one branch for regression localization to determine the detected target object, and the other branch for classification to determine the corresponding defect detection result. The regression can use a structure of 4 convolutions + BN + ReLU, and the classification can use a structure of 2 convolutions + BN + ReLU.
[0081] This invention reconstructs the attention encoder through an attention mechanism. For multi-level image features extracted by the feature extraction layer, the attention encoder pays more attention to feature maps at different stages. It not only focuses more on local features, but also, due to the addition of the attention mechanism block, the global representation of features is richer, which is beneficial for covering objects at all scales and improving the accuracy of target defect detection. Therefore, the target defect detection method proposed in this invention can improve the accuracy of target defect detection.
[0082] like Figure 2 The diagram shown is a functional block diagram of a target defect detection device provided in an embodiment of the present invention.
[0083] The target object defect detection device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the target object defect detection device 100 may include a defect detection model construction module 101, a feature enhancement module 102, and a target object defect detection module 103. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.
[0084] In this embodiment, the functions of each module / unit are as follows:
[0085] The defect detection model construction module 101 is used to construct an attention encoder, construct a target defect detection network based on the attention encoder, train the target defect detection network, and obtain a target defect detection model.
[0086] The feature enhancement module 102 is used to acquire the appearance image of the drug to be detected, extract multi-level image features of the appearance image of the drug to be detected using the feature extraction layer in the target defect detection model, and perform attention enhancement processing on the multi-level image features using the attention encoder in the target defect detection model to obtain enhanced image features.
[0087] The target defect detection module 103 is used to perform feature decoding on the enhanced image features using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
[0088] In detail, the specific implementation methods of each module of the target object defect detection device 100 are as follows:
[0089] Step 1: Construct an attention encoder, build a target defect detection network based on the attention encoder, train the target defect detection network to obtain a target defect detection model.
[0090] In this embodiment of the invention, the attention encoder adopts a multi-input single-output structure, including: a parallel attention coding network, a convolutional layer, and a residual block. The parallel attention coding network consists of an improved residual block, a 1×1 convolutional layer connected in series, and then another 1×1 convolutional layer connected in parallel. The improved residual block uses an MHSA (Multi-Head Self-Attention) layer instead of a 3×3 convolution to improve the global representation of image features.
[0091] Specifically, the construction of the attention encoder includes:
[0092] An improved residual block is obtained by replacing the convolutional layer in the pre-constructed residual block with a multi-head self-attention mechanism layer.
[0093] An attention encoder is constructed based on the improved residual block.
[0094] Furthermore, the construction of the attention encoder based on the improved residual block includes:
[0095] A convolutional layer is connected in series after the improved residual block, and the series network and the convolutional layer are connected in parallel to obtain a parallel attention coding network.
[0096] The attention encoder is obtained by concatenating a first preset number of convolutional layers and a second preset number of residual blocks after the parallel attention encoding network.
[0097] In an optional embodiment of the present invention, for example, the residual block includes 1×1 convolution, 3×3 convolution and 1×1 convolution. An improved residual block is obtained by replacing the 3×3 convolution with an MHSA (Multi-Head Self-Attention) layer. At the same time, a 1×1 convolution is concatenated after the improved residual block. The concatenated network and the 1×1 convolution are connected in parallel to obtain a parallel attention encoding network. After the parallel attention encoding network, a 1×1 kernel convolution (used to reduce the feature channel) or a 3×3 kernel convolution with a stride of 2 is concatenated (used to fuse features). Finally, four consecutive residual blocks with extended convolutions with different expansion factors (used to cover object information at different scales) are concatenated to obtain the attention encoder.
[0098] In detail, the step of constructing a target defect detection network based on the attention encoder, and training the target defect detection network to obtain a target defect detection model includes:
[0099] Obtain a pre-built single-level feature detection network that includes a feature extraction network, an encoder, and a decoder;
[0100] The attention encoder is used to replace the encoder in the single-level feature detection network to obtain the replacement detection network;
[0101] The replacement detection network is trained using a pre-constructed set of business images to obtain a target defect detection model.
[0102] In an optional embodiment of the present invention, the single-level feature detection network refers to the YOLOF (You Only Look One-level Feature) network, which includes: a feature extraction network (Backbone), a dilated encoder, and a decoder. Because YOLOF focuses too much on reducing network structure and neglects an important issue: the Backbone network will lose information about small targets during multiple downsampling processes, by using an attention encoder to replace the encoder to construct a replacement detection network, image features can be better mined, improving the accuracy of target defect detection.
[0103] In an optional embodiment of the present invention, in the medical field, the business image set can be a set of labeled images of different types of drugs. By adding MHSA blocks, it is beneficial to cover all scale feature objects and better capture drug appearance defects.
[0104] Step 2: Obtain the appearance image of the drug to be tested, and use the feature extraction layer in the target defect detection model to extract multi-level image features of the appearance image of the drug to be tested.
[0105] In this embodiment of the invention, the image of the drug to be detected can be an image in the medical field to be used for appearance defect detection. The feature extraction layer can be a ResNet50 network, etc., used to extract multiple feature maps of different scales from the image to be detected, including C2, C3, C4, and C5.
[0106] Step 3: Use the attention encoder in the target defect detection model to perform attention enhancement processing on the multi-level image features to obtain enhanced image features.
[0107] Specifically, the step of using the attention encoder in the target defect detection model to perform attention enhancement processing on the multi-level image features to obtain enhanced image features includes:
[0108] The parallel attention coding network in the attention encoder is used to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image.
[0109] The initial fused feature image is fused with channel features by using the convolutional layer after the parallel attention coding network to obtain a standard fused feature image.
[0110] The standard fused feature image is enhanced by using the residual block with a preset second number in the attention encoder to obtain enhanced image features.
[0111] In an optional embodiment of the present invention, the input of the attention encoder comes from the last two levels of output image features C4 and C5 of the backbone. After passing through the parallel attention coding network, the output image features M4 and M5 are obtained. M5 is upsampled to be rescaled to the same scale as M4. Then, M5 and M4 are concatenated into the same feature map Ocat to obtain the initial fused feature image. A 1x1 kernel convolution is used to reduce the channels of Ocat, and a 3×3 kernel convolution with a stride of 2 is used to fuse the features and downsample Ocat to Ofuse, i.e., the standard fused feature image. Finally, four consecutive residual blocks are used to cover object information at different scales and output Oout, i.e., to enhance the image features.
[0112] Specifically, the step of using the parallel attention encoding network in the attention encoder to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image includes:
[0113] The last level of image features in the multi-level image features is processed by the branch containing the improved residual block in the parallel attention coding network to obtain the first enhanced image, and the first enhanced image is then upsampled.
[0114] The penultimate level image features in the multi-level image features are processed using the branch of the parallel attention coding network that does not contain the improved residual block to obtain the second enhanced image;
[0115] The first enhanced image and the second enhanced image after upsampling are fused to obtain an initial fused feature image.
[0116] In an optional embodiment of the present invention, the last-level image features (C5) of the parallel attention coding network containing improved residual blocks are processed to obtain a first enhanced image (M5). The penultimate-level image features (C4) of the parallel attention coding network without improved residual blocks are processed to obtain a second enhanced image (M4). M5 is upsampled to the same scale as M4. Then, M5 and M4 are concatenated into the same feature map Ocat to obtain an initial fused feature image.
[0117] In this embodiment of the invention, the last output of the Backbone network may contain a lot, but not all, of information. Through the feature fusion process of the attention encoder, C5 is enhanced by MHSA for global representation to obtain M5, while M4 is used to refine the features of M5 to enhance local details. Furthermore, unlike FPN, the attention encoder connects M5 and M4 with different numbers of channels; M4 has only half the number of channels as M5, but the size of M5 has been enlarged to the size of M4. Therefore, M5 can still receive more attention in subsequent encoding processes. Through the above processing, the attention encoder retains information about small objects by sacrificing less detection speed and general computation, and can better display the features of the attention encoder. M4 focuses more on local features, and due to the addition of the MHSA block, the global representation of M5 is richer, which is beneficial for covering objects of all scales. By fusing M4 and M5 into Ofuse, information about objects of different scales is concentrated in the same feature map, greatly improving the accuracy of detecting small targets such as drug appearance defects.
[0118] Step 4: Use the decoder in the target defect detection model to perform feature decoding on the enhanced image features to obtain the target defect detection result of the appearance image of the drug to be detected.
[0119] In detail, the step of using the decoder in the target defect detection model to perform feature decoding on the enhanced image features to obtain the target defect detection result of the appearance image of the drug to be detected includes:
[0120] The enhanced image features are classified and regressed using the decoder in the target object defect detection model to obtain the detected target object and the corresponding defect detection result.
[0121] The detected target object and the corresponding defect detection result are determined as the target object defect detection result of the appearance image of the drug to be detected.
[0122] In this embodiment of the invention, the decoder uses two parallel branches: one branch for regression localization to determine the detected target object, and the other branch for classification to determine the corresponding defect detection result. The regression can use a structure of 4 convolutions + BN + ReLU, and the classification can use a structure of 2 convolutions + BN + ReLU.
[0123] This invention reconstructs the attention encoder through an attention mechanism. For multi-level image features extracted by the feature extraction layer, the attention encoder pays more attention to feature maps at different stages. It not only focuses more on local features, but also, due to the addition of the attention mechanism block, the global representation of features is richer, which is beneficial for covering objects at all scales and improving the accuracy of target defect detection. Therefore, the target defect detection device proposed in this invention can improve the accuracy of target defect detection.
[0124] like Figure 3 The diagram shown is a schematic diagram of an electronic device for implementing the target defect detection method according to an embodiment of the present invention.
[0125] The electronic device may include a processor 10, a memory 11, a communication interface 12 and a bus 13, and may also include a computer program, such as a target defect detection program, stored in the memory 11 and capable of running on the processor 10.
[0126] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a target defect detection program, but also to temporarily store data that has been output or will be output.
[0127] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules (such as target defect detection programs) stored in the memory 11, and calls data stored in the memory 11 to perform various functions of the electronic device and process data.
[0128] The communication interface 12 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.
[0129] The bus 13 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 13 can be divided into an address bus, a data bus, a control bus, etc. The bus 13 is configured to enable communication between the memory 11 and at least one processor 10, etc.
[0130] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0131] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.
[0132] Furthermore, the electronic device may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device and other electronic devices.
[0133] Optionally, the electronic device may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.
[0134] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0135] The target defect detection program stored in the memory 11 of the electronic device is a combination of multiple instructions, which, when run in the processor 10, can achieve the following:
[0136] An attention encoder is constructed, and a target defect detection network is constructed based on the attention encoder. The target defect detection network is trained to obtain a target defect detection model.
[0137] Acquire an image of the appearance of the drug to be tested, and extract multi-level image features of the appearance image of the drug to be tested using the feature extraction layer in the target defect detection model;
[0138] The attention encoder in the target defect detection model is used to perform attention enhancement processing on the multi-level image features to obtain enhanced image features.
[0139] The enhanced image features are decoded using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
[0140] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.
[0141] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0142] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:
[0143] An attention encoder is constructed, and a target defect detection network is constructed based on the attention encoder. The target defect detection network is trained to obtain a target defect detection model.
[0144] Acquire an image of the appearance of the drug to be tested, and extract multi-level image features of the appearance image of the drug to be tested using the feature extraction layer in the target defect detection model;
[0145] The attention encoder in the target defect detection model is used to perform attention enhancement processing on the multi-level image features to obtain enhanced image features.
[0146] The enhanced image features are decoded using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
[0147] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.
[0148] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0149] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0150] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0151] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.
[0152] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0153] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.
[0154] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.
[0155] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.
[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for detecting defects in a target object, characterized in that, The method includes: An improved residual block is obtained by replacing the convolutional layer in the pre-constructed residual block with a multi-head self-attention mechanism layer. A convolutional layer is then connected in parallel after the improved residual block. The convolutional network and the convolutional layer are then connected in parallel to obtain a parallel attention encoding network. A first preset number of convolutional layers and a second preset number of residual blocks are then connected in parallel after the parallel attention encoding network to obtain an attention encoder. A target defect detection network is constructed based on the attention encoder, and the target defect detection network is trained to obtain a target defect detection model. Acquire an image of the appearance of the drug to be tested, and use the feature extraction network in the target defect detection model to extract multi-level image features of the appearance image of the drug to be tested; The attention encoder in the target defect detection model is used to perform attention enhancement processing on the multi-level image features to obtain enhanced image features. The enhanced image features are decoded using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
2. The method for detecting defects in a target object as described in claim 1, characterized in that, The step of constructing a target defect detection network based on the attention encoder, and training the target defect detection network to obtain a target defect detection model includes: Obtain a pre-built single-level feature detection network that includes a feature extraction network, an encoder, and a decoder; The attention encoder is used to replace the encoder in the single-level feature detection network to obtain the replacement detection network; The replacement detection network is trained using a pre-constructed set of business images to obtain a target defect detection model.
3. The target object defect detection method as described in claim 1, characterized in that, The step of using the attention encoder in the target defect detection model to perform attention enhancement processing on the multi-level image features to obtain enhanced image features includes: The parallel attention coding network in the attention encoder is used to fuse the last two levels of image features in the multi-level image features to obtain an initial fused feature image. The initial fused feature image is fused with channel features by using the convolutional layer after the parallel attention coding network to obtain a standard fused feature image. The standard fused feature image is enhanced by using the residual block with a preset second number in the attention encoder to obtain enhanced image features.
4. The target object defect detection method as described in claim 3, characterized in that, The step of fusing the last two levels of image features in the multi-level image features using the parallel attention coding network in the attention encoder to obtain an initial fused feature image includes: The last level of image features in the multi-level image features is processed by the branch containing the improved residual block in the parallel attention coding network to obtain the first enhanced image, and the first enhanced image is then upsampled. The penultimate level image features in the multi-level image features are processed using the branch of the parallel attention coding network that does not contain the improved residual block to obtain the second enhanced image; The first enhanced image and the second enhanced image after upsampling are fused to obtain an initial fused feature image.
5. The method for detecting defects in a target object as described in claim 1, characterized in that, The step of using the decoder in the target defect detection model to perform feature decoding on the enhanced image features to obtain the target defect detection result of the appearance image of the drug to be detected includes: The enhanced image features are classified and regressed using the decoder in the target object defect detection model to obtain the detected target object and the corresponding defect detection result. The detected target object and the corresponding defect detection result are determined as the target object defect detection result of the appearance image of the drug to be detected.
6. A target object defect detection device, used to implement the target object defect detection method as described in any one of claims 1 to 5, characterized in that, The device includes: The defect detection model construction module is used to construct an attention encoder, construct a target object defect detection network based on the attention encoder, train the target object defect detection network, and obtain a target object defect detection model. The feature enhancement module is used to acquire the appearance image of the drug to be detected, extract multi-level image features of the appearance image of the drug to be detected using the feature extraction network in the target defect detection model, and perform attention enhancement processing on the multi-level image features using the attention encoder in the target defect detection model to obtain enhanced image features. The target defect detection module is used to perform feature decoding on the enhanced image features using the decoder in the target defect detection model to obtain the target defect detection result of the appearance image of the drug to be detected.
7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the target defect detection method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the target defect detection method as described in any one of claims 1 to 5.