Image defect intelligent detection method, device and equipment and storage medium
By using image enhancement and region detection model feature extraction and fusion layer processing, the problem of low efficiency in traditional image defect detection is solved, and efficient and accurate identification of medical image defects is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-06-06
- Publication Date
- 2026-04-17
AI Technical Summary
Traditional image defect detection methods are inefficient in the medical field, requiring a lot of computational processing and prone to misjudgment or missed diagnosis.
By combining image enhancement, slicing, convolutional feature extraction, attention mechanisms, and fusion layers, a trained region detection model is used for image feature extraction and defect identification, including the application of noise reduction, grayscale equalization, convolutional layers, attention mechanisms, and fusion layers.
It improves the efficiency and accuracy of image defect detection, reduces misjudgments, and enhances the efficiency of medical work.
Smart Images
Figure CN117011161B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to an intelligent method, apparatus, device, and computer-readable storage medium for detecting image defects. Background Technology
[0002] Image defect detection is a technique for detecting defects in images. Traditional medical image analysis requires manual analysis and judgment by doctors, which often takes a long time and may lead to misjudgments or missed diagnoses due to personal experience and perception. However, applying intelligent image defect detection methods to the medical field can quickly and accurately identify abnormalities in diseases through intelligent detection algorithms, thereby greatly improving doctors' work efficiency and reducing diagnostic errors.
[0003] Current image defect detection methods mainly employ histogram equalization to enhance the contrast between the background and the defective target. Based on the difference in contrast, image defects are detected. However, this method requires extensive computational processing of the image, which reduces the efficiency of image defect detection in fields such as medicine. Summary of the Invention
[0004] This invention provides an intelligent image defect detection method, apparatus, device, and storage medium, the main purpose of which is to improve the efficiency of intelligent image defect detection in fields such as medicine.
[0005] To achieve the above objectives, the present invention provides an intelligent image defect detection method, comprising:
[0006] Obtain an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing on the enhanced image to obtain a sliced image;
[0007] The slice image is subjected to first convolutional feature extraction using the convolutional layer in a pre-trained region detection model to obtain first image features. Based on the first image features, the first feature channel of the slice image is constructed.
[0008] The attention mechanism in the region detection model is used to calculate the channel weight of each channel in the first feature channel, so as to select the second feature channel of the slice image from the first feature channel, and according to the second feature channel, the convolutional layer is used to extract the second convolutional features of the slice image to obtain the second image features;
[0009] The first image features and the second image features are fused using the fusion layer in the region detection model to obtain fused image features. Based on the fused image features, the region defect probability of the slice image is calculated using the fully connected layer in the region detection model. Based on the region defect probability, the defect region of the slice image is determined.
[0010] Optionally, the step of enhancing the initial image to obtain an enhanced image includes:
[0011] The initial image is subjected to noise reduction processing to obtain a noise-reduced image;
[0012] Extract the pixel values from the denoised image, and perform grayscale equalization on the pixel values to obtain equalized pixel values;
[0013] The denoised image is updated by equalizing the pixel values to obtain an enhanced image.
[0014] Optionally, the step of slicing the enhanced image to obtain a sliced image includes:
[0015] Obtain the grayscale image corresponding to the enhanced image, and extract the grayscale values from the grayscale image; based on the grayscale values;
[0016] Construct a slice matrix corresponding to the grayscale value, locate the image center of the enhanced image, and construct a coordinate system corresponding to the enhanced image based on the image center;
[0017] The coordinate information of the grayscale value in the coordinate system is obtained, and the grayscale image is sliced by combining the coordinate information and the slicing matrix to obtain a sliced image.
[0018] Optionally, the step of using the convolutional layer in the trained region detection model to perform a first convolutional feature extraction on the slice image to obtain a first image feature includes:
[0019] Obtain the convolution kernel in the convolutional layer of the region detection model, detect the pixel value in the slice image, and extract features from the slice image based on the pixel value using the convolution kernel to obtain the first feature;
[0020] The convolution kernel is flipped to obtain a flipped convolution kernel, and the flipped convolution kernel is used to extract features from the slice image to obtain a second feature;
[0021] The first feature and the second feature are added sequentially to obtain the target feature, and the target feature is used as the first image feature of the slice image.
[0022] Optionally, the step of calculating the channel weight of each channel in the first feature channel using the attention mechanism in the trained region detection model includes:
[0023] The channel weight of each channel in the first feature channel can be calculated using the following formula:
[0024]
[0025] in, Let Z represent the channel weight of each channel in the first feature channel, Z represent the sum of the mapping values of the first feature channel, and i represent the mapping value corresponding to each channel. This represents the weight coefficient corresponding to each channel in the feature channel.
[0026] Optionally, the step of fusing the first image features and the second image features using the fusion layer in the trained region detection model to obtain fused image features includes:
[0027] The vectors corresponding to each feature in the first image feature and the second image feature are obtained respectively to obtain the first feature vector and the second feature vector;
[0028] Calculate the similarity between each feature vector in the first feature vector and the second feature vector, and merge the feature vectors in the first feature vector and the second feature vector according to the similarity to obtain a merged feature vector;
[0029] The fusion layer performs vector fusion on the merged feature vectors to obtain a fused feature vector, and the fused image features are obtained based on the fused feature vector.
[0030] Optionally, calculating the similarity between each feature vector in the first feature vector and the second feature vector includes:
[0031] The similarity between each feature vector in the first feature vector and the second feature vector can be calculated using the following formula:
[0032]
[0033] in, This represents the similarity between each feature vector in the first and second feature vectors. This represents the eigenvalue of the d-th vector in the first eigenvector. This represents the eigenvalue of the d-th vector in the second eigenvector. This represents the vector coordinates of the d-th vector in the first eigenvector. This represents the vector coordinates of the d-th vector in the second eigenvector.
[0034] To address the above problems, the present invention also provides an intelligent image defect detection device, the device comprising:
[0035] An image processing module is used to acquire an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing processing on the enhanced image to obtain a sliced image;
[0036] The feature channel construction module is used to perform a first convolution feature extraction on the slice image using the convolutional layer in the pre-trained region detection model to obtain a first image feature, and to construct a first feature channel of the slice image based on the first image feature.
[0037] The feature extraction module is used to calculate the channel weight of each channel in the first feature channel using the attention mechanism in the region detection model, so as to select the second feature channel of the slice image from the first feature channel, and to perform second convolution feature extraction on the slice image using the convolutional layer based on the second feature channel to obtain the second image features;
[0038] The defect detection module is used to fuse the first image features and the second image features using the fusion layer in the region detection model to obtain fused image features, and to calculate the region defect probability of the slice image using the fully connected layer in the region detection model based on the fused image features, and to determine the defect region of the slice image based on the region defect probability.
[0039] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:
[0040] At least one processor; and,
[0041] A memory communicatively connected to the at least one processor; wherein,
[0042] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the image defect intelligent detection method described above.
[0043] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described intelligent image defect detection method.
[0044] This invention acquires an initial image to be detected, enhances the initial image to obtain an enhanced image, and can enhance and repair unclear or dark areas in the initial image. By processing the enhanced image, the difficulty of image processing is reduced. This invention uses the convolutional layer in a trained region detection model to perform first convolutional feature extraction on the slice image to obtain first image features. This can extract the feature parts in the slice image, so as to facilitate subsequent processing of the slice image in combination with the first convolutional features. In this invention, the attention mechanism in the trained region detection model is used to calculate the channel weight of each channel in the first feature channel. The channel weight can be used to understand the proportion of each feature channel in the whole, and the feature channels can be reconstructed according to the channel weight. In addition, this invention uses the fusion layer in the trained region detection model to fuse the first image features and the second image features to obtain fused image features. The fusion of the first image features and the second image features can obtain the comprehensive features of the slice image, which provides a guarantee for subsequent regional defect identification. Therefore, the intelligent image defect detection method, apparatus, device and storage medium provided by the embodiments of the present invention are applied in the medical field and can help medical staff improve the efficiency of intelligent image defect detection, thereby improving work efficiency and the efficiency of medical work operation. Attached Figure Description
[0045] Figure 1 This is a flowchart illustrating an intelligent image defect detection method according to an embodiment of the present invention.
[0046] Figure 2 This is a functional block diagram of an image defect intelligent detection device provided in an embodiment of the present invention;
[0047] Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the intelligent image defect detection method according to an embodiment of the present invention.
[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0049] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0050] This application provides an intelligent image defect detection method. In this application, the executing entity of the intelligent image defect detection method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the intelligent image defect detection method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0051] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent image defect detection method according to an embodiment of the present invention. In this embodiment, the intelligent image defect detection method includes steps S1-S5:
[0052] S1. Obtain the initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing processing on the enhanced image to obtain a sliced image. This invention, by obtaining the initial image to be detected and performing image enhancement on the initial image to obtain an enhanced image, can enhance and repair unclear or dark areas in the initial image. By processing the enhanced image, the difficulty of image processing is reduced.
[0053] The initial image to be detected can be a medical image. The enhanced image is the image obtained after the initial image has undergone enhancement processing such as sharpness and color gamut distortion. For example, due to equipment limitations, some parts of a medical image may be unclear or insufficiently bright.
[0054] As an embodiment of the present invention, the step of enhancing the initial image to obtain an enhanced image includes: performing noise reduction processing on the initial image to obtain a noise-reduced image, extracting pixel values from the noise-reduced image, performing grayscale equalization processing on the pixel values to obtain equalized pixel values, and updating the noise-reduced image using the equalized pixel values to obtain the enhanced image.
[0055] The denoised image is the image obtained after suppressing the noise in the initial image, which can avoid interfering with the main objects in the initial image. The pixel value is the basic component of the image, representing the brightness information of the image. The equalized pixel value is the pixel value obtained after equalization processing, so that the numerical difference of individual pixel values is not large, which can better display image information.
[0056] Furthermore, the initial image can be denoised using a filter, and the pixel values in the denoised image can be extracted using a pixel extractor. The pixel values can be grayscale balanced using a cumulative distribution function.
[0057] This invention obtains sliced images by slicing the enhanced image, which divides the enhanced image into multiple images to facilitate subsequent feature extraction. The extracted image features can be maximized. For example, some medical images have too many similar regions, and some features are easily missed during feature extraction. Moreover, feature similarity increases the difficulty of computer processing and reduces efficiency. The sliced images are obtained by dividing the enhanced image according to grayscale values.
[0058] As an embodiment of the present invention, the step of slicing the enhanced image to obtain a sliced image includes: obtaining a grayscale image corresponding to the enhanced image and extracting grayscale values from the grayscale image; constructing a slice matrix corresponding to the grayscale values based on the grayscale values; locating the image center of the enhanced image and constructing a coordinate system corresponding to the enhanced image based on the image center; obtaining the coordinate information of the grayscale values in the coordinate system; and combining the coordinate information and the slice matrix to slice the grayscale image to obtain a sliced image.
[0059] Wherein, the grayscale image is the grayscale digital image corresponding to the enhanced image, that is, an image where each pixel has only one sampled color, the grayscale value is the value corresponding to each pixel in the grayscale image, and the slicing matrix is a matrix constructed by selecting multiple grayscale values. Further, the grayscale image corresponding to the enhanced image can be calculated by a floating-point algorithm, the grayscale values in the grayscale image can be extracted by the electromagnetic wave spectrum, the slicing matrix corresponding to the grayscale values can be constructed by a matrix function, the matrix function includes a matrix exponential function, the coordinate system corresponding to the enhanced image can be constructed by a coordinate system tool, the coordinate system tool is compiled by a scripting language, and the grayscale image can be sliced by a slicing tool.
[0060] S2. The first convolutional feature extraction is performed on the slice image using the convolutional layer in the pre-trained region detection model to obtain the first image feature. Based on the first image feature, the first feature channel of the slice image is constructed.
[0061] This invention extracts first image features from the slice image by using the convolutional layer in a trained region detection model. This allows for the extraction of feature parts from the slice image, which can then be combined with the first convolutional features for subsequent processing of the slice image. The first image features are images with important representations in the slice image, such as pathological features in medicine or texture features corresponding to pathological conditions. The severity of the pathological condition can be indicated by the density and color intensity of the pathological features.
[0062] As an embodiment of the present invention, the step of using the convolutional layer in the trained region detection model to perform a first convolutional feature extraction on the slice image to obtain a first image feature includes: obtaining the convolutional kernel in the convolutional layer of the region detection model, detecting the pixel values in the slice image, performing feature extraction on the slice image using the convolutional kernel based on the pixel values to obtain a first feature, flipping the convolutional kernel to obtain a flipped convolutional kernel, and using the flipped convolutional kernel to perform feature extraction on the slice image to obtain a second feature, adding the first feature and the second feature sequentially to obtain a target feature, and using the target feature as the first image feature of the slice image.
[0063] The first feature is obtained by extracting features from the slice image using the convolution kernel. The flipped convolution kernel is obtained by rotating the convolution kernel by a certain angle, which can extract features from the edge region of the slice image. The second feature is obtained by extracting features from the edge region of the slice image using the flipped convolution kernel. The target feature is the feature obtained by merging the first feature and the second feature.
[0064] Furthermore, as an optional embodiment of the present invention, the pixel values in the sliced image can be obtained from the electromagnetic spectrum described above, the convolution kernel can be flipped using a filter, and the target feature is obtained by weighting the first feature and the second feature using a weighting function.
[0065] The present invention constructs a first feature channel of the sliced image based on the first image features, thereby understanding the number of feature channels and channel information corresponding to the first image features. The first feature channel is the channel formed by the corresponding features when extracting features.
[0066] As an embodiment of the present invention, constructing a first feature channel of the sliced image based on the first image features includes: obtaining the number of features of the first image features, constructing a feature channel corresponding to each feature in the number of features, extracting the channel attributes of each channel in the feature channels, and merging the feature channels according to the channel attributes to obtain a first feature channel.
[0067] Wherein, the number of features is the number of features corresponding to the first image features, the feature channel is the distribution data corresponding to each feature in the first image features, the channel attribute is the identification information corresponding to each channel in the feature channel, further, the feature channel corresponding to each feature in the number of features can be constructed by a filter, the channel attribute of each channel in the feature channel can be extracted by an attribute extraction tool, and the merging of the feature channels can be achieved by the add algorithm.
[0068] S3. Calculate the channel weight of each channel in the first feature channel using the attention mechanism in the region detection model, so as to select the second feature channel of the slice image from the first feature channel, and extract the second convolutional feature of the slice image using the convolutional layer according to the second feature channel to obtain the second image feature.
[0069] This invention calculates the channel weight of each channel in the first feature channel by utilizing the attention mechanism in a trained region detection model. The channel weight indicates the proportion of each feature channel in the overall picture. Based on the channel weight, the feature channels can be reconstructed. For example, some features in medical images are not obvious or are very detailed, so the size of the feature channels needs to be modified to facilitate the extraction of hidden or detailed features in medical images. The channel weight represents the importance of each channel in the first feature channel.
[0070] As an embodiment of the present invention, the step of calculating the channel weight of each channel in the first feature channel using the attention mechanism in the trained region detection model includes:
[0071] The channel weight of each channel in the first feature channel can be calculated using the following formula:
[0072]
[0073] in, Let Z represent the channel weight of each channel in the first feature channel, Z represent the sum of the mapping values of the first feature channel, and i represent the mapping value corresponding to each channel. This represents the weight coefficient corresponding to each channel in the feature channel.
[0074] This invention selects a second feature channel from the first feature channel of the sliced image, and performs second convolutional feature extraction on the sliced image using the convolutional layer based on the second feature channel to obtain second image features. It can combine the second feature channel to perform secondary feature extraction on the sliced image, thereby obtaining dual image features of the sliced image. This allows for the understanding of hidden features or more detailed features in the sliced image, such as texture features in medical images, including texture, curvature, and amplitude, resulting in more comprehensive feature extraction of the sliced image.
[0075] The second feature channel is obtained by adjusting the first feature channel according to the channel weights. The second image feature is extracted by the convolutional layer in combination with the second feature channel. Furthermore, the extraction principle of the second image feature is the same as that of the first image feature, which will not be elaborated on here.
[0076] S4. The first image features and the second image features are fused using the fusion layer in the region detection model to obtain fused image features. Based on the fused image features, the region defect probability of the slice image is calculated using the fully connected layer in the trained region detection model. Based on the region defect probability, the defect region of the slice image is determined.
[0077] This invention utilizes a fusion layer in the trained region detection model to fuse the first image features and the second image features to obtain fused image features. The fusion of the first image features and the second image features yields comprehensive features of the sliced image, providing assurance for subsequent region defect identification. The fused image features are obtained by fusing the first image features and the second image features. For example, combining some features from medical images can improve the accuracy of treatment and enable rapid completion of relevant treatments for patients. Similarly, combining brain CT image features and EEG features can quickly analyze relevant symptoms and corresponding pathologies of patients.
[0078] As an embodiment of the present invention, the step of fusing the first image features and the second image features using the fusion layer in the trained region detection model to obtain fused image features includes: obtaining the vector corresponding to each feature in the first image features and the second image features respectively to obtain a first feature vector and a second feature vector; calculating the similarity of each feature vector in the first feature vector and the second feature vector; merging the feature vectors in the first feature vector and the second feature vector according to the similarity to obtain a merged feature vector; performing vector fusion on the merged feature vector using the fusion layer to obtain a fused feature vector; and obtaining fused image features based on the fused feature vector.
[0079] Wherein, the first feature vector and the second feature vector are the vector representations of the first image feature and the second image feature, respectively; the similarity is the degree of similarity between the first feature vector and the second feature vector; the merged feature vector is obtained by merging the first feature vector and the second feature vector with high similarity; and the fused feature vector is the vector obtained by fusing the merged feature vectors together.
[0080] Furthermore, as an optional embodiment of the present invention, the first feature vector and the second feature vector can be obtained by the word2vec algorithm, the merging of the feature vectors in the first feature vector and the second feature vector can be achieved by adding the feature vectors, and the merged feature vectors can be fused by the fusion function in the fusion layer.
[0081] As an optional embodiment of the present invention, calculating the similarity between each feature vector in the first feature vector and the second feature vector includes:
[0082] The similarity between each feature vector in the first feature vector and the second feature vector can be calculated using the following formula:
[0083]
[0084] in, This represents the similarity between each feature vector in the first and second feature vectors. This represents the eigenvalue of the d-th vector in the first eigenvector. This represents the eigenvalue of the d-th vector in the second eigenvector. This represents the vector coordinates of the d-th vector in the first eigenvector. This represents the vector coordinates of the d-th vector in the second eigenvector.
[0085] This invention calculates the regional defect probability of the sliced image based on the fused image features using the fully connected layer in the trained region detection model. The defective regions in the sliced image can be determined by the regional defect probability, where the regional defect probability is the probability that a defective region exists in the sliced image. Furthermore, the regional defect probability of the sliced image can be calculated by a probability algorithm in the fully connected layer, such as a frequency algorithm.
[0086] This invention determines the defective region of the sliced image based on the regional defect probability. The defective region in the sliced image can be identified based on the regional defect probability, which improves the accuracy of defective region detection and helps the subsequent work of medical staff. The defective region is the region with defects in the initial image. Furthermore, the defective region can be determined by the highest probability value among the regional defect probabilities.
[0087] This invention acquires an initial image to be detected, enhances the initial image to obtain an enhanced image, and can enhance and repair unclear or dark areas in the initial image. Processing the enhanced image reduces the difficulty of image processing. This invention uses a convolutional layer in a trained region detection model to extract first convolutional features from the slice image, obtaining first image features. This extracts the feature components in the slice image, facilitating subsequent processing of the slice image in conjunction with the first convolutional features. Furthermore, this invention uses an attention mechanism in the trained region detection model to calculate the channel weight of each channel in the first feature channel. The channel weights reveal the proportion of each feature channel in the overall image, allowing for the reconstruction of the feature channels. In addition, this invention uses a fusion layer in the trained region detection model to fuse the first image features and the second image features to obtain fused image features. The fusion of the first and second image features yields the comprehensive features of the slice image, providing assurance for subsequent region defect identification. Therefore, the intelligent image defect detection method, device, equipment, and storage medium provided by this invention can improve the efficiency of intelligent image defect detection in fields such as medicine.
[0088] like Figure 2 The diagram shown is a functional block diagram of an intelligent image defect detection device provided in an embodiment of the present invention.
[0089] The intelligent image defect detection device 100 of the present invention can be installed in an electronic device. Depending on the functions implemented, the intelligent image defect detection device 100 may include an image processing module 101, a feature channel construction module 102, a feature extraction module 103, and a defect detection module 104. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.
[0090] In this embodiment, the functions of each module / unit are as follows:
[0091] The image processing module 101 is used to acquire an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing processing on the enhanced image to obtain a sliced image;
[0092] The feature channel construction module 102 is used to perform a first convolution feature extraction on the slice image using the convolutional layer in the pre-trained region detection model to obtain a first image feature, and to construct a first feature channel of the slice image based on the first image feature.
[0093] The feature extraction module 103 is used to calculate the channel weight of each channel in the first feature channel using the attention mechanism in the region detection model, so as to select the second feature channel of the slice image from the first feature channel, and perform second convolution feature extraction on the slice image using the convolution layer based on the second feature channel to obtain the second image feature;
[0094] The defect detection module 104 is used to fuse the first image features and the second image features using the fusion layer in the region detection model to obtain fused image features, and to calculate the region defect probability of the slice image using the fully connected layer in the region detection model based on the fused image features, and to determine the defect region of the slice image based on the region defect probability.
[0095] In detail, each module in the image defect intelligent detection device 100 described in this application embodiment adopts the same characteristics as described above during use. Figure 1 The method uses the same techniques as the intelligent image defect detection method described above and can produce the same technical effects, so it will not be repeated here.
[0096] like Figure 3 The diagram shown is a structural schematic of an electronic device 1 for implementing an intelligent image defect detection method according to an embodiment of the present invention.
[0097] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as an image defect intelligent detection method program.
[0098] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device 1, connecting various components of the electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing an intelligent image defect detection method program) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.
[0099] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of an intelligent image defect detection method program, but also to temporarily store data that has been output or will be output.
[0100] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.
[0101] The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.
[0102] Figure 3 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0103] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.
[0104] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0105] The image defect intelligent detection method program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which, when run in the processor 10, can achieve the following:
[0106] Obtain an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing on the enhanced image to obtain a sliced image;
[0107] The slice image is subjected to first convolutional feature extraction using the convolutional layer in a pre-trained region detection model to obtain first image features. Based on the first image features, the first feature channel of the slice image is constructed.
[0108] The attention mechanism in the region detection model is used to calculate the channel weight of each channel in the first feature channel, so as to select the second feature channel of the slice image from the first feature channel, and according to the second feature channel, the convolutional layer is used to extract the second convolutional features of the slice image to obtain the second image features;
[0109] The first image features and the second image features are fused using the fusion layer in the region detection model to obtain fused image features. Based on the fused image features, the region defect probability of the slice image is calculated using the fully connected layer in the region detection model. Based on the region defect probability, the defect region of the slice image is determined.
[0110] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.
[0111] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0112] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:
[0113] Obtain an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing on the enhanced image to obtain a sliced image;
[0114] The slice image is subjected to first convolutional feature extraction using the convolutional layer in a pre-trained region detection model to obtain first image features. Based on the first image features, the first feature channel of the slice image is constructed.
[0115] The attention mechanism in the region detection model is used to calculate the channel weight of each channel in the first feature channel, so as to select the second feature channel of the slice image from the first feature channel, and according to the second feature channel, the convolutional layer is used to extract the second convolutional features of the slice image to obtain the second image features;
[0116] The first image features and the second image features are fused using the fusion layer in the region detection model to obtain fused image features. Based on the fused image features, the region defect probability of the slice image is calculated using the fully connected layer in the region detection model. Based on the region defect probability, the defect region of the slice image is determined.
[0117] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.
[0118] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0119] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0120] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0121] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.
[0122] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0123] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.
[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. An intelligent image defect detection method, characterized in that, The method includes: Obtain an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing on the enhanced image to obtain a sliced image; The slice image is subjected to first convolutional feature extraction using the convolutional layer in a pre-trained region detection model to obtain first image features. Based on the first image features, the first feature channel of the slice image is constructed. The attention mechanism in the region detection model is used to calculate the channel weight of each channel in the first feature channel, so as to select the second feature channel of the slice image from the first feature channel, and according to the second feature channel, the convolutional layer is used to extract the second convolutional features of the slice image to obtain the second image features; The first image features and the second image features are fused using the fusion layer in the region detection model to obtain fused image features. Based on the fused image features, the region defect probability of the slice image is calculated using the fully connected layer in the region detection model. Based on the region defect probability, the defect region of the slice image is determined.
2. The intelligent image defect detection method as described in claim 1, characterized in that, The step of enhancing the initial image to obtain an enhanced image includes: The initial image is subjected to noise reduction processing to obtain a noise-reduced image; Extract the pixel values from the denoised image, and perform grayscale equalization on the pixel values to obtain equalized pixel values; The denoised image is updated by equalizing the pixel values to obtain an enhanced image.
3. The intelligent image defect detection method as described in claim 1, characterized in that, The step of slicing the enhanced image to obtain a sliced image includes: Obtain the grayscale image corresponding to the enhanced image, and extract the grayscale values from the grayscale image; based on the grayscale values; Construct a slice matrix corresponding to the grayscale value, locate the image center of the enhanced image, and construct a coordinate system corresponding to the enhanced image based on the image center; The coordinate information of the grayscale value in the coordinate system is obtained, and the grayscale image is sliced by combining the coordinate information and the slicing matrix to obtain a sliced image.
4. The intelligent image defect detection method as described in claim 1, characterized in that, The first image features are obtained by using the convolutional layer in the trained region detection model to perform a first convolutional feature extraction on the slice image, including: Obtain the convolution kernel in the convolutional layer of the region detection model, detect the pixel value in the slice image, and extract features from the slice image based on the pixel value using the convolution kernel to obtain the first feature; The convolution kernel is flipped to obtain a flipped convolution kernel, and the flipped convolution kernel is used to extract features from the slice image to obtain a second feature; The first feature and the second feature are added sequentially to obtain the target feature, and the target feature is used as the first image feature of the slice image.
5. The intelligent image defect detection method as described in claim 1, characterized in that, The step of calculating the channel weight of each channel in the first feature channel using the attention mechanism in the trained region detection model includes: The channel weight of each channel in the first feature channel can be calculated using the following formula: in, Let Z represent the channel weight of each channel in the first feature channel, Z represent the sum of the mapping values of the first feature channel, and i represent the mapping value corresponding to each channel. This represents the weight coefficient corresponding to each channel in the feature channel.
6. The intelligent image defect detection method as described in claim 1, characterized in that, The step of fusing the first image features and the second image features using the fusion layer in the trained region detection model to obtain fused image features includes: The vectors corresponding to each feature in the first image feature and the second image feature are obtained respectively to obtain the first feature vector and the second feature vector; Calculate the similarity between each feature vector in the first feature vector and the second feature vector, and merge the feature vectors in the first feature vector and the second feature vector according to the similarity to obtain a merged feature vector; The fusion layer is used to perform vector fusion on the merged feature vector to obtain a fused feature vector, and the fused image features are obtained based on the fused feature vector.
7. The intelligent image defect detection method as described in claim 6, characterized in that, The calculation of the similarity between each feature vector in the first feature vector and the second feature vector includes: The similarity between each feature vector in the first feature vector and the second feature vector is calculated using the following formula: in, This represents the similarity between each feature vector in the first and second feature vectors. This represents the eigenvalue of the d-th vector in the first eigenvector. This represents the eigenvalue of the d-th vector in the second eigenvector. This represents the vector coordinates of the d-th vector in the first eigenvector. This represents the vector coordinates of the d-th vector in the second eigenvector.
8. An intelligent image defect detection device, characterized in that, The device includes: An image processing module is used to acquire an initial image to be detected, perform image enhancement on the initial image to obtain an enhanced image, and perform slicing processing on the enhanced image to obtain a sliced image; The feature channel construction module is used to perform a first convolution feature extraction on the slice image using the convolutional layer in the pre-trained region detection model to obtain a first image feature, and to construct a first feature channel of the slice image based on the first image feature. The feature extraction module is used to calculate the channel weight of each channel in the first feature channel using the attention mechanism in the region detection model, so as to select the second feature channel of the slice image from the first feature channel, and perform second convolution feature extraction on the slice image using the convolutional layer based on the second feature channel to obtain the second image features; The defect detection module is used to fuse the first image features and the second image features using the fusion layer in the region detection model to obtain fused image features, and to calculate the region defect probability of the slice image using the fully connected layer in the region detection model based on the fused image features, and to determine the defect region of the slice image based on the region defect probability.
9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intelligent image defect detection method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent image defect detection method as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Workpiece defect detection method and device fusing multi-attention mechanism
CN113822885A
Image-based defects identification and semi-supervised localization
US20210319546A1