Detection method and device

Abstract

The invention provides a detection method and device, and the method comprises the steps: obtaining a first feature map and at least one second feature map of a to-be-detected image, and enabling the position structure information of the first feature map to be more than the position structure information of the second feature map; obtaining spatial position compensation information corresponding to the at least one second feature map based on the first feature map, and performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map; performing enhanced boundary processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps; and determining a detection result of the to-be-detected image based on the at least two position information enhanced feature maps.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a detection method and device. Background Art

[0002] Computer-aided surface defect detection is extremely challenging due to its small size and complex background. Related technologies can introduce redundant information during the calculation process, increasing computational complexity. Furthermore, small target features are easily lost in high-level feature maps, affecting detection accuracy and making them susceptible to noise interference. Therefore, a more efficient and accurate small target detection method is urgently needed to address the limitations of existing technologies. Summary of the Invention

[0003] The present disclosure provides a detection method and device to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a detection method is provided, comprising:

[0005] Obtaining a first feature map and at least one second feature map of an image to be detected, wherein the first feature map has more positional structure information than the second feature map;

[0006] Obtaining spatial position compensation information corresponding to each of the at least one second feature map based on the first feature map, and performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map;

[0007] Performing boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps;

[0008] A detection result of the image to be detected is determined based on the at least two position information enhanced feature maps.

[0009] In the above solution, obtaining the spatial position compensation information corresponding to the at least one second feature map based on the first feature map includes:

[0010] Inputting the first feature map into a weight generation network, and determining a ratio of propagation of the first feature map to the at least one second feature map;

[0011] Inputting the first feature map into a feature size matching network to adjust the size of the first feature map so that the adjusted first feature map is adapted to the at least one second feature map;

[0012] Based on the ratio of propagation of the first feature map to the at least one second feature map and the adjusted first feature map, spatial position compensation information corresponding to the at least one second feature map is determined.

[0013] In the above solution, inputting the first feature map into the weight generation network and determining the ratio of propagation of the first feature map to the at least one second feature map includes:

[0014] Performing convolution processing on the first feature map to adjust the number of channels of the first feature map;

[0015] Input the first feature map into a global average pooling layer, a first fully connected layer, and a second fully connected layer, and determine an output of the second fully connected layer, which is, in order, a ratio of propagation of the first feature map to the at least one second feature map;

[0016] The number of neurons in the second fully connected layer is the same as the number of the at least one second feature map.

[0017] In the above solution, determining the spatial position compensation information corresponding to the at least one second feature map based on the ratio of propagation of the first feature map to the at least one second feature map and the adjusted first feature map includes performing the following operations on the fourth feature map in the at least one second feature map:

[0018] The ratio of the first feature image propagating to the fourth feature map is calculated based on scalar multiplication, and the product of the first feature map adjusted to fit the size of the fourth feature map is determined, and the result of the scalar multiplication is determined to be the spatial position compensation information corresponding to the fourth feature map.

[0019] In the above solution, performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information includes:

[0020] Performing convolution processing on the at least two second feature maps respectively to obtain at least two initial feature maps; each initial feature map has the same number of channels; determining, among the at least two initial feature maps, the initial feature map with more positional structure information as the first initial feature map, and the initial feature map with less positional structure information as the second initial feature map;

[0021] Upsampling the second initial feature map, and adding the upsampling result to the first initial feature map to obtain a content enhancement feature map;

[0022] The content enhancement feature map and the corresponding spatial position compensation information are added together, and the addition result is the spatial compensation result.

[0023] In the above solution, the step of performing boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps includes performing the following operations on any one of the first feature map and the at least one third feature map:

[0024] Performing convolution processing on any of the feature maps to obtain a first sub-feature map of any of the feature maps;

[0025] Performing convolution processing on the high-frequency component of the spatial domain of any one of the feature maps to obtain a second sub-feature map of any one of the feature maps;

[0026] Determine the sum of the first sub-feature map and the second sub-feature map as the position information enhanced feature map corresponding to any one of the feature maps.

[0027] In the above solution, performing convolution processing on the high-frequency component of the spatial domain of any feature map to obtain a second sub-feature map of any feature map includes:

[0028] Performing a two-dimensional discrete Fourier transform on any of the feature maps to obtain a high-frequency component;

[0029] Perform a two-dimensional inverse discrete Fourier transform on the high-frequency component to obtain a second sub-feature map of any one of the feature maps.

[0030] In the above solution, determining the detection result of the image to be detected based on the at least two position information enhanced feature maps includes:

[0031] Inputting the at least two position information enhanced feature maps into a candidate region extraction network and outputting a marking feature map;

[0032] Inputting the identification feature map and the at least two position information enhanced feature maps into a region of interest pooling network, and outputting a detection result of the image to be detected;

[0033] The detection result includes the presence or absence of a defect.

[0034] In the above solution, the identification feature map includes an anchor box;

[0035] In response to the detection result indicating that a defect exists, the region of interest pooling network outputs the image to be detected with an anchor frame identification; the anchor frame is used to identify the defect in the image to be detected.

[0036] According to a second aspect of the present disclosure, a detection device is provided, comprising:

[0037] A residual network module, configured to obtain a first feature map and at least one second feature map of the image to be detected, wherein the first feature map has more positional structural information than the second feature map;

[0038] a spatial compensation module, configured to obtain spatial position compensation information corresponding to the at least one second feature map based on the first feature map, and perform spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map;

[0039] a boundary enhancement module, configured to perform boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps;

[0040] A detection module is used to determine a detection result of the image to be detected based on the at least two position information enhanced feature maps.

[0041] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:

[0043] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0044] Figure 1 A first optional flow chart of the detection method provided by an embodiment of the present disclosure is shown;

[0045] Figure 2 A second optional flow chart of the detection method provided in an embodiment of the present disclosure is shown;

[0046] Figure 3 A schematic structural diagram of a spatial position compensation module provided by an embodiment of the present disclosure is shown;

[0047] Figure 4 A third optional flow chart of the detection method provided in an embodiment of the present disclosure is shown;

[0048] Figure 5 A schematic structural diagram of a boundary reinforcement module provided by an embodiment of the present disclosure is shown;

[0049] Figure 6 A fourth optional flow chart of the detection method provided in an embodiment of the present disclosure is shown;

[0050] Figure 7 A schematic diagram of the structure of a faster regional convolutional neural network provided by an embodiment of the present disclosure is shown;

[0051] Figure 8 A fifth optional flow chart of the detection method provided in an embodiment of the present disclosure is shown;

[0052] Figure 9A schematic diagram of the structure of the enhanced feature pyramid network provided by an embodiment of the present disclosure is shown;

[0053] Figure 10 An optional structural diagram of a detection device provided in an embodiment of the present disclosure is shown;

[0054] Figure 11 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] To make the purposes, features, and advantages of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative work shall fall within the scope of protection of the present disclosure.

[0056] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0057] In the following description, the terms "first\second" are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0058] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by those skilled in the art in the art of this disclosure. The terms used in this disclosure are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.

[0059] It should be understood that in the various embodiments of the present disclosure, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0060] Due to the small size of computer surface defects and the complex background, their detection is an extremely challenging task. Traditional machine learning algorithms suffer from slow recognition and low accuracy. Currently, common approaches are based on deep learning, such as Faster R-CNN. For small object detection, the main research direction is to obtain multi-scale features in images. A typical approach is to fuse multi-scale information through Feature Pyramid Networks (FPN). Embedding FPN into detection algorithms such as Faster R-CNN can achieve better detection results.

[0061] When performing convolution on an image, shallow features typically contain more positional structural information but lack semantic information. Deep features, on the other hand, contain more semantic information but lose positional structural information. In FPN, deep features are upsampled and then fused with features from the previous layer. This allows the fused feature map to contain both deep semantic information and shallow positional information. Basic FPN allows shallow layers to contain deeper semantic information, but the deep feature maps do not receive the added positional information.

[0062] In addition, considering that the boundary information of small targets is easily ignored when extracting features, but the boundary information of the detected object is particularly important for classification and positioning, strengthening the feature extraction of boundary information can effectively improve the accuracy of target detection.

[0063] In response to the problems existing in the related technology, the embodiments of the present disclosure propose a detection method based on the Position-aware Enhanced Feature Pyramid Network (PAE-FPN). Based on FPN, the embodiments of the present disclosure compensate for the position structure information of the deep feature map through the Space Position Compensation (SPC) module, and enhance the expression of boundary information in the feature map through the Boundary Reinforcement (BR) module, thereby solving some or all of the above-mentioned technical problems.

[0064] The detection method provided in the embodiments of the present disclosure can be applied to surface defect detection of electronic devices such as computers, tablet computers, and mobile phones. Its usage scenarios include detecting whether there are scratches, cracks, and other defects on the surface of electronic devices when recycling electronic devices to assist in pricing recycled computers. It can also be used to detect newly produced electronic devices to prevent defective electronic devices from causing a bad user experience for consumers.

[0065] Figure 1A first optional flow chart of the detection method provided in an embodiment of the present disclosure is shown, and will be explained according to each step.

[0066] Step S101: Obtain a first feature map and at least one second feature map of an image to be detected.

[0067] In some embodiments, the carrier implementing the detection method (hereinafter referred to as the carrier) extracts features of the image to be detected and obtains multiple feature maps of different hierarchical structures. The multiple feature maps can be obtained according to different network layers, and are divided into shallow feature maps and deep feature maps according to the distance between the network layer and the input layer, or the distance between the network layer and the output layer. Among them, the shallow feature map refers to the feature map extracted by the network layer close to the input layer (such as the first few layers). These feature maps usually contain more detailed information, such as low-level features such as edges, textures, and colors. Because the receptive field of shallow features is small, they are sensitive to local details of the target and are suitable for capturing the fine structure of small targets, but are also easily interfered by background noise. Deep feature maps refer to feature maps extracted by network layers close to the output layer (such as the last few layers). These feature maps usually contain higher-level semantic information, such as the overall shape and category information of the target. The receptive field of deep features is large and can capture the global information of the target, but more detailed information of small targets may be lost.

[0068] In some embodiments, the first feature map is a shallow feature map that includes more positional structural information; the second feature map is a deep feature map relative to the first feature map that includes more semantic information. The positional structural information includes information related to the spatial position, shape, boundary, outline, and other geometric structures of the target (such as a defect) in the image. Specifically, the position information includes: the specific position of the target in the image (such as coordinates, relative position, etc.); the structural information includes geometric features such as the shape, edge, and outline of the target.

[0069] In some embodiments, the first feature map corresponds to at least one feature map output by the network layer closest to the input layer; the second feature map corresponds to multiple feature maps output by multiple network layers other than the network layer closest to the input layer; that is, each network layer outputs multiple feature maps. The number of the second feature maps is greater than or equal to 2. In the disclosed embodiments, "more" means greater than or equal to 2.

[0070] In some embodiments, the carrier can be a computer program, electronic circuit, database, mobile application, electronic device, cloud computing platform, distributed system, artificial intelligence framework, mathematical model, automation tool and microcontroller, etc., which can implement software or hardware of algorithm and method process.

[0071] Step S102: obtaining spatial position compensation information corresponding to the at least one second feature map based on the first feature map, and performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map.

[0072] In some embodiments, the first feature map is a shallow feature map, which includes more position structure information than the at least one second feature map. The carrier obtains the spatial position compensation information corresponding to each second feature map in the at least one second feature map based on the first feature map, and performs spatial compensation on the corresponding second feature map based on the spatial position compensation information corresponding to each feature map, and determines that the feature map obtained after compensation is the third feature map.

[0073] In some optional embodiments, since the number of channels of the first feature map and the at least one second feature map is different, in order to facilitate subsequent calculations, before step S102, the carrier can unify the number of channels of the first feature map and the at least one second feature map through a convolution operation.

[0074] Step S103: performing boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps.

[0075] In some embodiments, since boundary information is easily ignored when extracting small target features in related technologies, in order to enhance the boundary information so as to accurately locate the target position, the carrier performs enhanced boundary processing on the first feature map and at least one third feature map to obtain at least two position information enhanced feature maps.

[0076] In some optional embodiments, the carrier can perform enhanced boundary processing on the first feature map and at least one third feature map based on edge detection operators, edge extraction (such as extracting boundaries through the difference of image dilation and erosion operations), skeletonization (refining the image to its stock price to highlight the boundary structure), Gaussian-Laplacian algorithm, calculating the complex amplitude and direction of the gradient of the image, highlighting the intensity change of the boundary area, attention mechanism, boundary detection head and other methods.

[0077] Step S104: determining a detection result of the image to be detected based on the at least two position information enhanced feature maps.

[0078] In some embodiments, the carrier inputs the at least two position information enhanced feature maps into a detection module, and the detection module outputs a detection result, such as the presence of a defect or the absence of a defect; if the detection result output by the detection module is that a defect exists, it can also output an image to be detected with an anchor frame mark; the anchor frame is used to identify defects in the image to be detected.

[0079] In this way, through the detection method provided by the embodiment of the present disclosure, the deep feature map is compensated by the spatial position information in the shallow feature map, and at the same time, the boundary information of each feature map is strengthened, thereby improving the feature extraction capability of the network, and then performing image detection based on the feature map that compensates for the spatial position information and strengthens the boundary information, thereby improving the accuracy of surface defect detection of electronic equipment.

[0080] Figure 2 A second optional flow chart of the detection method provided in an embodiment of the present disclosure is shown, and will be explained according to each step.

[0081] Step S201: Obtain a first feature map and at least one second feature map of an image to be detected.

[0082] In some embodiments, the carrier can obtain a multi-scale feature map of the image to be detected from shallow to deep based on the ResNet50 network, wherein the shallowest feature map is the first feature map C2, and the other feature maps from shallow to deep are at least one second feature map, denoted as C3, C4, and C5. Among them, the first feature map C2 includes multiple feature maps output by shallow network layers; the second feature maps C3 to C5 respectively include multiple feature maps output by relatively deep network layers, that is, the second feature map C3 includes multiple feature maps output by the same network layer, the second feature map C4 includes multiple feature maps output by the same network layer, and the second feature map C5 includes multiple feature maps output by the same network layer.

[0083] Among them, the first feature map C2 includes more positional structural information than at least one second feature map C3~C5. As the network deepens (or the network layer deepens), the semantic information in C3, C4, and C5 will gradually become prominent, that is, the second feature map C5 includes more semantic information than C2~C4.

[0084] In some embodiments, the number of channels of each feature map in the first feature map and the at least one second feature map is different, and the carrier can perform convolution processing on the first feature map and the at least one second feature map respectively so that the number of channels of all feature maps is the same.

[0085] The convolution process may be a 1×1 convolution process. After the process, the first feature map C2 is identified as the first feature map F2, and the at least one second feature map C3 to C5 is identified as the at least one second feature map F3 to F5. It should be noted that in the disclosed embodiment, the feature map Ci is the output of the i-th network layer in ResNet50; the feature map Fi is a feature map with a uniform number of channels obtained by performing a 1×1 convolution on Ci, where i is [2, 5].

[0086] In some embodiments, the carrier may implement steps S202 to S203 based on a space position compensation (SPC) module.

[0087] Figure 3 A schematic structural diagram of a spatial position compensation module provided in an embodiment of the present disclosure is shown.

[0088] In some embodiments, as Figure 3 As shown, the carrier determines the proportion of propagation of the first feature map to the at least one second feature map based on the weight generation network (WGN) included in the spatial position compensation module, and adjusts the size (or size) of the first feature map based on the feature size adaptation (FSA) network included in the spatial position compensation module to adapt to the at least one second feature map respectively.

[0089] Step S202: Determine a propagation ratio of the first feature map to at least one second feature map.

[0090] In some embodiments, the weight generation network includes a global average pooling layer (GAP) and two fully connected layers (FC). Specifically, the first fully connected layer ( Figure 3 128d FC), and the second fully connected layer ( Figure 3 3d FC in the 3d); the number of neurons in the second fully connected layer is the same as the number of the at least one feature map; in the embodiment of the present disclosure, assuming that there are three second feature maps (i.e., C3 to C5), the number of neurons in the corresponding second fully connected layer is 3. Those skilled in the art will understand that if the number of second feature maps is different in actual applications, the number of neurons in the second fully connected layer in the corresponding weight generation network will also be adaptively adjusted according to the number of second feature maps. The embodiment of the present disclosure does not limit the number of second feature maps or the number of neurons in the second fully connected layer.

[0091] In some embodiments, the carrier inputs the first feature map into a weight generation network, and determines a proportion of propagation of the first feature map to each second feature map based on a global average pooling layer, a first fully connected layer, and a second fully connected layer.

[0092] In some embodiments, the carrier inputs the first feature map F2 after 1×1 convolution processing to the global average pooling layer, the first fully connected layer and the second fully connected layer, and determines the output of the second fully connected layer, which is the ratio of the first feature map to the at least one second feature map. Figure 3As shown in the figure, the outputs of the second fully connected layer are w3 to w5, which correspond to the propagation ratios of at least one second feature map C3 to C5. That is, w3 is the propagation ratio of the first feature map to the second feature map C3; w4 is the propagation ratio of the first feature map to the second feature map C4; and w5 is the propagation ratio of the first feature map to the second feature map C5.

[0093] Step S203: input the first feature map into a feature size matching network to adjust the size of the first feature map.

[0094] In some embodiments, the feature size matching network includes at least one maximum pooling layer of different sizes and different step sizes. Optionally, the number of maximum pooling layers in the feature size matching network is the same as the number of the at least one second feature map, and each maximum pooling layer corresponds to a second feature map, that is, the output of the maximum pooling layer is a first feature map adapted to the size of the corresponding second feature map.

[0095] In some embodiments, the carrier inputs the first feature map into each maximum pooling layer included in the feature size matching network, and determines the output of each maximum pooling layer as a feature map corresponding to the size of the second feature map. Figure 3 As shown in FIG, the feature size matching network includes three maximum pooling layers, and the outputs are U3, U4, and U5, which are feature maps corresponding to the second feature maps C3, C4, and C5 respectively.

[0096] Step S204: determining spatial position compensation information corresponding to the at least one second feature map based on the propagation ratio of the first feature map to the at least one second feature map and the adjusted first feature map.

[0097] In some embodiments, the carrier calculates the ratio of the propagation of the first feature image to the fourth feature map based on scalar multiplication and the product of the first feature map adjusted to fit the size of the fourth feature map, and determines the result of the scalar multiplication as the spatial position compensation information corresponding to the fourth feature map. Wherein, the fourth feature map is any second feature map after convolution processing (i.e., 1×1 convolution), that is, the fourth feature map is one of C3 to C5.

[0098] In some embodiments, the carrier may calculate the spatial position compensation information O corresponding to at least one second feature map by the following formula:

[0099] Oj=Uj×wj

[0100] Among them, j is the second feature map identifier and has a value of [3,5]. It can also correspond to the network layer identifier of ResNet50. That is, when j is 3, it corresponds to the third network layer in ResNet50 (corresponding to C3, F3, etc.). O3 corresponds to the spatial position compensation information of the second feature map C3; O4 corresponds to the spatial position compensation information of the second feature map C4; O5 corresponds to the spatial position compensation information of the second feature map C5.

[0101] Step S205 , performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map.

[0102] In some embodiments, the carrier may directly add the corresponding spatial position compensation information to the corresponding second feature map to obtain a corresponding third feature map, that is, directly add Ci to Oi to obtain at least one third feature map.

[0103] In other embodiments, the carrier may further pre-convolute at least one second feature map with a 1×1 convolution operation to make the number of channels of each feature map (including the first feature map and the second feature map) the same, thereby obtaining second feature maps F3 to F5. The at least one second feature map after the convolution operation is then added to the corresponding spatial position compensation information to obtain at least one third feature map. That is, Fi is directly added to Oi to obtain at least one third feature map.

[0104] In some further embodiments, the carrier may perform 1×1 convolution processing on at least one second feature map respectively so that the number of channels of each feature map is the same, to obtain second feature maps F3~F5; then perform content enhancement processing on the other feature maps F2~F4 except the second feature map with the least position structure information (i.e., F5) to obtain at least one content enhanced feature map (M2~M4), and perform spatial compensation on the at least one content enhanced feature map based on the corresponding spatial position compensation information to obtain at least one third feature map.

[0105] Specifically, the feature map with more positional structure information among the first feature map and at least one second feature map is determined to be the first initial feature map, and the feature map with less positional structure information is determined to be the second initial feature map; the second initial feature map is upsampled, and the upsampled result is added to the first initial feature map to obtain a content-enhanced feature map; the content-enhanced feature map and the corresponding spatial position compensation information are added, and the sum is the spatial compensation result. Among them, the second feature map (i.e., F5), which has the least positional structure information and is also the output of the deepest network layer, contains the most semantic information and is not subjected to content enhancement processing.

[0106] For example, the carrier upsamples the second feature map F5, adds the upsampled result to the second feature map F4, obtains a content-enhanced feature map M4 corresponding to the second feature map F4, and adds the content-enhanced feature map M4 to the spatial position compensation information O4, resulting in a spatial compensation result N4. The carrier upsamples the second feature map F4, adds the upsampled result to the second feature map F3, obtains a content-enhanced feature map M3 corresponding to the second feature map F3, and adds the content-enhanced feature map M3 to the spatial position compensation information O3, resulting in a spatial compensation result N3. The carrier upsamples the second feature map F3, adds the upsampled result to the first feature map F2, obtains a content-enhanced feature map M2 corresponding to the first feature map F2. The carrier directly adds the second feature map F5 to the spatial position compensation information O5, resulting in a spatial compensation result N5.

[0107] Step S206: performing boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps.

[0108] The specific steps of step S206 are the same as those of step S103 and will not be repeated here.

[0109] Step S207: determining a detection result of the image to be detected based on the at least two position information enhanced feature maps.

[0110] The specific steps of step S207 are the same as those of step S104 and will not be repeated here.

[0111] In this way, through the detection method provided by the embodiment of the present disclosure, the deep feature map is compensated by the spatial position information in the shallow feature map, and at the same time, the boundary information of each feature map is strengthened, thereby improving the feature extraction capability of the network, and then performing image detection based on the feature map that compensates for the spatial position information and strengthens the boundary information, thereby improving the accuracy of surface defect detection of electronic equipment.

[0112] Figure 4 FIG. 4 shows a third optional flow chart of the detection method provided in an embodiment of the present disclosure, Figure 5 A schematic structural diagram of a boundary reinforcement module provided in an embodiment of the present disclosure is shown; the module will be described according to each step.

[0113] In some embodiments, the carrier may implement the boundary reinforcement processing method described in step S103 or step S206 through a boundary reinforcement (BR) module, specifically including:

[0114] Step S301: Perform convolution processing on any of the feature maps to obtain a first sub-feature map of the feature map.

[0115] The feature map described in the embodiment of the present disclosure may be at least one third feature map obtained through spatial compensation in step S102 and any feature map in the first feature map. Furthermore, the first feature map may be the first feature map C2 directly output by the network layer of ResNet50, or the first feature map F2 obtained through 1×1 convolution with a uniform number of channels, or the first feature map M2 obtained through content enhancement; the third feature map may be at least one second feature map (i.e., N3 to N5) that has undergone spatial compensation.

[0116] In some embodiments, the boundary enhancement module includes two branches, namely a 3×3 convolution and a high frequency enhancement (HFE) module; the 3×3 convolution is used to further extract features in the feature map to obtain a first sub-feature map; the high frequency enhancement module is used to enhance the boundary information in the feature map to obtain a second sub-feature map.

[0117] In some embodiments, the carrier performs a 3×3 convolution on each feature map to obtain the first sub-feature map of the corresponding feature map, i.e. Figure 5 Middle K.

[0118] Step S302: performing convolution processing on the high-frequency component of the spatial domain of any feature map to obtain a second sub-feature map of any feature map.

[0119] In some embodiments, the carrier performs a two-dimensional discrete Fourier transform (2D-DFT) on any feature map to convert the spatial domain feature map into a frequency domain feature map; then separates the high-frequency component from the frequency domain feature map, performs a two-dimensional discrete Fourier inverse transform on the high-frequency component to obtain an inverse transform result, and then performs a convolution process on the inverse transform result to obtain a second sub-feature map, that is, Figure 5 L. The performing convolution processing on the inverse transformation result may include performing 3×3 convolution processing on the inverse transformation result.

[0120] Step S303: Determine the sum of the first sub-feature map and the second sub-feature map as the position information enhanced feature map corresponding to any one of the feature maps.

[0121] In some embodiments, the carrier adds the first sub-feature map K and the second sub-feature map L obtained by the two branches of the boundary enhancement module to obtain a position information enhanced feature map V with enhanced boundary information. The carrier can perform steps S301 to S303 on the first feature map and at least one third feature map, respectively, to obtain position information enhanced feature maps V2 to V5.

[0122] In this way, to address the problem that the boundary information of small objects is easily ignored, the embodiment of the present disclosure provides a boundary enhancement module, which utilizes the high-frequency part of the image frequency domain information that is rich in boundary information to enhance the expression of the boundary information of the feature map.

[0123] Figure 6 A fourth optional flow chart of the detection method provided in the embodiment of the present disclosure is shown. Figure 7 A schematic diagram of the structure of a faster regional convolutional neural network provided by an embodiment of the present disclosure is shown; it will be explained according to each step.

[0124] In some embodiments, the carrier can implement the detection method described in step S104 or step S207, that is, step S401 to step S402, through a faster region convolutional neural network (Faster Region-based Convolutional Neural Network, Faster R-CNN).

[0125] In some embodiments, Faster R-CNN includes a backbone network with PAE-FPN, a Region Proposal Networks (RPN), a Region of Interest Pooling Network (ROI pooling), and a third fully connected layer.

[0126] Step S401: Input at least two position information enhanced feature maps into a candidate region extraction network, and output a marker feature map.

[0127] In some embodiments, after the carrier obtains the position information enhanced feature maps V2 to V5 based on step S103 or step S207, it processes each of the position information enhanced feature maps separately to obtain at least two feature maps P2 to P6 to be detected.

[0128] Specifically, the carrier can perform convolution processing on the position information enhanced feature maps V2 to V5 respectively to obtain corresponding feature maps to be detected P2 to P5; optionally, the convolution processing can be a 3×3 convolution processing. The feature map to be detected P5 is input to the maximum pooling layer, and the output of the maximum pooling layer is determined to be the feature map to be detected P6.

[0129] In some embodiments, the carrier inputs at least two feature maps to be detected into a candidate region extraction network and outputs an identification feature map.

[0130] Specifically, the example of processing any feature map to be detected by the candidate region extraction network is used as an example: the RPN generates multiple anchor frames (such as 9 anchor frames in 3 sizes and 3 aspect ratios) at each position of the feature map to be detected; the anchor frames are of predefined sizes and aspect ratios, covering areas of different sizes and shapes. The RPN classifies each anchor frame to determine whether each anchor frame includes a target (defect), and the classification results include foreground (including target / defect) or background (excluding target / defect). For anchor frames classified as foreground, the RPN further adjusts the position and size of the anchor frame to more accurately frame the target; specifically, the center point coordinates, width and height of the anchor frame are adjusted through the boundary regression task to make the anchor frame fit the boundary of the target more closely; after classification and boundary regression, the RPN outputs the adjusted anchor frame, which is the candidate region, that is, the region that may include the target, that is, the identification feature map; the identification feature map includes the anchor frame.

[0131] Step S402: input the identification feature map and the at least two position information enhanced feature maps into a region of interest pooling network, and output a detection result of the image to be detected.

[0132] In some embodiments, the identification feature map corresponds to part or all of the area of the position information enhancement feature map; the carrier maps the identification feature map to the at least two position information enhancement feature maps, and extracts at least one target identification feature map from the at least two position information enhancement feature maps; the at least one target identification feature map is input into the third fully connected layer for detection (classification) and boundary regression.

[0133] In some optional embodiments, in response to the detection result indicating the presence of a defect, the region of interest pooling network outputs the image to be detected with an anchor frame identification; the anchor frame is used to identify the defect in the image to be detected.

[0134] Specifically, the identification feature map can be represented by (x, y, w, h), where (x, y) is the coordinate of the upper left corner of the candidate region, and (w, h) is the width and height of the candidate region. The multi-scale position information enhanced feature maps (such as P2, P3, P4, P5, P6) output by PAE-FPN contain rich semantic information. Since the candidate region is generated based on the feature map, ROIPooling first maps the candidate region to the corresponding feature map. For example, if the candidate region is generated based on the position information enhanced feature map P4, then ROIPooling will directly extract the corresponding region on the position information enhanced feature map P4 to obtain the target identification feature map. For each candidate region, ROIPooling divides it into a grid of fixed size (such as a 7×7 grid).

[0135] For example, if the output size of ROIPooling is 7×7, each target identification feature map will be divided into 49 grids. The feature value in each grid is max-pooled, that is, the maximum value in each grid is taken as the output of the grid. Through max-pooling, ROIPooling uniformly adjusts candidate regions of different sizes to a fixed size (such as a 7×7 feature map). The output of ROI Pooling is a fixed-size feature map (such as 7×7), which is input into the subsequent third fully connected layer for target classification (detection) and boundary regression.

[0136] In this way, through the detection method provided by the embodiment of the present disclosure, by processing multiple position information enhanced feature maps (V2~V5), multiple feature maps to be detected (P2~P6) are generated, which can capture feature information of different scales and enhance the model's detection ability for targets of different sizes; the candidate region extraction network (RPN) generates multiple anchor frames and classifies and regresses the boundaries of each anchor frame, which can effectively identify the target area and optimize its position and size, thereby improving the accuracy of target detection; by mapping the candidate area to the corresponding feature map and performing the maximum pooling operation, the candidate areas of different sizes can be uniformly adjusted to a feature map of a fixed size, which is convenient for subsequent classification and boundary regression processing; the image to be detected with the anchor frame mark output by the region of interest pooling network can accurately identify the defect position in the image, thereby improving the accuracy and visualization effect of defect detection.

[0137] Figure 8 FIG. 5 shows a fifth optional flow chart of the detection method provided in the embodiment of the present disclosure. Figure 9 The schematic diagram of the structure of the enhanced feature pyramid network provided by the embodiment of the present disclosure is shown. Figure 8 and Figure 9 Provide explanation.

[0138] like Figure 9 As shown, the enhanced feature pyramid network (PAE-FPN) includes a ResNet50 network for generating multi-scale feature maps from shallow to deep; a feature pyramid network (FPN) for fusing multi-scale feature maps; wherein, a spatial position compensation module (SPC) is embedded in the feature pyramid network for adding spatial position compensation information to the multi-scale feature map; the feature pyramid network also includes a boundary reinforcement module (BR) for strengthening the boundary of the feature map.

[0139] Step S501: Generate at least two multi-scale feature maps.

[0140] In some embodiments, as Figure 9As shown in the figure, the ResNet50 network includes five network layers from shallow to deep: Conv1, Conve2_x, Conve3_x, Conve4_x and Conve5_x; among them, Conv1 is closest to the input layer and is the shallowest network layer, and its output includes more positional structure information; Conve5_x is farthest from the input layer and closest to the output layer, and is the deepest network layer, and its output includes more semantic information.

[0141] In some embodiments, the carrier inputs the image to be detected into the ResNet50 network, extracts typical features of the image to be detected, and obtains data of several layers such as C1, C2, C3, C4, and C5. In order to reduce calculation, the data of C2, C3, C4, and C5 are taken as the input of the subsequent PAE-FPN network, where the shallow first feature map C2 has more positional structural information, and as the network deepens, the semantic information in the second feature maps C3, C4, and C5 will gradually become prominent.

[0142] Among them, the first feature map C2 includes multiple feature maps output by the network layer Conve2_x; the second feature maps C3 to C5 respectively include multiple feature maps output by relatively deep network layers, that is, the second feature map C3 includes multiple feature maps output by the network layer Conve3_x, the second feature map C4 includes multiple feature maps output by the network layer Conve4_x, and the second feature map C5 includes multiple feature maps output by the network layer Conve5_x.

[0143] Step S502: construct an FPN network.

[0144] In some embodiments, the FPN network includes at least one 1×1 convolutional network, an upsampling unit, at least one 3×3 convolutional network, a maximum pooling layer, an SPC module, and a BR module. The FPN first operates on C2, C3, C4, and C5 using a 1×1 convolution kernel to unify the number of channels in each feature map, resulting in F2, F3, F4, and F5, respectively. The deep semantic features are then upsampled and transferred to the shallow layers to obtain M2, M3, and M4, enriching the semantic information in the shallow layers. The SPC module is used to supplement the spatial information of M3, M4, and F5. The feature maps N3, N4, and N5 are obtained. Based on the BR module, the boundaries of M2, N3, N4, and N5 are enhanced respectively. The feature maps after boundary enhancement are finally convolved with a convolution kernel size of 3×3 to obtain the final outputs P2, P3, P4, and P5. At the same time, the maximum pooling is applied to P5 to obtain P6. The P6 feature map has a larger receptive field, which makes it more suitable for detecting targets that occupy a larger area in the original image. The specific method for processing the feature map is as follows:

[0145] Step S503: Perform spatial compensation on at least two feature maps based on the SPC module.

[0146] In some embodiments, the main function of the SPC module is to adaptively propagate the position structure information in the first feature map F2 obtained by ResNet50 and 1×1 convolution in step S502 to the second feature maps F3, F4, F5, etc. The structure of the SPC module is as follows: Figure 3 As shown, from Figure 3 It can be seen that SPC mainly consists of two main parts: the weight generation network (WGN) and the feature size adaptation (FSA). The weight generation network is used to generate the ratio of the first feature map F2 to the second feature map F3, F4, and F5. The feature size matching network is used to adjust the size of the first feature map F2 to adapt to the size of the second feature map F3, F4, and F5 to facilitate subsequent fusion.

[0147] In some embodiments, the weight generation network first performs global average pooling (GAP) on the first feature map F2, and the formula is:

[0148]

[0149] Among them, z c is the result of global average pooling of the cth channel, u c (m,n) is the value of the cth channel at position (m,n), and W and H are the width and height of the first feature map respectively.

[0150] In some embodiments, the carrier inputs the output after global average pooling into the following two fully connected layers (the first fully connected layer and the second fully connected layer). The first fully connected layer is set with 128 neurons and the activation function is ReLU. The second fully connected layer is set with 3 neurons and the activation function is Sigmoid, which is used to compress the output to the range of (0,1), so that the weights w3, w4, and w5 corresponding to the F3, F4, and F5 layers can be obtained respectively.

[0151] In some embodiments, the feature size matching network adapts the sizes of the second feature maps F3, F4, and F5 respectively by applying maximum pooling operations of different sizes and step sizes to the input data, and obtains U3, U4, and U5 by inputting the first feature map F2 and applying maximum pooling.

[0152] Finally, by scalar multiplication, that is:

[0153] Oj=Uj×wj

[0154] Among them, j is the second feature map identifier and has a value of [3,5]. It can also correspond to the network layer identifier of ResNet50. That is, when j is 3, it corresponds to the third network layer in ResNet50 (corresponding to C3, F3, etc.). O3 corresponds to the spatial position compensation information of the second feature map C3; O4 corresponds to the spatial position compensation information of the second feature map C4; O5 corresponds to the spatial position compensation information of the second feature map C5.

[0155] In some embodiments, when the carrier performs spatial compensation on the second feature maps F3, F4, and F5, it can first determine that among the first feature map and at least one second feature map, the feature map with more positional structure information is the first initial feature map, and the feature map with less positional structure information is the second initial feature map; upsample the second initial feature map, and add the upsampled result to the first initial feature map to obtain a content-enhanced feature map; add the corresponding spatial position compensation information in the content-enhanced feature map, and the addition result is the spatial compensation result. Among them, the second feature map (i.e., F5) with the least positional structure information and the output of the deepest network layer contains the most semantic information and is not subjected to content enhancement processing.

[0156] For example, the carrier upsamples the second feature map F5, adds the upsampled result to the second feature map F4, obtains the content enhancement feature map M4 corresponding to the second feature map F4, adds the content enhancement feature map M4 to the spatial position compensation information O4, and the addition result is the spatial compensation result N4; the carrier upsamples the second feature map F4, adds the upsampled result to the second feature map F3, obtains the content enhancement feature map M3 corresponding to the second feature map F3, adds the content enhancement feature map M3 to the spatial position compensation information O3, and the addition result is the spatial compensation result N3; the carrier upsamples the second feature map F3, adds the upsampled result to the first feature map F2, and obtains the content enhancement feature map M2 corresponding to the first feature map F2. The carrier directly adds the second feature map F5 and the spatial position compensation information O5, and the addition result is the spatial compensation result N5. That is:

[0157] N3=O3+M3

[0158] N4=O4+M4

[0159] N5=O5+M5

[0160] Step S504: perform boundary enhancement on at least two feature maps based on the BR module.

[0161] In some embodiments, the main function of the BR module is to further enhance the expression of boundary features in the feature maps N3, N4, and N5. The structure of the constructed BR module is as follows: Figure 5As shown in the figure, its main structure also contains two branches. The upper part is a 3×3 convolution for further feature extraction, and the lower part is a high frequency enhancement module (High Frequency Enhancement, HFE) for strengthening the boundary information in the feature map.

[0162] In some embodiments, the high frequency enhancement module first uses a two-dimensional discrete Fourier transform to transform the spatial domain feature map x∈R W×G Convert to frequency domain features Figure X ∈C W×H , X = F(x), and then separate the high frequency component X from X h ,

[0163]

[0164] Where M and N are the size parameters of the frequency domain feature map in the two-dimensional discrete Fourier transform (2D-DFT). They can be the number of rows and columns of the frequency domain feature map. For example, if the input spatial domain feature map has an H × W size, after 2D-DFT, the frequency domain feature map has an M × N size, where M and N are usually related to H and W.

[0165] In some embodiments, the carrier performs a two-dimensional discrete Fourier inverse transform on the high-frequency component and converts the data back to the spatial domain to obtain x h ;x h =F -1 (X h ), and finally a 3×3 convolution operation is performed to extract the high-frequency component x h Features, and by adding the feature maps obtained by the two branches, we can get the feature map V that enhances the boundary information, that is, V = K + L.

[0166] The feature maps M2, N3, N4, and N5 are respectively input into the BR module to obtain the position information enhanced feature maps V2 to V5.

[0167] Step S505 : Acquire at least two feature maps to be detected based on at least two position information enhanced feature maps.

[0168] In some embodiments, the carrier may perform convolution processing on the position information enhanced feature maps V2 to V5 respectively to obtain corresponding feature maps to be detected P2 to P5; optionally, the convolution processing may be a 3×3 convolution processing. The feature map to be detected P5 is input to the maximum pooling layer, and the output of the maximum pooling layer is determined to be the feature map to be detected P6.

[0169] Step S506: Obtain a detection result of the image to be detected based on the feature map to be detected.

[0170] In some embodiments, the carrier embeds PAE-FPN into Faster R-CNN, mainly adding PAE-FPN after the backbone network to extract multi-scale features P2, P3, P4, P5, P6 after position information enhancement, and input P2, P3, P4, P5, P6 into the candidate region extraction network to generate various basic anchor frames, and then ROI pooling inputs the collected anchor frames and P2, P3, P4, P5, and generates a predicted feature map by calculation. Finally, after processing through the fully connected layer, classification and boundary regression tasks are performed respectively.

[0171] The candidate region extraction network, the region of interest pooling network and the third fully connected layer process the feature map to be detected to obtain the detection result of the image to be detected. The specific steps are the same as steps S401 to S402 and will not be repeated here.

[0172] In some embodiments, after the carrier constructs the PAE-FPN network based on steps S501 to S505 , the carrier further includes training the PAE-FPN network.

[0173] In this way, through the detection method provided by the embodiment of the present disclosure, the performance of the defect detection model is significantly improved by introducing the spatial position compensation module and the boundary enhancement module. First, in response to the problem of insufficient position structure information in the deep feature map, the spatial position compensation module can adaptively transfer the shallow position information to the deep layer, ensuring that the model can still retain accurate position information when extracting deep features, thereby improving the accuracy of target positioning. Secondly, in response to the problem that the boundary information of small targets is easily ignored, the boundary enhancement module effectively enhances the boundary expression ability of the feature map by utilizing the high-frequency part rich in boundary information in the image frequency domain, so that the model can better capture the detailed features of small targets. Finally, the spatial position compensation module and the boundary enhancement module are integrated into the FPN, and the PAE-FPN structure is proposed, which further optimizes the multi-scale feature fusion capability of the feature pyramid and significantly improves the accuracy and robustness of computer surface defect detection. This solution not only solves the problems of loss of deep feature position information and insufficient boundary information of small targets, but also provides an efficient and reliable solution for defect detection in complex scenarios.

[0174] Figure 10 An optional structural diagram of the detection device provided in an embodiment of the present disclosure is shown, and will be described according to each part.

[0175] In some embodiments, the detection device 700 includes a residual network module 701 , a space compensation module 702 , a boundary enhancement module 703 and a detection module 704 .

[0176] The residual network module 701 is used to obtain a first feature map and at least one second feature map of the image to be detected, wherein the first feature map has more positional structure information than the second feature map;

[0177] The spatial compensation module 702 is configured to obtain spatial position compensation information corresponding to the at least one second feature map based on the first feature map, and perform spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map;

[0178] The boundary enhancement module 703 is configured to perform boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps;

[0179] The detection module 704 is configured to determine a detection result of the image to be detected based on the at least two position information enhanced feature maps.

[0180] The space compensation module 702 is specifically configured to input the first feature map into a weight generation network and determine a propagation ratio of the first feature map to the at least one second feature map;

[0181] Inputting the first feature map into a feature size matching network to adjust the size of the first feature map so that the adjusted first feature map is adapted to the at least one second feature map;

[0182] Based on the ratio of propagation of the first feature map to the at least one second feature map and the adjusted first feature map, spatial position compensation information corresponding to the at least one second feature map is determined.

[0183] The space compensation module 702 is specifically configured to perform convolution processing on the first feature map to adjust the number of channels of the first feature map;

[0184] Input the first feature map into a global average pooling layer, a first fully connected layer, and a second fully connected layer, and determine an output of the second fully connected layer, which is, in order, a ratio of propagation of the first feature map to the at least one second feature map;

[0185] The number of neurons in the second fully connected layer is the same as the number of the at least one second feature map.

[0186] The spatial compensation module 702 is specifically used to calculate the ratio of the propagation of the first feature image to the fourth feature map based on scalar multiplication, and the product of the first feature map adjusted to adapt to the size of the fourth feature map, and determine that the result of the scalar multiplication is the spatial position compensation information corresponding to the fourth feature map.

[0187] The space compensation module 702 is specifically configured to determine that the feature map with more position structure information in the first feature map and the at least one second feature map is the first initial feature map, and the feature map with less position structure information is the second initial feature map;

[0188] Upsampling the second initial feature map, and adding the upsampling result to the first initial feature map to obtain a content enhancement feature map;

[0189] The content enhancement feature map and the corresponding spatial position compensation information are added together, and the addition result is the spatial compensation result.

[0190] The boundary enhancement module 703 is specifically configured to perform convolution processing on any of the feature maps to obtain a first sub-feature map of any of the feature maps;

[0191] Performing convolution processing on the high-frequency component of the spatial domain of any one of the feature maps to obtain a second sub-feature map of any one of the feature maps;

[0192] Determine the sum of the first sub-feature map and the second sub-feature map as the position information enhanced feature map corresponding to any one of the feature maps.

[0193] The boundary enhancement module 703 is specifically configured to perform a two-dimensional discrete Fourier transform on any of the feature maps to obtain high-frequency components;

[0194] Perform a two-dimensional inverse discrete Fourier transform on the high-frequency component to obtain a second sub-feature map of any one of the feature maps.

[0195] The detection module 704 is specifically configured to input the at least two location information enhancement feature maps into a candidate region extraction network and output a marking feature map;

[0196] Inputting the identification feature map and the at least two position information enhanced feature maps into a region of interest pooling network, and outputting a detection result of the image to be detected;

[0197] The detection result includes the presence or absence of a defect.

[0198] The detection module 704 is specifically configured to include an anchor frame in the identification feature map;

[0199] In response to the detection result indicating that a defect exists, the region of interest pooling network outputs the image to be detected with an anchor frame identification; the anchor frame is used to identify the defect in the image to be detected.

[0200] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0201] Figure 11A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0202] like Figure 11 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0203] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0204] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the detection method. For example, in some embodiments, the detection method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the detection method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the detection method in any other appropriate manner (e.g., by means of firmware).

[0205] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0206] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0207] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0209] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0210] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0211] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0212] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means two or more, unless otherwise specifically defined.

[0213] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A detection method, comprising: Obtaining a first feature map and at least one second feature map of an image to be detected, wherein the first feature map has more positional structure information than the second feature map; Obtaining spatial position compensation information corresponding to each of the at least one second feature map based on the first feature map, and performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map; Performing boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps; A detection result of the image to be detected is determined based on the at least two position information enhanced feature maps.

2. The method according to claim 1, wherein obtaining the spatial position compensation information corresponding to the at least one second feature map based on the first feature map comprises: Inputting the first feature map into a weight generation network, and determining a ratio of propagation of the first feature map to the at least one second feature map; Inputting the first feature map into a feature size matching network to adjust the size of the first feature map; so that the adjusted first feature map adapts to the at least one second feature map; Based on the ratio of propagation of the first feature map to the at least one second feature map and the adjusted first feature map, spatial position compensation information corresponding to the at least one second feature map is determined.

3. The method according to claim 2, wherein inputting the first feature map into a weight generation network and determining a ratio of propagation of the first feature map to the at least one second feature map comprises: Performing convolution processing on the first feature map to adjust the number of channels of the first feature map; Input the first feature map into a global average pooling layer, a first fully connected layer, and a second fully connected layer, and determine an output of the second fully connected layer, which is, in order, a ratio of propagation of the first feature map to the at least one second feature map; The number of neurons in the second fully connected layer is the same as the number of the at least one second feature map.

4. The method according to claim 2, wherein determining the spatial position compensation information corresponding to the at least one second feature map based on the ratio of propagation of the first feature map to the at least one second feature map and the adjusted first feature map comprises performing the following operations on a fourth feature map in the at least one second feature map: The ratio of the first feature image propagating to the fourth feature map is calculated based on scalar multiplication, and the product of the first feature map adjusted to fit the size of the fourth feature map is determined, and the result of the scalar multiplication is determined to be the spatial position compensation information corresponding to the fourth feature map.

5. The method according to claim 1, wherein the step of performing spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information comprises: Determine, in the first feature map and the at least one second feature map, the feature map with more position structure information as the first initial feature map, and the feature map with less position structure information as the second initial feature map; Upsampling the second initial feature map, and adding the upsampling result to the first initial feature map to obtain a content enhancement feature map; The content enhancement feature map and the corresponding spatial position compensation information are added together, and the addition result is the spatial compensation result.

6. The method according to claim 1, wherein the performing boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps comprises performing the following operations on any one of the first feature map and the at least one third feature map: Performing convolution processing on any of the feature maps to obtain a first sub-feature map of any of the feature maps; Performing convolution processing on the high-frequency component of the spatial domain of any one of the feature maps to obtain a second sub-feature map of any one of the feature maps; Determine the sum of the first sub-feature map and the second sub-feature map as the position information enhanced feature map corresponding to any one of the feature maps.

7. The method according to claim 6, wherein the convolution processing is performed on the high-frequency component of the spatial domain of any feature map to obtain the second sub-feature map of any feature map, comprising: Performing a two-dimensional discrete Fourier transform on any of the feature maps to obtain a high-frequency component; Perform a two-dimensional inverse discrete Fourier transform on the high-frequency component to obtain a second sub-feature map of any one of the feature maps.

8. The method according to claim 1, wherein determining the detection result of the image to be detected based on the at least two position information enhanced feature maps comprises: Inputting the at least two position information enhanced feature maps into a candidate region extraction network and outputting a marking feature map; Inputting the identification feature map and the at least two position information enhanced feature maps into a region of interest pooling network, and outputting a detection result of the image to be detected; The detection result includes the presence or absence of a defect.

9. The method according to claim 8, The identification feature map includes an anchor box; In response to the detection result indicating that a defect exists, the region of interest pooling network outputs the image to be detected with an anchor frame identification; the anchor frame is used to identify the defect in the image to be detected.

10. A detection device, comprising: A residual network module, configured to obtain a first feature map and at least one second feature map of the image to be detected, wherein the first feature map has more positional structural information than the second feature map; a spatial compensation module, configured to obtain spatial position compensation information corresponding to the at least one second feature map based on the first feature map, and perform spatial compensation on the at least one second feature map based on the corresponding spatial position compensation information to obtain at least one third feature map; a boundary enhancement module, configured to perform boundary enhancement processing on the first feature map and the at least one third feature map to obtain at least two position information enhanced feature maps; A detection module is used to determine a detection result of the image to be detected based on the at least two position information enhanced feature maps.

Images