Image detection method, module, computer device and computer readable storage medium

Abstract

The application discloses an image detection method, a module, a computer device and a computer readable storage medium, which can improve the network's perception ability of target boundary and details in combination with space information to generate more accurate output feature maps. The method comprises the following steps: a space structure perception module acquires a to-be-processed feature map output by a first convolutional layer in an identity block to perform feature extraction, and obtains a first feature map, a second feature map and a third feature map; the space structure perception module calculates a first covariance feature map and a second covariance feature map corresponding to the to-be-processed feature map according to the first feature map and the second feature map respectively, and takes the product of the first covariance feature map and the second covariance feature map as an initial feature map; a spatial weight matrix corresponding to the to-be-processed feature map is determined, and a spatial attention feature map is determined according to the spatial weight matrix and the initial feature map; the product of the spatial attention feature map and the third feature map is calculated, the product is added to the to-be-processed feature map, and an output feature map is obtained.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an image detection method, module, computer device, and computer-readable storage medium. Background Technology

[0002] With the continuous development of computer vision technology, the rapid advancement of deep learning technology has brought new methods to billet image localization. Among related technologies, billet image localization and detection based on deep learning requires the use of deep learning models such as convolutional neural networks (CNN) to extract image features, and then uses the extracted feature representations to locate the billet image.

[0003] In the process of developing this application, the applicant discovered that the relevant technology has at least the following problems:

[0004] Deep learning models such as convolutional neural networks (CNNs) typically focus on extracting global features for feature integration and representation learning, while neglecting the perception of target boundary details in complex scenes. This makes it difficult to accurately capture the positional relationship between the billet image and its surrounding area, resulting in inaccurate detection results. Summary of the Invention

[0005] In view of this, this application provides an image detection method, module, computer device and computer-readable storage medium, the main purpose of which is to solve the problem that current global image features cannot accurately capture the positional relationship between the billet image and its surrounding area, thus resulting in inaccurate billet positioning results.

[0006] According to a first aspect of this application, an image detection method is provided, the method comprising:

[0007] The spatial structure perception module in the identity block obtains the feature map to be processed from the output of the first convolutional layer in the identity block and performs feature extraction to obtain the first feature map, the second feature map and the third feature map;

[0008] The spatial structure perception module calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map, and uses the product of the first covariance feature map and the second covariance feature map as the initial feature map.

[0009] The spatial structure perception module determines the spatial weight matrix corresponding to the feature map to be processed, and determines the spatial attention feature map based on the spatial weight matrix and the initial feature map.

[0010] The spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, and adds the product to the feature map to be processed to obtain the output feature map.

[0011] According to a second aspect of this application, an image detection module is provided, the module comprising:

[0012] The spatial structure perception module is used to obtain the feature map to be processed from the output of the first convolutional layer in the identity block and extract features to obtain the first feature map, the second feature map and the third feature map;

[0013] The spatial structure perception module is further configured to calculate the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map respectively, and use the product of the first covariance feature map and the second covariance feature map as the initial feature map.

[0014] The spatial structure perception module is also used to determine the spatial weight matrix corresponding to the feature map to be processed;

[0015] The spatial structure perception module is further configured to determine a spatial attention feature map based on the spatial weight matrix and the initial feature map; calculate the product of the spatial attention feature map and the third feature map; and add the product to the feature map to be processed to obtain an output feature map.

[0016] According to a third aspect of this application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0017] According to a fourth aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0018] Using the above technical solution, this application provides an image detection method. The spatial structure perception module in the identity block of this application obtains the feature map to be processed output by the first convolutional layer in the identity block and performs feature extraction to obtain a first feature map, a second feature map, and a third feature map. Then, the spatial structure perception module in the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed, respectively, based on the first and second feature maps. The product of the first and second covariance feature maps is used as the initial feature map. The spatial structure perception module also needs to determine the spatial weight matrix corresponding to the feature map to be processed, and determine the spatial attention feature map based on the spatial weight matrix and the initial feature map. Finally, the spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, adds the product to the feature map to be processed, and obtains the output feature map. In this embodiment, by adding a spatial structure perception module after the first convolutional layer of the identity block, the expressive power of features can be improved by combining spatial information, enhancing the network's perception of target boundaries and details, generating a more accurate output feature map, and thus improving the accuracy of billet positioning results.

[0019] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0021] Figure 1 This paper illustrates a flowchart of an image detection method provided in an embodiment of this application.

[0022] Figure 2A This paper illustrates a flowchart of an image detection method provided in an embodiment of this application.

[0023] Figure 2B A schematic diagram of an image detection network architecture provided in an embodiment of this application is shown;

[0024] Figure 2C A schematic diagram of an image detection network architecture provided in an embodiment of this application is shown;

[0025] Figure 2D A schematic diagram of an image detection network architecture provided in an embodiment of this application is shown;

[0026] Figure 2E This illustration shows a schematic diagram of an image detection module provided in an embodiment of this application;

[0027] Figure 2F This illustration shows a schematic diagram of an image detection module provided in an embodiment of this application;

[0028] Figure 3 This illustration shows a schematic diagram of an image detection module provided in an embodiment of this application;

[0029] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0030] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0031] This application provides an image detection method, such as... Figure 1 As shown, the method includes:

[0032] 101. The spatial structure perception module in the identity block obtains the feature map to be processed from the output of the first convolutional layer in the identity block and performs feature extraction to obtain the first feature map, the second feature map and the third feature map.

[0033] Billet image localization is a crucial task in steel production, aiming to accurately determine the position and bounding box of the billet within an image. Billet image localization enables automated billet detection, classification, and tracking, thereby improving production efficiency and quality control. Traditional billet image localization methods often employ feature engineering and rule-based approaches to extract image features and locate the billet. However, these methods typically rely on expert experience and manually designed rules, and struggle to handle image complexity and variability. In recent years, the rapid development of deep learning technology has brought new opportunities for billet image localization. Deep learning-based billet image localization methods utilize deep learning models such as convolutional neural networks (CNNs) to learn feature representations and positional information within images. Trained on large amounts of labeled data, these models can automatically extract and learn complex features and patterns, thus achieving billet image localization. However, deep learning models like CNNs extract global features for image understanding and analysis. In complex scenes, the billet may be located at specific angles and positions. In such cases, global image features cannot accurately capture the positional relationship between the billet image and its surrounding area, leading to inaccurate billet localization results. Therefore, this application provides an image detection method. The spatial structure perception module in the identity block of this application obtains the feature map to be processed output by the first convolutional layer in the identity block and performs feature extraction to obtain a first feature map, a second feature map, and a third feature map. Then, the spatial structure perception module in the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed, respectively, based on the first and second feature maps. The product of the first and second covariance feature maps is used as the initial feature map. The spatial structure perception module also needs to determine the spatial weight matrix corresponding to the feature map to be processed, and determine the spatial attention feature map based on the spatial weight matrix and the initial feature map. Finally, the spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, adds the product to the feature map to be processed, and obtains the output feature map. In this application embodiment, by adding a spatial structure perception module after the first convolutional layer of the identity block, the expressive power of features can be improved by combining spatial information, enhancing the network's perception of target boundaries and details, generating a more accurate output feature map, and thus improving the accuracy of billet positioning results.

[0034] This method is applicable to object detection systems, which include a ResNet-50 deep neural network with added Spatial Structure Aware Non-local Modules (SSANM), additional layers, and a predictor. The ResNet-50 deep neural network comprises convolutional layers, pooling layers, and four spatial awareness stages. Each of the four spatial awareness stages includes a convolutional block and an identity block. The convolutional block consists of four convolutional layers, and the identity block consists of three convolutional layers and a spatial structure awareness module. The spatial structure awareness module is designed to capture long-range dependencies between different regions in an image through a self-attention mechanism, and to fuse spatial location and covariance information to enhance the global context representation. This structure-aware representation capability enables the network to better understand the structural information in the image, thereby achieving more accurate object detection. In this application, the ResNet-50 deep neural network removes the final average pooling layer and fully connected layer, and adds an SSANM module after the first convolutional layer of the identity block in each stage. The improved ResNet-50 network is used as the backbone network in the Single Shot Detector (SSD) method for feature extraction, thereby improving the performance and robustness of object detection.

[0035] In this embodiment, after the identity block in any spatial perception stage receives the feature map output by the convolutional block, it needs to undergo a convolution operation in the first convolutional layer to obtain the feature map to be processed. The first convolutional layer inputs the feature map to be processed into the spatial structure perception module for feature extraction, resulting in multiple feature representations, including the first feature map F. θ Second feature map and the third feature map F g These three features have different semantic meanings and can extract and represent different aspects of information from the feature map to be processed.

[0036] 102. The spatial structure perception module in the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map, and uses the product of the first covariance feature map and the second covariance feature map as the initial feature map.

[0037] In this embodiment, the SSANM of the identity block includes a Spatial Covariance Module (SC), a Spatial Weight Module (SW), and a Spatial Non-local Block (NLB). During actual runtime, the Spatial Covariance Module determines the identity block based on the first feature map F. θ Second feature map Calculate the first covariance feature map corresponding to the feature map to be processed. Second covariance feature map Covariance can reveal the similarity between neighboring features, highlighting structural boundaries in an image. It is used to obtain more structure-aware feature representations for similarity measurement, thus demonstrating good discriminative power and robustness. The first covariance feature map is then calculated by the spatial nonlocal module. Second covariance feature map The product of these factors is used as the initial feature map. It should be noted that the initial feature map has a global spatial view of all locations, which can capture long-range contextual information to extract the global background.

[0038] 103. The spatial structure perception module determines the spatial weight matrix corresponding to the feature map to be processed, and determines the spatial attention feature map based on the spatial weight matrix and the initial feature map.

[0039] In this embodiment, considering that the initial feature map ignores the spatial relationship between features, that is, it ignores the fact that the same feature that is spatially closer to the target location should contribute more than the feature that is farther away from the target location. Therefore, this application calculates and aggregates the weights between any two locations in the feature map to be processed through the spatial weight module to obtain the spatial weight matrix M. After multiplying the spatial weight M with the initial feature map element by element, the spatial attention feature map with spatial features is obtained by applying the softmax activation function.

[0040] 104. The spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, adds the product to the feature map to be processed, and obtains the output feature map.

[0041] In this embodiment of the application, the spatial nonlocal module in SSANM calculates the spatial attention feature map and the third feature map F. gThe product of the input and output features is added to the feature map to be processed to obtain the output feature map. It should be noted that SSANM can output feature maps with enhanced global context. Spatial nonlocal blocks are used to capture long-range dependencies between different locations in the input feature map. The similarity between each location in the feature map to be processed and all other locations can be calculated, and a weight can be assigned to each location based on the similarity value. To further enhance the structure awareness capability, we introduce a spatial weight module and a spatial covariance module. The spatial covariance module is used to capture the spatial covariance information between different locations in the feature map, replacing the original feature map extracted by the spatial nonlocal blocks with the captured first and second covariance feature maps to obtain the initial feature map. The spatial weight module further adjusts the weights of each location in the initial feature map, thereby better capturing important global context, improving the network's ability to perceive target boundaries and details, and enhancing the accuracy and robustness of saliency detection.

[0042] The method provided in this application embodiment involves a spatial structure perception module in the identity block acquiring the feature map to be processed output from the first convolutional layer in the identity block and extracting features to obtain a first feature map, a second feature map, and a third feature map. Then, the spatial structure perception module in the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed, respectively, based on the first and second feature maps. The product of the first and second covariance feature maps is used as the initial feature map. The spatial structure perception module also needs to determine the spatial weight matrix corresponding to the feature map to be processed, and determine the spatial attention feature map based on the spatial weight matrix and the initial feature map. Finally, the spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, adds the product to the feature map to be processed, and obtains the output feature map. In this application embodiment, by adding a spatial structure perception module after the first convolutional layer of the identity block, the expressive power of features can be improved by combining spatial information, enhancing the network's perception of target boundaries and details, generating a more accurate output feature map, and thus improving the accuracy of billet positioning results.

[0043] This application provides an image detection method, such as... Figure 2A As shown, the method includes:

[0044] 201. The detection device receives the image to be detected and inputs it into a deep convolutional neural network.

[0045] In this embodiment of the application, the detection device is a Single Shot Detector (SSD) network, such as... Figure 2BAs shown, the SSD network consists of an input layer, a ResNet-50 deep convolutional neural network, additional layers, and a predictor. The additional layers comprise four convolutional blocks (conv), each consisting of a 1x1 convolutional layer and a 3x3 convolutional layer, with an activation function applied after each convolutional layer. The predictor includes a bounding box generator (detections) and a bounding box suppressor (Non-Maximum Suppression). In this step, after the SSD network's input layer receives the image to be detected uploaded by the user from the client, it passes it to the ResNet-50 deep convolutional neural network, which outputs a target feature map focusing on spatial relationships.

[0046] 202. The deep convolutional neural network inputs the image to be detected into a 7x7 convolutional layer for convolution operation, inputs the extracted feature map into a pooling layer, halves the size of the feature map while keeping the number of channels unchanged, and obtains pooling features, which are then input into the spatial perception stage.

[0047] In the embodiments of this application, such as Figure 2C As shown, the deep convolutional neural network includes an input layer (input), a 7x7 convolutional layer (conv), a 3x3 pooling layer (Max pool), four spatial awareness stages, and an output layer (output). Each spatial awareness stage includes one convolutional block (conv) and one identity block (identity). The convolutional block (conv) consists of four convolutional layers and one stacking layer (Add), while the identity block (identityconv) consists of three convolutional layers, one spatial structure awareness module (Attention mechanism), and one stacking layer (Add). After receiving the image to be detected, the input layer of the deep convolutional neural network feeds the image into a 7x7 convolutional layer for convolution. The convolutional layer then feeds the extracted feature map into the pooling layer, which halves the size of the feature map while maintaining the number of channels, resulting in pooled features. These pooled features are then fed into the spatial awareness stage for image processing, outputting a target feature map that focuses on spatial relationships.

[0048] 203. In the spatial perception stage, spatial perception is performed on the feature map to be perceived, and the target feature map that focuses on the spatial positional relationship is output. The target feature map is then output to the predictor and the additional layer.

[0049] For each of the four spatial perception stages, the spatial perception feature map output by the previous spatial perception stage is used as the input feature map for the next spatial perception stage, until the last spatial perception stage outputs the final spatial perception feature map, which is the target feature map.

[0050] The following describes the process of outputting spatial perception feature maps in the spatial perception stage, such as... Figure 2D As shown:

[0051] First, the convolutional block receives the feature map to be perceived and extracts features. The extracted feature map is then input into the first convolutional layer of the identity block for convolution operation to obtain the feature map to be processed.

[0052] First, such as Figure 2C As shown, if the convolutional block is located in the first spatial perception stage, then the convolutional block in the current spatial perception stage obtains the pooling features output by the pooling layer in the deep neural network for feature extraction. The extracted feature map is then input into the first convolutional layer of the identity block for convolution operation to obtain the feature map to be processed. If the convolutional block is located in a spatial perception stage other than the first spatial perception stage, then the convolutional block in the current spatial perception stage obtains the spatial perception feature map output by the identity block in the previous spatial perception stage for feature extraction. The extracted feature map is then input into the first convolutional layer of the identity block for convolution operation to obtain the feature map to be processed.

[0053] 2. The spatial structure perception module of the identity block obtains the feature map to be processed from the output of the first convolutional layer in the identity block and performs feature extraction to obtain the first feature map, the second feature map and the third feature map. The first feature map, the second feature map and the third feature map are then input into the spatial structure perception module.

[0054] After the identity block in any spatial perception stage receives the feature map output by the convolutional block, it needs to undergo a convolution operation in the first convolutional layer to obtain the feature map to be processed. The first convolutional layer inputs the feature map to be processed into the spatial structure perception module for feature extraction, resulting in multiple feature representations, including the first feature map F. θ Second feature map and the third feature map F g The first feature map F θ Second feature map and the third feature map F g The input is fed into the spatial structure perception module. These three feature representations have different semantic meanings and can extract and represent different aspects of information from the feature map to be processed.

[0055] Third, the spatial structure perception module of the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map, and uses the product of the first covariance feature map and the second covariance feature map as the initial feature map.

[0056] Furthermore, the spatial covariance module in the spatial structure perception module, for each position in the feature map to be processed, takes that position as the center position and determines a local spatial region centered on that position. Then, it traverses the first feature map, determines the first feature corresponding to each position within the local spatial region, calculates the average covariance feature corresponding to the local spatial region based on the first feature, and uses the average covariance feature as the first covariance feature corresponding to the center position. It then traverses the second feature map, determines the second feature corresponding to each position within the local spatial region, calculates the average covariance feature corresponding to the local spatial region based on the second feature, and uses the average covariance feature as the second covariance feature corresponding to the center position. This process is repeated for each position in the feature map to be processed, taking it as the center position, and calculating the first and second covariance features corresponding to each center position. Finally, it aggregates the first covariance features corresponding to each position in the feature map to obtain the first covariance feature map, and aggregates the second covariance features corresponding to each position in the feature map to obtain the second covariance feature map. The specific process for calculating the average covariance feature is as follows:

[0057] The spatial covariance feature at position (i,j) in the feature map to be processed, i.e. This represents the average covariance characteristic within a (2k+1) x (2k+1) spatial region centered at position (i,j). Here, k is a preset range coefficient; for example, if k is 1, then the average covariance characteristic is calculated within a 3x3 region centered at position (i,j). Figure 2E As shown, the first or second feature at each location in the local spatial region is multiplied by its transpose to obtain a matrix product. Then, the matrix product corresponding to each location (including the center) is calculated, and the mean of multiple matrix products is used as the average covariance feature. Specifically, this can be calculated based on the following formula 1:

[0058] Formula 1:

[0059] Among them, F m,n This represents the input features from location (m,n). By aggregating the covariance features within a spatial region as a refined feature representation of the central location, the discriminative power of the features can be improved. The spatial nonlocal module in the spatial structure perception module uses the element-wise product of the first and second covariance feature maps as the initial feature map. The regional covariance of features is a statistical measure describing the relationship between features. By analyzing the spatial distribution and variation of features, the spatial covariance module can extract structural information between features. This structural information includes the correlation between features and the pattern of spatial layout, which helps to accurately capture the structural features of salient targets.

[0060] IV. The spatial structure perception module of the identity block determines the spatial weight matrix corresponding to the feature map to be processed.

[0061] In this step, the spatial weight module in the spatial structure perception module takes the spatial index of the initial feature map as input, calculates the weight between any two positions, and aggregates all calculated weights into a spatial weight matrix. Specifically, it obtains the spatial index corresponding to the initial feature map, and reads the two x-coordinates and two y-coordinates corresponding to two positions from the spatial index. Then, it calculates the first difference between the two x-coordinates and the second difference between the two y-coordinates, inputting the first and second differences into a preset distance function to calculate the weight between the two positions. Finally, it iterates through the spatial index, calculates and aggregates the weights between any two positions, and obtains the spatial weight matrix corresponding to the feature map to be processed. For example, as shown... Figure 2F As shown, a shape of R was constructed. NxN The spatial weight matrix M, where M (iH+j,pH+q) This represents the weight from spatial position (i,j) to position (p,q). The closer spatial positions (i,j) and (p,q) are, the greater their influence on each other should be. Specifically, this can be calculated based on Equations 2 and 3 below:

[0062] Formula 2: M (iH+j,pH+q) =D(ip,jq)

[0063] Formula 3::

[0064] Where a = ip, b = jq; ε is a hyperparameter that controls the deceleration rate of spatial weights relative to spatial distance. By calculating the spatial distance between features and converting it into weights, the spatial weight module can accurately express the differences in importance between different features, thus providing an effective basis for structure perception.

[0065] V. The spatial structure perception module of the identity block determines the spatial attention feature map based on the spatial weight matrix and the initial feature map, calculates the product of the spatial attention feature map and the third feature map, and adds the product to the feature map to be processed to obtain the output feature map.

[0066] In this embodiment, the spatial nonlocal module of the spatial structure perception module can specifically multiply the spatial weight matrix element-wise with the initial feature map according to the following formula 4 to obtain a matrix product, and then activate the matrix product using preset activation parameters to obtain a spatial attention feature map. Then, the spatial attention feature map and the third feature map F are calculated. g The product of the two features is added to the feature map to be processed to obtain the output feature map F. SSNAM .

[0067] Formula 4:

[0068] in, For the initial feature map, softmax is an activation function used to transform feature values ​​into a probability distribution such that all output values ​​are between 0 and 1, and their sum equals 1.

[0069] 6. The second convolutional layer of the identity block extracts features from the output feature map. The extracted features are then input into the third convolutional layer of the identity block for further feature extraction to obtain an enhanced feature map. The stacking layer of the identity block adds the enhanced feature map to the feature map to be processed element by element to obtain the spatially perceptual feature map.

[0070] The second convolutional layer of the identity block extracts features from the output feature map. The extracted features are then fed into the third convolutional layer of the identity block for further feature extraction, resulting in a reinforced feature map. The stacking layers of the identity block add the reinforced feature map and the feature map to be processed element-wise to obtain a spatially perceptual feature map. Further, the stacking layers input the spatially perceptual feature map into the next spatial perception stage for further spatial feature extraction, until the last stacking layer in the final spatial perception stage outputs the last spatially perceptual feature map, which is then used as the target feature map.

[0071] This application improves the ResNet-50 network by introducing the SSANM module. The SSANM module adaptively adjusts the importance of different features, enabling the network to focus more on features relevant to the current task. This helps improve the discriminative and expressive power of features, allowing the network to better distinguish between different object categories. Through feature weighting and selection, the SSANM module reduces the ResNet-50 network's sensitivity to irrelevant information. This means the network focuses more on task-relevant features, reducing its response to noise and interference, and improving the model's robustness and generalization ability. This is crucial for improving the network's performance in practical applications, especially in complex scenarios with noise and interference. Using the improved ResNet-50 as the backbone of the SSD network for feature extraction also has many advantages. First, ResNet-50 has a deeper architecture than shallower networks, enabling it to extract more abstract and discriminative features. This allows SSD to detect objects in images more accurately and with higher detection precision. Second, as the backbone of the SSD network, ResNet-50 can provide more feature pyramid levels to detect targets of different scales and sizes. This helps improve SSD's ability to detect objects at different scales and enables object detection and location regression to be completed simultaneously in a single forward propagation.

[0072] 204. The additional layer generates a multi-scale feature map based on the target feature map and inputs the multi-scale feature map into the predictor.

[0073] In the embodiments of this application, such as Figure 2B As shown, the deep neural network inputs the target feature map to an additional layer, which then feeds the target feature map into multiple unidirectionally connected convolutional blocks. By using the specified feature map output from the previous specified convolutional block as the input feature map for the next specified convolutional block, the transfer of feature maps between different convolutional blocks is achieved. Simultaneously, this process can also aggregate the specified feature maps output from each specified convolutional block, thus obtaining multi-scale feature maps.

[0074] 205. The predictor generates multiple bounding boxes for the target feature map and multi-scale feature map, and uses a non-maximum suppression algorithm to determine the specified bounding box that meets the preset conditions among the multiple bounding boxes as the billet positioning result.

[0075] Each feature map and target feature map at each scale generate a set of bounding boxes, with a maximum of four bounding boxes per set. These bounding boxes are typically centered on the feature map location and scaled and adjusted according to the scale factor and aspect ratio. Multiple sets of bounding boxes are aggregated and scored. A non-maximum suppression algorithm is used to determine the specified bounding box that meets preset conditions as the billet positioning result. Specifically, all bounding boxes are sorted according to their category scores, with the highest-scoring bounding box placed first. The highest-scoring bounding box is then selected as the initial selected bounding box and added to the final filtering list. Starting with the second-highest-scoring bounding box, overlap (e.g., IoU) is calculated sequentially with the selected bounding boxes. If the overlap between two bounding boxes exceeds a preset threshold (e.g., 0.5 or 0.6), the lower-scoring bounding box is suppressed (not selected). This process continues for the next higher-scoring bounding box, calculating overlap and suppressing until all bounding boxes have been traversed. The remaining unsuppressed bounding boxes are then selected as the specified bounding boxes.

[0076] The method provided in this application embodiment involves a spatial structure perception module in the identity block acquiring the feature map to be processed output from the first convolutional layer in the identity block and extracting features to obtain a first feature map, a second feature map, and a third feature map. Then, the spatial structure perception module in the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed, respectively, based on the first and second feature maps. The product of the first and second covariance feature maps is used as the initial feature map. The spatial structure perception module also needs to determine the spatial weight matrix corresponding to the feature map to be processed, and determine the spatial attention feature map based on the spatial weight matrix and the initial feature map. Finally, the spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, adds the product to the feature map to be processed, and obtains the output feature map. In this application embodiment, by adding a spatial structure perception module after the first convolutional layer of the identity block, the expressive power of features can be improved by combining spatial information, enhancing the network's perception of target boundaries and details, generating a more accurate output feature map, and thus improving the accuracy of billet positioning results.

[0077] Furthermore, as Figure 1 To specifically implement the method, this application provides an image detection module, such as... Figure 3 As shown, the module includes: a first convolutional layer 301 and a spatial structure perception module 302.

[0078] The first convolutional layer 301 is used to output the feature map to be processed;

[0079] The spatial structure perception module 302 is used to acquire the feature map to be processed and perform feature extraction to obtain a first feature map, a second feature map and a third feature map;

[0080] The spatial structure perception module 302 is used to calculate the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map respectively, and to use the product of the first covariance feature map and the second covariance feature map as the initial feature map.

[0081] The spatial structure perception module 302 is also used to determine the spatial weight matrix corresponding to the feature map to be processed;

[0082] The modified spatial structure perception module 302 is further configured to determine a spatial attention feature map based on the spatial weight matrix and the initial feature map; calculate the product of the spatial attention feature map and the third feature map; and add the product to the feature map to be processed to obtain an output feature map.

[0083] In specific application scenarios, if the identity block is located in the first spatial perception stage, the convolutional block in the first spatial perception stage obtains the pooling features output by the pooling layer in the deep neural network for feature extraction, and inputs the extracted feature map into the first convolutional layer in the identity block for convolution operation to obtain the feature map to be processed; if the identity block is located in other spatial perception stages, the convolutional block in the current spatial perception stage obtains the spatial perception feature map output by the identity block in the previous spatial perception stage for feature extraction, and inputs the extracted feature map into the first convolutional layer in the identity block for convolution operation to obtain the feature map to be processed. The other spatial perception stages are the second, third, or fourth spatial perception stages other than the first spatial perception stage; wherein, the deep neural network includes the pooling layer and multiple spatial perception stages, and each of the multiple spatial perception stages includes a convolutional block and an identity block.

[0084] In a specific application scenario, the spatial structure perception module 302 is used to: for each position in the feature map to be processed, take the position as the center position and determine a local spatial region centered on the position; traverse the first feature map, determine the first feature corresponding to each position in the local spatial region, calculate the average covariance feature corresponding to the local spatial region based on the first feature, and take the average covariance feature as the first covariance feature corresponding to the center position; traverse the second feature map, determine the second feature corresponding to each position in the local spatial region, calculate the average covariance feature corresponding to the local spatial region based on the second feature, and take the average covariance feature as the second covariance feature corresponding to the center position; sequentially take each position in the feature map to be processed as the center position and calculate the first covariance feature and the second covariance feature corresponding to each center position; aggregate the first covariance features corresponding to each position in the feature map to be processed to obtain the first covariance feature map, and aggregate the second covariance features corresponding to each position in the feature map to be processed to obtain the second covariance feature map.

[0085] In specific application scenarios, the spatial structure perception module 302 is also used to obtain the spatial index corresponding to the initial feature map, read the two horizontal coordinates and two vertical coordinates corresponding to two positions in the spatial index; calculate the first difference between the two horizontal coordinates and the second difference between the two vertical coordinates, input the first difference and the second difference into a preset distance function for calculation to obtain the weight between the two positions; traverse the spatial index, calculate and aggregate the weight between any two positions to obtain the spatial weight matrix corresponding to the feature map to be processed.

[0086] In specific application scenarios, the spatial structure perception module 302 is also used to multiply the spatial weight matrix element by element with the initial feature map to obtain a matrix product, and to activate the matrix product using preset activation parameters to obtain the spatial attention feature map.

[0087] In specific application scenarios, this module also includes a second convolutional layer 303, a third convolutional layer 304, and a stacking layer 305.

[0088] The second convolutional layer 303 is used to extract features from the output feature map and input the extracted features into the third convolutional layer 304 of the identity block;

[0089] The third convolutional layer 304 is used to perform feature extraction again to obtain an enhanced feature map;

[0090] The overlay layer 305 adds the enhanced feature map to the feature map to be processed element by element to obtain a spatial perception feature map; the spatial perception feature map is input to the next spatial perception stage for spatial feature extraction again, until the identity block in the last spatial perception stage outputs the last spatial perception feature map, and the last spatial perception feature map is used as the target feature map.

[0091] In specific application scenarios, the overlay layer 305 is used to input the target feature map to an additional layer, so that the additional layer inputs the target feature map to multiple unidirectionally connected specified convolutional blocks, and uses the specified feature map output by the previous specified convolutional block as the input feature map of the next specified convolutional block, and aggregates the specified feature maps output by each specified convolutional block to obtain a multi-scale feature map. The additional layer includes the multiple specified convolutional blocks. The additional layer inputs the target feature map and the multi-scale feature map to a preset predictor, so that the preset predictor generates multiple bounding boxes for the target feature map and the multi-scale feature map, and uses a non-maximum suppression algorithm to determine the specified bounding box that meets the preset conditions among the multiple bounding boxes as the billet positioning result.

[0092] The module provided in this application embodiment involves a spatial structure perception module within the identity block. This module extracts the feature map output from the first convolutional layer of the identity block, resulting in a first feature map, a second feature map, and a third feature map. The spatial structure perception module then calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed, based on the first and second feature maps, respectively. The product of the first and second covariance feature maps is used as the initial feature map. The spatial structure perception module also needs to determine the spatial weight matrix corresponding to the feature map to be processed, and based on the spatial weight matrix and the initial feature map, determines the spatial attention feature map. Finally, the spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, and adds the product to the feature map to obtain the output feature map. In this application embodiment, by adding a spatial structure perception module after the first convolutional layer of the identity block, the expressive power of features can be improved by combining spatial information, enhancing the network's perception of target boundaries and details, thereby generating a more accurate output feature map and improving the accuracy of billet positioning results.

[0093] It should be noted that other corresponding descriptions of the functional units involved in the image detection module provided in this application embodiment can be found in the following references. Figure 1 and Figures 2A to 2F The corresponding description in [the document] will not be repeated here.

[0094] Based on the above, Figure 1 , Figures 2A to 2F Accordingly, this embodiment also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the image detection method described above.

[0095] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), including several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0096] Based on the above, Figure 1 , Figures 2A to 2F The method shown, and Figure 3 The image detection module embodiment shown above, in order to achieve the above objectives, in the exemplary embodiment, see [reference needed]. Figure 4Furthermore, a device is provided, comprising a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device, wherein the various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory to perform the image detection method described in the above embodiments.

[0097] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0098] Those skilled in the art will understand that the physical device structure for image detection provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0099] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs to be identified. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the technical solution of this application, the spatial structure perception module in the identity block of this application obtains the feature map to be processed output by the first convolutional layer in the identity block and performs feature extraction to obtain a first feature map, a second feature map, and a third feature map. Then, the spatial structure perception module in the identity block calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed, respectively, based on the first feature map and the second feature map. The product of the first covariance feature map and the second covariance feature map is used as the initial feature map. The spatial structure perception module also needs to determine the spatial weight matrix corresponding to the feature map to be processed, and determine the spatial attention feature map based on the spatial weight matrix and the initial feature map. Finally, the spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, adds the product to the feature map to be processed, and obtains the output feature map. Compared with the prior art, the embodiments of this application add a spatial structure perception module after the first convolutional layer of the identity block, which can combine spatial information to improve the feature expression ability, improve the network's perception of target boundaries and details, generate more accurate output feature maps, and thus improve the accuracy of billet positioning results.

[0101] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0102] Those skilled in the art will understand that the modules in the apparatus of the implementation scenario can be distributed within the apparatus of the implementation scenario as described, or they can be located in one or more apparatuses different from this implementation scenario, with corresponding changes. The modules of the above-described implementation scenario can be combined into one module, or they can be further divided into multiple sub-modules.

Claims

1. An image detection method for determining the position and bounding box of a steel billet in an image, characterized in that, include: The system consists of four spatial awareness stages, each comprising a convolutional block and an identity block. Each identity block includes three convolutional layers, one spatial structure awareness module, and one stacking layer. Each identity block performs the following operations: The spatial structure perception module in the identity block obtains the feature map to be processed from the output of the first convolutional layer in the identity block and performs feature extraction to obtain the first feature map, the second feature map and the third feature map; The spatial structure perception module calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map, and uses the product of the first covariance feature map and the second covariance feature map as the initial feature map. The spatial structure perception module determines the spatial weight matrix corresponding to the feature map to be processed, and determines the spatial attention feature map based on the spatial weight matrix and the initial feature map. The spatial structure perception module calculates the product of the spatial attention feature map and the third feature map, and adds the product to the feature map to be processed to obtain the output feature map; The second convolutional layer of the identity block extracts features from the output feature map, and the extracted features are input into the third convolutional layer of the identity block for further feature extraction to obtain an enhanced feature map. The superposition layer of the identity block adds the enhanced feature map to the feature map to be processed element by element to obtain the spatial perception feature map; The overlay layer inputs the spatial perception feature map into the next spatial perception stage for further spatial feature extraction, until the overlay layer in the last spatial perception stage outputs the last spatial perception feature map, which is then used as the target feature map.

2. The method according to claim 1, characterized in that, The spatial structure perception module acquires the feature map to be processed from the output of the first convolutional layer and performs feature extraction, including: If the identity block is located in the first spatial perception stage, the convolutional block located in the first spatial perception stage obtains the pooling features output by the pooling layer in the deep neural network for feature extraction, and inputs the extracted feature map into the first convolutional layer in the identity block for convolution operation to obtain the feature map to be processed. If the identity block is located in another spatial perception stage, the convolutional block in the current spatial perception stage obtains the spatial perception feature map output by the identity block in the previous spatial perception stage for feature extraction, and inputs the extracted feature map into the first convolutional layer in the identity block for convolution operation to obtain the feature map to be processed. The other spatial perception stage is the second, third, or fourth spatial perception stage other than the first spatial perception stage. The deep neural network includes the pooling layer and multiple spatial perception stages, each of which includes a convolutional block and an identity block.

3. The method according to claim 1, characterized in that, The spatial structure perception module calculates the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map, respectively, including: For each position in the feature map to be processed, the position is taken as the center position, and a local spatial region centered on the position is determined. Traverse the first feature map to determine the first feature corresponding to each position in the local spatial region, calculate the average covariance feature corresponding to the local spatial region based on the first feature, and use the average covariance feature as the first covariance feature corresponding to the center position. Traverse the second feature map to determine the second feature corresponding to each position in the local spatial region. Calculate the average covariance feature corresponding to the local spatial region based on the second feature, and use the average covariance feature as the second covariance feature corresponding to the center position. Each position in the feature map to be processed is taken as the center position, and the first covariance feature and the second covariance feature corresponding to each center position are calculated. The first covariance feature map is obtained by aggregating the first covariance features corresponding to each position in the feature map to be processed, and the second covariance feature map is obtained by aggregating the second covariance features corresponding to each position in the feature map to be processed.

4. The method according to claim 1, characterized in that, The spatial structure perception module determines the spatial weight matrix corresponding to the feature map to be processed, including: Obtain the spatial index corresponding to the initial feature map, and read the two horizontal coordinates and two vertical coordinates corresponding to the two positions in the spatial index; Calculate the first difference between the two horizontal coordinates and the second difference between the two vertical coordinates. Input the first difference and the second difference into a preset distance function for calculation to obtain the weight between the two positions. By traversing the spatial index, the weights between any two positions are calculated and aggregated to obtain the spatial weight matrix corresponding to the feature map to be processed.

5. The method according to claim 1, characterized in that, The spatial structure perception module determines a spatial attention feature map based on the spatial weight matrix and the initial feature map, including: The spatial weight matrix is ​​multiplied element-wise with the initial feature map to obtain a matrix product, and the matrix product is activated using preset activation parameters to obtain the spatial attention feature map.

6. The method according to claim 1, characterized in that, After using the last spatially perceived feature map as the target feature map, the method further includes: The target feature map is input to an additional layer, which in turn inputs the target feature map to multiple unidirectionally connected specified convolutional blocks. The specified feature map output by the previous specified convolutional block is used as the input feature map of the next specified convolutional block. The specified feature maps output by each specified convolutional block are aggregated to obtain a multi-scale feature map. The additional layer includes the multiple specified convolutional blocks. The additional layer inputs the target feature map and the multi-scale feature map into a preset predictor, so that the preset predictor generates multiple bounding boxes for the target feature map and the multi-scale feature map, and uses a non-maximum suppression algorithm to determine a specified bounding box that meets preset conditions as the billet positioning result.

7. An image detection module, characterized in that, include: There are four spatial perception stages. Each spatial perception stage includes a convolutional block and an identity block. Each identity block includes three convolutional layers, one spatial structure perception module, and one stacking layer. The spatial structure perception module is used to obtain the feature map to be processed from the output of the first convolutional layer in the identity block and extract features to obtain the first feature map, the second feature map and the third feature map; The spatial structure perception module is further configured to calculate the first covariance feature map and the second covariance feature map corresponding to the feature map to be processed based on the first feature map and the second feature map respectively, and use the product of the first covariance feature map and the second covariance feature map as the initial feature map. The spatial structure perception module is also used to determine the spatial weight matrix corresponding to the feature map to be processed; The spatial structure perception module is further configured to determine a spatial attention feature map based on the spatial weight matrix and the initial feature map; Calculate the product of the spatial attention feature map and the third feature map, and add the product to the feature map to be processed to obtain the output feature map: The second convolutional layer of the identity block is used to extract features from the output feature map. The extracted features are then input into the third convolutional layer of the identity block for further feature extraction to obtain an enhanced feature map. The superposition layer of the identity blocks is used to add the enhanced feature map to the feature map to be processed element by element to obtain a spatially perceptual feature map; The overlay layer inputs the spatial perception feature map into the next spatial perception stage for further spatial feature extraction, until the overlay layer in the last spatial perception stage outputs the last spatial perception feature map, which is then used as the target feature map.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images