A cross-scale feature extraction method of visual information

Abstract

The application provides a cross-scale feature extraction method of visual information, which fuses feature maps of different levels to generate a first feature map, a second feature map and a third feature map; the first feature map, the second feature map and the third feature map are fused by features of different levels, and feature information of different levels is retained; and when identification is further performed, more features can be identified, and identification accuracy is provided.

Description

Technical Field

[0001] This invention relates to a method for steel identification, specifically to a method for cross-scale feature extraction of visual information. Background Technology

[0002] Image recognition technology, also known as visual recognition technology, refers to the use of computers to process and analyze images, identify the category of objects, and make judgments. An image recognition system generally consists of three parts: preprocessing, analysis, and recognition. Preprocessing includes image segmentation, image enhancement, image restoration, image reconstruction, and image thinning, among other things. Image analysis mainly refers to extracting features from the preprocessed image. Finally, a classifier matches and classifies the image based on the extracted features to make a recognition.

[0003] Chinese patent application number 202010274297.6, published on June 30, 2020, discloses a steel code recognition method, a steel code recognition system, and its training method. The method involves real-time acquisition of images to be recognized using an image acquisition device, and processing each frame of the acquired images separately. When an object to be recognized is detected within a defined area of ​​the current frame of the image, the object and its number are identified using an object recognition model. The direction of the number in the image is obtained, and based on this direction, an image of the number oriented towards a reference direction is obtained. Based on this image, a character extraction image of the number oriented towards the recognition direction is obtained. The character corresponding to the number is extracted from the character extraction image. However, this recognition method cannot fuse features at different levels, resulting in fewer recognized features and inaccurate recognition. Summary of the Invention

[0004] This invention provides a cross-scale feature extraction method for visual information, which superimposes and fuses feature maps at different levels to retain features at different levels. As a result, during recognition, more features can be identified, thus improving recognition accuracy.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for cross-scale feature extraction of visual information, comprising the following steps:

[0006] S1. Input the preprocessed image into the YOLOv7-tiny model.

[0007] The S2 and YOLOv7-tiny models consist of an input terminal, a backbone, a TS-FAN module, and an output terminal. The input terminal receives data, and the output terminal outputs data. The backbone extracts features from the image.

[0008] The S3 and Backbone parts perform convolution operations on the extracted features to obtain four feature maps at different levels.

[0009] The S4 and TS-FAN modules fuse adjacent feature maps from different levels to obtain the first feature map, the second feature map, and the third feature map.

[0010] The above method extracts feature maps at different levels by performing convolution operations on the features in the backbone part, thereby obtaining feature information at different levels. Then, the TS-FAN module fuses adjacent feature maps at different levels, thus retaining more feature information in the first, second, and third feature maps. This allows for the capture of more feature information during the recognition process, improving the accuracy of image recognition.

[0011] Furthermore, in S4, fusing adjacent feature maps at different levels involves superimposing adjacent feature maps at different levels.

[0012] The above methods, through superposition and fusion, preserve features at different levels.

[0013] Furthermore, the Backbone component includes a first extraction module, a second extraction module, a third extraction module, and a fourth extraction module; the first extraction module includes a first CBS module and a first C3 module; the second extraction module includes a second CBS module and a second C3 module; the third extraction module includes a third CBS module and a third C3 module; the fourth extraction module includes a fourth CBS module and a fourth C3 module; the first C3 module and the second C3 module are C3-1 modules; the third C3 module and the fourth C3 module are C3-3 modules.

[0014] The above methods use different extraction modules to extract feature maps at different levels.

[0015] Furthermore, S3 specifically includes the following steps:

[0016] S3.1 First, the first extraction module extracts the features of the image and performs a convolution operation to obtain a first-level feature map.

[0017] S3.2. The second extraction module extracts features from the feature map of the first level and performs convolution operation to obtain the feature map of the second level.

[0018] S3.3. The third extraction module extracts features from the feature map of another level and performs convolution operation to obtain a feature map of yet another level.

[0019] S3.4. The features in the feature map of the next level are extracted by the fourth extraction module and convolution operation is performed to obtain the feature map of the next level.

[0020] The above method involves the first extraction module performing a convolution operation on the image to extract a first-level feature map; then the second extraction module performing a convolution operation on the first-level feature map to extract another level of feature map; then the third extraction module performing a convolution operation on the second-level feature map to extract yet another level of feature map; and then the fourth extraction module performing a convolution operation on yet another level of feature map to extract yet yet another level of feature map. The first, second, third, and fourth extraction modules sequentially extract features from the feature maps at different levels, thereby obtaining a higher-level feature map.

[0021] Furthermore, the TS-FAN module includes both top-down and bottom-up paths.

[0022] In S4, the TS-FAN module fuses feature maps from adjacent different levels, including the following steps:

[0023] S4.1, Overlay feature maps of one level and another level; S4.2, Overlay feature maps of one level, another level, and yet another level; S4.3, Overlay feature maps of another level, yet another level, and yet another level. S4.4.

[0024] S4.2. By using a top-down path, the spatial resolution of the high-level feature map is expanded, and the features in the high-level feature map are fused into the low-level feature map; thus, the first feature map is obtained.

[0025] S4.3. By using a top-down path, the spatial resolution of the high-level feature map is increased, while by using a bottom-up path, the spatial resolution of the low-level feature map is reduced, so that the high-level feature map, the mid-level feature map, and the low-high-level feature map are of the same size. The high-level feature map, the mid-level feature map, and the low-high-level feature map are then superimposed to obtain the second feature map.

[0026] S4.4. By using a top-down path, the spatial resolution of the high-level feature map is expanded, while by using a bottom-up path, the spatial resolution of the low-level feature map is reduced, so that the high-level feature map, the mid-level feature map, and the low-high-level feature map are of the same size. The high-level feature map, the mid-level feature map, and the low-high-level feature map are then superimposed to obtain the third feature map.

[0027] The above methods simultaneously perform top-down and bottom-up path fusion; merge feature information from different levels, scales, and channels to obtain richer and more accurate feature representations; improve feature representation capabilities; utilize multi-level, multi-scale, and multi-channel information in images to reduce feature loss; and enhance the model's feature extraction and generalization capabilities, thereby making the model more adaptable to different tasks and data.

[0028] Furthermore, the C3-3 module includes three residual modules, two CONV modules with a kernel size of 3*3 and a stride of 1, two CBS modules with a kernel size of 3*3 and a stride of 2, and a concat module.

[0029] Furthermore, the C3-1 module includes a residual module, two CONV modules with a kernel size of 3*3 and a stride of 1, two CBS modules with a kernel size of 3*3 and a stride of 2, and a concat module.

[0030] Furthermore, before step S1, the process includes: step S0, setting the size of the standard image; scaling the image so that the pre-processed image in the image is the same size as the standard image.

[0031] The above settings scale the images before recognition, ensuring that the size of the images to be recognized remains consistent across different images; reduce the impact of information other than the pre-processed images on the recognition results; and improve the accuracy of image recognition. Attached Figure Description

[0032] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, a method for cross-scale feature extraction of visual information includes the following steps:

[0035] S0: Preset the size of the standard image; scale the image so that the pre-processed image in the image is the same size as the standard image.

[0036] S1. Input the preprocessed image into the YOLOv7-tiny model.

[0037] The S2 and YOLOv7-tiny models include an input terminal, a backbone section, a TS-FAN module, and an output terminal; the input terminal is used to receive data, and the output terminal is used to output data; the backbone section collects features from images of steel billets.

[0038] The Backbone section includes a first extraction module, a second extraction module, a third extraction module, and a fourth extraction module. The first extraction module includes a first CBS module and a first C3 module. The second extraction module includes a second CBS module and a second C3 module. The third extraction module includes a third CBS module and a third C3 module. The fourth extraction module includes a fourth CBS module and a fourth C3 module. The first C3 module and the second C3 module are C3-1 modules. The third C3 module and the fourth C3 module are C3-3 modules.

[0039] The S3 and Backbone parts perform convolution operations on the extracted features to obtain four feature maps at different levels. These are feature maps at one level, another level, yet another level, and yet another level; each feature map at a different level contains feature information from a different level.

[0040] The S4 and TS-FAN modules fuse adjacent feature maps from different levels to obtain the first feature map, the second feature map, and the third feature map.

[0041] The Backbone module performs convolution operations on the features to extract feature maps at different levels, thus obtaining feature information at different levels. Then, the TS-FAN module fuses adjacent feature maps at different levels, thereby retaining more feature information in the first, second, and third feature maps. This allows for the capture of more feature information during the recognition process, improving the accuracy of image recognition.

[0042] The above S3 specifically includes the following steps:

[0043] S3.1 First, the first extraction module extracts the features of the image and performs a convolution operation to obtain a first-level feature map.

[0044] S3.2. The second extraction module extracts features from the feature map of the first level and performs convolution operation to obtain the feature map of the second level.

[0045] S3.3. The third extraction module extracts features from the feature map of another level and performs convolution operation to obtain a feature map of yet another level.

[0046] S3.4. The features in the feature map of the next level are extracted by the fourth extraction module and convolution operation is performed to obtain the feature map of the next level.

[0047] The above method involves the first extraction module performing a convolution operation on the image to extract a first-level feature map; then the second extraction module performing a convolution operation on the first-level feature map to extract another level of feature map; then the third extraction module performing a convolution operation on the second-level feature map to extract yet another level of feature map; and finally the fourth extraction module performing a convolution operation on yet another level of feature map to extract an even higher level of feature map. The first, second, third, and fourth extraction modules sequentially extract features from the feature maps at different levels, thereby obtaining higher-level feature maps.

[0048] In S4 above, the TS-FAN module includes a top-down path and a bottom-up path; the TS-FAN module fuses feature maps from adjacent different levels, including the following steps:

[0049] S4.1, Overlay feature maps of one level and another level; S4.2, Overlay feature maps of one level, another level, and yet another level; S4.3, Overlay feature maps of another level, yet another level, and yet another level. S4.4.

[0050] S4.2. By using a top-down path, the spatial resolution of the high-level feature map is expanded, and the features in the high-level feature map are fused into the low-level feature map; thus, the first feature map is obtained.

[0051] S4.3. By using a top-down path, the spatial resolution of the high-level feature map is increased, while by using a bottom-up path, the spatial resolution of the low-level feature map is reduced, so that the high-level feature map, the mid-level feature map, and the low-high-level feature map are of the same size. The high-level feature map, the mid-level feature map, and the low-high-level feature map are then superimposed to obtain the second feature map.

[0052] S4.4. By using a top-down path, the spatial resolution of the high-level feature map is expanded, while by using a bottom-up path, the spatial resolution of the low-level feature map is reduced, so that the high-level feature map, the mid-level feature map, and the low-high-level feature map are of the same size. The high-level feature map, the mid-level feature map, and the low-high-level feature map are then superimposed to obtain the third feature map.

[0053] Since the feature maps at different levels contain different feature information, the first, second, and third feature maps formed by superimposing these feature maps contain different feature information; thus, the model of the steel plate billet is determined by multiple feature information.

[0054] In this embodiment, the spatial resolution of the feature map at the next level is smaller than that of the feature map at the next level; the spatial resolution of the feature map at the next level is smaller than that of the feature map at another level; and the spatial resolution of the feature map at another level is smaller than that of the feature map at the first level. In S4.2, the feature map at the next level is reduced in spatial resolution and then merged with the feature map at the first level to obtain the first feature map; thus, the spatial resolution of the first feature map is the same as that of the feature map at the first level.

[0055] In S4.3, a second feature map is obtained by expanding the spatial resolution of the feature map at another level, reducing the spatial resolution of the feature map at one level, and merging it with the feature map at another level; thus, the spatial resolution of the first feature map is the same as that of the feature map at another level.

[0056] In S4.4, a third feature map is obtained by expanding the spatial resolution of the feature map at one level, reducing the spatial resolution of the feature map at another level, and merging it with the feature map at yet another level; thus, the spatial resolution of the third feature map is the same as that of the feature map at the next level.

[0057] In this way, the spatial resolution of the first feature image is greater than that of the second feature image, and the spatial resolution of the second feature image is greater than that of the third feature image. By setting up three cameras to identify the first, second, and third feature images, and by using different cameras to identify feature images with different spatial resolutions, the feature information in the feature images can be accurately identified, thereby accurately determining the model of the steel plate billet.

[0058] By simultaneously performing top-down and bottom-up path fusion, feature information from different levels, scales, and channels is merged to obtain richer and more accurate feature representations, thereby improving feature representation capabilities. Utilizing multi-level, multi-scale, and multi-channel information in the image reduces feature loss, enhancing the model's feature extraction and generalization abilities, thus making the model more adaptable to different tasks and data. Furthermore, by changing the spatial resolution of feature maps at different levels, feature maps from different levels can be superimposed.

Claims

1. A method for cross-scale feature extraction of visual information, characterized in that: Includes the following steps: S1. Input the preprocessed image into the YOLOv7-tiny model; The S2 and YOLOv7-tiny models consist of an input terminal, a backbone, a TS-FAN module, and an output terminal. The input terminal receives data, and the output terminal outputs data. The backbone extracts features from the image. The S3 and Backbone parts perform convolution operations on the extracted features to obtain four feature maps at different levels. S4. The TS-FAN module fuses adjacent feature maps from different levels, including the following steps: S4.

1. Superimposing feature maps from one level and feature maps from another level; S4.

2. Superimposing feature maps from one level, feature maps from another level, and feature maps from yet another level; S4.

3. Superimposing feature maps from another level, feature maps from yet another level, and feature maps from yet another level; S4.

4. S4.

2. By using a top-down path, the spatial resolution of the high-level feature map is expanded, and the features in the high-level features are fused into the low-level feature map; thus, the first feature map is obtained. S4.

3. By using a top-down path, the spatial resolution of the high-level feature map is increased, while by using a bottom-up path, the spatial resolution of the low-level feature map is reduced, so that the high-level feature map, the mid-level feature map, and the low-high-level feature map are of the same size. The high-level feature map, the mid-level feature map, and the low-high-level feature map are then superimposed to obtain the second feature map. S4.

4. By using a top-down path, the spatial resolution of the high-level feature map is expanded, while by using a bottom-up path, the spatial resolution of the low-level feature map is reduced, so that the high-level feature map, the mid-level feature map, and the low-high-level feature map are of the same size. The high-level feature map, the mid-level feature map, and the low-high-level feature map are then superimposed to obtain the third feature map.

2. The method for cross-scale feature extraction of visual information according to claim 1, characterized in that: In S4, fusing adjacent feature maps at different levels involves superimposing adjacent feature maps at different levels.

3. The method for cross-scale feature extraction of visual information according to claim 2, characterized in that: The Backbone section includes a first extraction module, a second extraction module, a third extraction module, and a fourth extraction module. The first extraction module includes a first CBS module and a first C3 module. The second extraction module includes a second CBS module and a second C3 module. The third extraction module includes a third CBS module and a third C3 module. The fourth extraction module includes a fourth CBS module and a fourth C3 module. The first C3 module and the second C3 module are C3-1 modules. The third C3 module and the fourth C3 module are C3-3 modules.

4. The method for cross-scale feature extraction of visual information according to claim 3, characterized in that: S3 specifically includes the following steps: S3.1 First, the features of the image are extracted by the first extraction module and convolution operation is performed to obtain a first-level feature map; S3.

2. The second extraction module extracts features from the feature map of the first level and performs convolution operation to obtain the feature map of the second level. S3.

3. The third extraction module extracts features from the feature map of another level and performs convolution operation to obtain a feature map of yet another level. S3.

4. The features in the feature map of the next level are extracted by the fourth extraction module and convolution operation is performed to obtain the feature map of the next level.

5. The method for cross-scale feature extraction of visual information according to claim 3, characterized in that: The C3-3 module includes three residual modules, two CONV modules with a kernel size of 3*3 and a stride of 1, two CBS modules with a kernel size of 3*3 and a stride of 2, and a concat module.

6. The method for cross-scale feature extraction of visual information according to claim 3, characterized in that: The C3-1 module includes a residual module, two CONV modules with a kernel size of 3*3 and a stride of 1, two CBS modules with a kernel size of 3*3 and a stride of 2, and a concat module.

7. The method for cross-scale feature extraction of visual information according to claim 1, characterized in that: Before step S1, the process also includes: step S0, setting the size of the standard image; scaling the image so that the pre-processed image in the image is the same size as the standard image.

Images