Image recognition method, device, electronic device and storage medium

Abstract

The present application provides an image recognition method, device, electronic device and storage medium. The method includes acquiring an image, inputting the image into a pre-built recognition model, determining initial features corresponding to the image through the recognition model; grouping the initial features according to a predetermined channel dimension, and performing feature extraction on each group of initial features to obtain intermediate features corresponding to each group of initial features, and fusing all intermediate features to obtain final features; based on the final features, outputting a recognition result corresponding to the image through the recognition model, thereby solving the technical problem of inaccurate image recognition in the prior art and achieving the purpose of accurately recognizing the image.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image recognition method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of deep learning and computer vision technology, traffic systems and autonomous driving have gradually become hot topics for research and application. Existing recognition algorithms based on deep learning have been widely used in the field of vehicle recognition, but they still face challenges when dealing with complex traffic scenes in the real world: First, due to the variability of vehicle shapes and scales, especially small target vehicles, the recognition accuracy of existing algorithms will be reduced; second, in bad weather, visibility is reduced, making it difficult for existing algorithms to locate and distinguish vehicles in images; third, the density and occlusion of congested traffic may mask the key features of vehicles, resulting in missed detections and false alarms. The above challenges make the accuracy of recognition algorithms based on deep learning relatively low in the field of vehicle recognition. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose an image recognition method, device, electronic device and storage medium to overcome all or part of the deficiencies in the prior art.

[0004] Based on the above purpose, the present application provides an image recognition method, including: acquiring an image, and inputting the image into a pre-built recognition model, determining the initial features corresponding to the image through the recognition model; grouping the initial features according to predetermined channel dimensions, and performing feature extraction on each group of initial features to obtain intermediate features corresponding to each group of initial features, and fusing all intermediate features to obtain final features; based on the final features, outputting the recognition result corresponding to the image through the recognition model.

[0005] Optionally, the recognition model includes a spatial attention mechanism, a channel attention mechanism and a cross-channel coordinated attention mechanism; determining the initial features corresponding to the image through the recognition model includes: extracting features from the image through the cross-channel coordinated attention mechanism to obtain a first sub-feature, extracting features from the image through the spatial attention mechanism to obtain a second sub-feature; extracting features from the image through the channel attention mechanism to obtain a third sub-feature; and fusing the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial features of the image.

[0006] Optionally, extracting features from the image by using the cross-channel coordinated attention mechanism to obtain a first sub-feature includes: determining the first sub-feature by using the following formula: Among them, y(p) is the first sub-feature, ψ is used to represent the cross-channel coordinated attention mechanism applied to the offset generated by the deformable convolution to optimize the sampling position, and w k is the weight of the kth position of the convolution kernel in the deformable convolution, p is the image, p k is the predetermined offset of the kth position of the convolution kernel in the deformable convolution, Δp k is the learnable offset of the kth position of the convolution kernel in the deformable convolution, Δm k is the modulation scalar of the kth position of the convolution kernel in the deformable convolution, and x() is used to represent the operation performed on the image.

[0007] Optionally, extracting features from the image by using the spatial attention mechanism to obtain a second sub-feature includes: determining the second sub-feature by using the following formula: in, is the second sub-feature, σ1 is the first Sigmoid activation function, is the image feature output in the vertical direction of the cth channel of the recognition model, is the weight of the image feature output in the vertical direction of the cth channel of the recognition model, is the image feature output in the horizontal direction of the cth channel of the recognition model, is the weight of the image feature output in the horizontal direction in the cth channel of the recognition model.

[0008] Optionally, extracting features from the image by using the channel attention mechanism to obtain a third sub-feature includes: determining the third sub-feature by using the following formula: Among them, g c is the third sub-feature, σ2 is the second Sigmoid activation function, H is the total height of the channel encoded in the vertical direction, W is the total width of the channel encoded in the horizontal direction, and x c (i,j) is the coordinate of the image, is the position weight of the image in the cth channel of the recognition model.

[0009] Optionally, the fusing the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature of the image includes: performing a product process on the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature.

[0010] Optionally, the recognition model includes a deformable convolution layer; the feature extraction for each group of initial features to obtain intermediate features corresponding to each group of initial features includes: for each group of initial features, randomly selecting one of the convolution kernels in all the convolution kernels in the deformable convolution layer to perform feature extraction on the group of initial features to obtain the intermediate features.

[0011] Based on the same inventive concept, the present application also provides an image recognition device, including: a determination module, configured to acquire an image, and input the image into a pre-built recognition model, and determine the initial features corresponding to the image through the recognition model; an extraction module, configured to group the initial features according to a predetermined channel dimension, and perform feature extraction on each group of initial features to obtain intermediate features corresponding to each group of initial features, and fuse all intermediate features to obtain final features; an output module, configured to output the recognition result corresponding to the image through the recognition model based on the final feature.

[0012] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0013] Based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method as described above.

[0014] From the above, it can be seen that the image recognition method, device, electronic device and storage medium provided by the present application, the method includes acquiring an image, and inputting the image into a pre-built recognition model, and determining the initial features corresponding to the image through the recognition model, so as to achieve the purpose of preliminarily and accurately determining the features of the image. The initial features are grouped according to a predetermined channel dimension, and feature extraction is performed on each group of initial features to obtain intermediate features corresponding to each group of initial features, and all intermediate features are fused to obtain final features, capturing multi-scale perceptual fields, thereby effectively improving the detection capability of multi-scale features, and achieving the purpose of accurately determining the final features of the image. Based on the final features, the recognition result corresponding to the image is output through the recognition model, so as to achieve the purpose of accurately recognizing the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic diagram of a flow chart of an image recognition method according to an embodiment of the present application;

[0017] Figure 2 A schematic diagram of the structure of an image recognition device according to an embodiment of the present application;

[0018] Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0020] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0021] As mentioned in the background technology section, with the rapid development of deep learning and computer vision technology, transportation systems and autonomous driving have gradually become hot topics in research and application. As one of the important foundations of transportation systems and autonomous driving, vehicle recognition algorithms have attracted much attention. When deploying vehicle recognition algorithms on edge devices with limited computing resources, accuracy and real-time performance become dual challenges. Therefore, deep learning-based algorithms with stronger robustness and accuracy are gradually replacing traditional algorithms and becoming the mainstream vehicle recognition algorithms.

[0022] Existing deep learning-based recognition algorithms have been widely used in the field of vehicle recognition, but they still face challenges when dealing with complex traffic scenes in the real world: First, due to the variability of vehicle shapes and scales, especially small target vehicles, the recognition accuracy of existing algorithms will be reduced; second, in bad weather, visibility is reduced, making it difficult for existing algorithms to locate and distinguish vehicles in images; third, the density and occlusion of congested traffic may mask the key features of vehicles, resulting in missed detections and false alarms. The above challenges make the accuracy of deep learning-based recognition algorithms in the field of vehicle recognition relatively low.

[0023] In view of this, the present application embodiment proposes an image recognition method, referring to Figure 1 , including the following steps:

[0024] Step 101 , acquiring an image, and inputting the image into a pre-built recognition model, and determining initial features corresponding to the image through the recognition model.

[0025] In this step, an image is acquired, and the image may be an image containing a vehicle. Accurately identifying the vehicle in the image is helpful for the development of traffic systems and autonomous driving. The image has multiple sources. For example, the image comes from an image capture of a vehicle traveling on an ordinary road or highway. The image is input into a pre-built recognition model, and features are initially extracted from the image. The recognition model can determine the initial features corresponding to the image, thereby achieving the purpose of initially and accurately determining the features of the image.

[0026] It should be noted that the recognition model can be obtained through data set training. When the data training set is a vehicle data set, the recognition model is a vehicle recognition model. The vehicle data set can be obtained by means of Python crawlers, surveillance cameras, existing data sets, etc. The images in the vehicle data set are photos of ordinary roads and / or highways under different lighting conditions, different weather conditions, and different scenes. It is also necessary to classify the vehicles in the image in advance. For example, they can be divided into four common categories: cars, buses, trucks and other vehicles. The annotation tool Labelimg is used to annotate the vehicle samples in all images with rectangular boxes, and the corresponding annotation labels are output in YOLO data format, wherein the YOLO data format refers to the data annotation format used for training and testing the recognition model in the YOLOv algorithm (You Only Look Once, an object recognition and positioning algorithm based on a deep neural network). Integrate the images and corresponding labels to create a vehicle data set, and divide the data set into a training set, a validation set, and a test set according to a predetermined ratio. For example, the data set is divided into a training set, a validation set, and a test set according to a ratio of 8:1:1. The obtained vehicle data set is subjected to data enhancement processing by the Mosaic data enhancement algorithm to improve the versatility of the data set. The Mosaic data enhancement algorithm is as follows: after scaling four or more images in a certain proportion, they are cropped and stitched together, and the coordinate position of the detection frame is adjusted to process the detection frame at the edge of the image. The advantage of the Mosaic data enhancement algorithm is that it enhances the diversity of data and the robustness of the model, and is beneficial to improving the detection performance of small targets, thereby enabling the trained recognition model to accurately identify relatively small targets to be identified in the image.

[0027] According to the specified training parameters, the improved YOLOv8 algorithm is iteratively trained, using BCE (binary cross entropy) as the classification loss. Each category needs to judge whether the image is of this category, directly output the "confidence score" of each category, and then find the maximum value, which is used as the "confidence" of this anchor box to measure the difference between the actual label and the model prediction. The classification loss function used to train the recognition model is determined by the following formula:

[0028]

[0029] Among them, BCELoss is the classification loss function, N is the number of image samples, y n is the true label, p n is the predicted probability of the recognition model for image sample n.

[0030] Use CIOU (Complete Intersection over Union, an indicator used to measure the degree of overlap between target boxes in object detection) as the regression loss. In the regression task, the regression degree of the box can be measured by the ratio of the "target box" to the "prediction box". CIOU loss is used to make the target box closer to the label value. The aspect ratio criterion is added to the (IOU) intersection over union loss to better fit the target box in three geometric parameters: overlapping area, center point distance, and aspect ratio. The calculation method is as follows:

[0031]

[0032] Among them, CIoU Loss is the first regression loss, IoU is the intersection over union of the ground truth and the predicted bounding box, d 2 is the Euclidean distance of the center point of the bounding box, R 2 is the area of ​​the ground-truth bounding box, T 2 is the area of ​​the predicted bounding box.

[0033] In order to cooperate with Anchor-Free and improve generalization, DFL (distributional feature loss, data average fidelity) loss is added to YOLOv8. DFL optimizes the probability of the left and right positions closest to the label y in the form of cross entropy, so that the network can focus on the target position and the distribution of the neighboring area more quickly. This loss of the position around the learning label can enhance the generalization of the model in complex situations, such as occlusion and moving objects. The calculation method is as follows:

[0034] DFL(S i ,S i+1 )=-((y i+1 -y i )log(S i )+(yy i )log(S i+1 )) Formula (3),

[0035] Among them, DFL(S i ,S i+1 ) is the second regression loss, y is the true value, y i+1 is the near-forecast value, y i is the predicted value.

[0036] In YOLOv8, the first regression loss is used in combination with the second regression loss, and the first regression loss can optimize the second regression loss. Through DFL, the loss of the "bounding box distribution probability" and the "distribution probability" of the label is calculated, so as to optimize each edge. The "bounding box distribution probability" is restored to the prediction box, and the loss of the prediction box and the "actual box" of the label is calculated through CIOU loss, so as to optimize the prediction box as a whole. The optimal model is continuously saved by verifying the accuracy. The early stopping mechanism is used to determine the final model. During the training process, if no significant improvement in model performance is observed within a certain number of rounds, the training will be stopped to obtain the final weight file, which can identify cars, buses, trucks, and other types of vehicles, and has high accuracy and speed in various scenarios.

[0037] Step 102, the initial features are grouped according to a predetermined channel dimension, and feature extraction is performed on each group of initial features to obtain intermediate features corresponding to each group of initial features, and all intermediate features are fused to obtain final features.

[0038] In this step, since the target to be identified in the image may be unclear or blocked, it is necessary to perform more detailed feature extraction on the initial features to obtain intermediate features. All intermediate features are fused to obtain the final features. In order to improve the recognition model's ability to detect targets to be identified in multi-scale images, a multi-scale grouped convolution (MGConv) is proposed. Multi-scale grouped convolution groups input features according to channels. The process can be expressed as:

[0039] z=Channel_Split(f) Formula (4),

[0040] Among them, Channel_Split(f) is located to indicate grouping of the initial features. Each group randomly selects different convolution kernel sizes for deep convolution to capture multi-scale receptive fields, thereby effectively improving the detection capability of multi-scale features. Subsequently, 1×1 point-by-point convolution is used to combine feature information from different channels at the same spatial position. Exemplarily, when the predetermined channel dimension is four, the process of taking four groups as an example is as follows:

[0041]

[0042] Among them, the group convolution kernel size of the detection head MGhead is {1×1, 3×3, 5×5, 7×7}. The detection head is used to predict the position (bounding box) and category of the target to be identified. Multi-scale group convolution combines depth convolution and point-by-point convolution, and reduces the number of parameters and computational cost required for convolution operations by separating the correlation of spatial and channel dimensions. It should be noted that in order to further reduce the number of parameters of the head in the recognition model, the concept of parameter sharing is borrowed to significantly reduce the volume of the model head by merging the main components of the classification branch and the regression branch. Specifically, the MGConv and CBS (consisting of Conv layer, BN layer and Silu layer) modules are combined as the detection head backbone of YOLOv8, and two ordinary convolutions are used as the classification branch and regression branch of the YOLOv8 detection head, respectively, to propose a new detection head MGhead.

[0043] By obtaining the final features and capturing the multi-scale receptive field, the detection capability of multi-scale features is effectively improved, thus achieving the purpose of accurately determining the final features of the image.

[0044] Step 103: Based on the final feature, output the recognition result corresponding to the image through the recognition model.

[0045] In this step, the final feature accurately and comprehensively reflects the features of the image. Even if the image is unclear or blocked, the final feature can be accurately determined by the recognition model, and then the recognition result corresponding to the final feature can be accurately determined by the recognition model, achieving the purpose of accurately recognizing the image. For example, if there is an obscured target to be identified in the image, and the target to be identified is a car, the recognition model can output the recognition result as a car; if there is an unclear target to be identified in the image, and the target to be identified is a truck, the recognition model can output the recognition result as a truck.

[0046] Through the above scheme, an image is acquired and input into a pre-built recognition model, and the initial features corresponding to the image are determined by the recognition model, thereby achieving the purpose of preliminarily and accurately determining the features of the image. The initial features are grouped according to the predetermined channel dimensions, and feature extraction is performed on each group of initial features to obtain the intermediate features corresponding to each group of initial features. All intermediate features are fused to obtain the final features, capturing the multi-scale perceptual field, thereby effectively improving the detection capability of multi-scale features, and achieving the purpose of accurately determining the final features of the image. Based on the final features, the recognition result corresponding to the image is output by the recognition model, thereby achieving the purpose of accurately recognizing the image.

[0047] In some embodiments, the recognition model includes a spatial attention mechanism, a channel attention mechanism and a cross-channel coordinated attention mechanism; determining the initial features corresponding to the image through the recognition model includes: extracting features from the image through the cross-channel coordinated attention mechanism to obtain a first sub-feature, extracting features from the image through the spatial attention mechanism to obtain a second sub-feature; extracting features from the image through the channel attention mechanism to obtain a third sub-feature; and fusing the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial features of the image.

[0048] In this embodiment, the geometric changes caused by factors such as the scale, posture, angle, (partial) deformation of the target to be identified in the image cause the accuracy of the ordinary recognition model to identify the target to be identified to be reduced. Therefore, the recognition model in this embodiment introduces deformable convolution to further enhance the adaptability of the recognition model to the deformed target to be identified. When using deformable convolution, the sampling position is no longer the standard rectangular area in conventional convolution, but a learnable offset and feature amplitude are added to adjust the sampling position to match the appearance characteristics of the target. In order to further refine the coverage of the sampling position of the deformable convolution, a cross-channel coordinated attention mechanism (Cross-channel Coordinate Attention, CCA attention mechanism) is introduced, combining the spatial attention mechanism and the channel attention mechanism, and the image is feature extracted respectively to obtain the first sub-feature, the second sub-feature and the third sub-feature. The attention mechanism has two main aspects: determining which part of the input needs to be paid attention to; allocating limited information processing resources to the important part, among which the cross-channel coordinated attention mechanism uses variance and mean as independent variables to generate attention, and the variance of each pixel in the image is solved and added to the mean, and then input into the multi-layer perceptron to finally obtain attention. When processing tasks, the spatial attention mechanism selects the most relevant areas for processing based on these importances, thereby improving the effect of the task. The channel attention mechanism is a special attention mechanism. Its main function is to enhance or suppress different channels for different tasks by modeling the importance of each feature channel. This mechanism can be understood as allocating attention to each channel of the neural network, thereby emphasizing or weakening different parts or features of the data. The representation of the target of interest is enhanced complementary from different angles, thereby optimizing the sampling position of the deformable convolution. The first sub-feature, the second sub-feature and the third sub-feature reflect the characteristics of the image from different angles. The initial features obtained by fusing the first sub-feature, the second sub-feature and the third sub-feature make the determined initial features more accurate.

[0049] In some embodiments, extracting features from the image through the cross-channel coordinated attention mechanism to obtain a first sub-feature includes: determining the first sub-feature through the following formula: Among them, y(p) is the first sub-feature, ψ is used to represent the cross-channel coordinated attention mechanism applied to the offset generated by the deformable convolution to optimize the sampling position, and w k is the weight of the kth position of the convolution kernel in the deformable convolution, p is the image, p k is the predetermined offset of the kth position of the convolution kernel in the deformable convolution, Δp k is the learnable offset of the kth position of the convolution kernel in the deformable convolution, Δm k is the modulation scalar of the kth position of the convolution kernel in the deformable convolution, and x() is used to represent the operation performed on the image.

[0050] In this embodiment, the deformable convolution DCN-CCA of the sampling position is optimized by the CCA attention mechanism. Given a convolution kernel with K sampling positions, the output result of the optimized deformable convolution is the first sub-feature. Through the CCA attention mechanism, the features of the targets to be identified of different geometric shapes in the image can be extracted, and even if the geometric shapes of the targets to be identified are different, the first sub-feature in the image can be accurately extracted. It should be noted that in the recognition model, the ordinary convolution is connected with the BN (Batch Normalization) module and the SiLU (Sigmoid Gated Linear Unit) function to form a CBS (composed of a Conv layer, a BN layer and a Silu layer) module, and the DCN (Deep & Cross Network)-CCA is connected with the BN module and the SiLU function to form a DBS module, and then the CBS module and the DBS module are combined to form a DC module. By replacing the Darkneck module in C2f with the DC module, the C2f-DC module is obtained, which is applied to the backbone network of the recognition model to enhance the network's ability to capture the features corresponding to the targets to be identified in images of different geometric shapes.

[0051] In some embodiments, extracting features from the image using the spatial attention mechanism to obtain a second sub-feature includes: determining the second sub-feature using the following formula: in, is the second sub-feature, σ1 is the first Sigmoid activation function, is the image feature output in the vertical direction of the cth channel of the recognition model, is the weight of the image feature output in the vertical direction of the cth channel of the recognition model, is the image feature output in the horizontal direction of the cth channel of the recognition model, is the weight of the image feature output in the horizontal direction in the cth channel of the recognition model.

[0052] In this embodiment, the image is feature extracted through the spatial attention mechanism. First, the coordinate-aware feature map of the spatial attention is generated. For a given input image X, two different pooling kernels (H, 1) or (1, W) are used to encode each channel along the horizontal coordinate and the vertical coordinate respectively. These two transformations enable the spatial attention mechanism to capture long-range dependencies along one spatial direction and retain precise position information along another spatial direction, which helps the network locate the object of interest more accurately. Therefore, the output of the cth channel at a height of h and a width of w can be expressed as:

[0053]

[0054] Next, a global receptive field is established and the precise location information is encoded, generating a pair of aggregated feature maps. In order to make full use of the rich representations contained therein, they are connected and then transformed through a shared convolution function Conv 3×1 Pass them. This function includes a 3×1 convolution, batch normalization, and SiLU activation function, and ensures that the input and output have the same shape. The output of the above process is as follows:

[0055] f=Conv 3×1 ([z h ,z w ]) Formula (8),

[0056] Among them, [·,·] represents the connection operation along the spatial dimension. The obtained feature map f is fed into the shared convolution transformation function Conv 1×1 , in order to model complex feature relationships. Then, the Sigmoid activation function is applied to map the output value of the convolution to the (0,1) range. The purpose of this step is to generate weights for adjusting the importance of positions, which can be expressed as:

[0057]

[0058] Among them, the generated intermediate coordinate feature map f and coordinate weight The shape is Then, f and f are transformed along the spatial dimension Split into two separate tensors, each with its corresponding weights: and The intermediate feature maps and weights are obtained by element-by-element multiplication in the horizontal and vertical directions, and the results are mapped to the (0,1) range through the Sigmoid function to obtain:

[0059]

[0060]

[0061] Among them, × represents element-by-element multiplication, and Feature weighted maps in the vertical and horizontal directions are generated respectively, so that the algorithm can better focus on the key spatial coordinates, thereby making the final second sub-feature more accurate.

[0062] In some embodiments, extracting features from the image using the channel attention mechanism to obtain a third sub-feature includes: determining the third sub-feature using the following formula: Among them, g c is the third sub-feature, σ2 is the second Sigmoid activation function, H is the total height of the channel encoded in the vertical direction, W is the total width of the channel encoded in the horizontal direction, and x c (i,j) is the coordinate of the image, is the position weight of the image in the cth channel of the recognition model.

[0063] In this embodiment, the image is feature extracted through the channel attention mechanism. Given an input image X, global average pooling and a convolution transformation function are first applied, which can be expressed as:

[0064]

[0065] The generated channel attention intermediate feature map f G ∈R C By calculating The average value of each channel of can be used to obtain the weight of channel attention, which is expressed as:

[0066] Among them, the generated channel attention weight The third sub-feature is obtained by element-wise multiplication of the intermediate feature map and the attention weight, and then mapping the result to the (0,1) range through the Sigmoid function: Through the channel attention mechanism, attention is allocated to each channel of the neural network, so as to emphasize or weaken different features of the data, making the final third sub-feature more accurate.

[0067] In some embodiments, the fusing of the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature of the image includes: performing a product process on the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature.

[0068] In this embodiment, the first sub-feature, the second sub-feature and the third sub-feature are multiplied so that the features extracted from different angles of the image can be merged to obtain the initial features. The initial features can better represent the features of the image, achieving the purpose of initially and accurately extracting features from the image.

[0069] In some embodiments, the recognition model includes a deformable convolution layer; the feature extraction of each group of initial features to obtain intermediate features corresponding to each group of initial features includes: for each group of initial features, randomly selecting one of the convolution kernels in all the convolution kernels in the deformable convolution layer to perform feature extraction on the group of initial features to obtain the intermediate features.

[0070] In this embodiment, in order to improve the recognition model's ability to detect targets to be identified in multi-scale images, the initial features are grouped for feature extraction, and each group randomly selects different convolution kernel sizes for deep convolution to capture the multi-scale receptive field, thereby effectively improving the detection ability of multi-scale features, and making the obtained intermediate features more accurate.

[0071] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0072] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0073] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an image recognition device.

[0074] refer to Figure 2 , the image recognition device comprises:

[0075] The determination module 10 is configured to acquire an image, input the image into a pre-built recognition model, and determine initial features corresponding to the image through the recognition model.

[0076] The extraction module 20 is configured to group the initial features according to a predetermined channel dimension, perform feature extraction on each group of initial features, obtain intermediate features corresponding to each group of initial features, and fuse all the intermediate features to obtain final features.

[0077] The output module 30 is configured to output the recognition result corresponding to the image through the recognition model based on the final feature.

[0078] Through the above-mentioned device, an image is acquired, and the image is input into a pre-built recognition model, and the initial features corresponding to the image are determined by the recognition model, thereby achieving the purpose of preliminarily and accurately determining the features of the image. The initial features are grouped according to the predetermined channel dimensions, and feature extraction is performed on each group of initial features to obtain the intermediate features corresponding to each group of initial features. All intermediate features are fused to obtain the final features, capturing the multi-scale perceptual field, thereby effectively improving the detection capability of multi-scale features, and achieving the purpose of accurately determining the final features of the image. Based on the final features, the recognition result corresponding to the image is output by the recognition model, thereby achieving the purpose of accurately recognizing the image.

[0079] In some embodiments, the determination module 10 is further configured to perform feature extraction on the image through the cross-channel coordinated attention mechanism to obtain a first sub-feature, perform feature extraction on the image through the spatial attention mechanism to obtain a second sub-feature; perform feature extraction on the image through the channel attention mechanism to obtain a third sub-feature; and fuse the first sub-feature, the second sub-feature and the third sub-feature to obtain an initial feature of the image.

[0080] In some embodiments, the determining module 10 is further configured to determine the first sub-feature by the following formula: Among them, y(p) is the first sub-feature, ψ is used to represent the cross-channel coordinated attention mechanism applied to the offset generated by the deformable convolution to optimize the sampling position, and w k is the weight of the kth position of the convolution kernel in the deformable convolution, p is the image, p k is the predetermined offset of the kth position of the convolution kernel in the deformable convolution, Δp k is the learnable offset of the kth position of the convolution kernel in the deformable convolution, Δm kis the modulation scalar of the kth position of the convolution kernel in the deformable convolution, and x() is used to represent the operation performed on the image.

[0081] In some embodiments, the determination module 10 is further configured to determine the second sub-feature by the following formula: in, is the second sub-feature, σ1 is the first Sigmoid activation function, is the image feature output in the vertical direction of the cth channel of the recognition model, is the weight of the image feature output in the vertical direction of the cth channel of the recognition model, is the image feature output in the horizontal direction of the cth channel of the recognition model, is the weight of the image feature output in the horizontal direction in the cth channel of the recognition model.

[0082] In some embodiments, the determination module 10 is further configured to determine the third sub-feature by the following formula: Among them, g c is the third sub-feature, σ2 is the second Sigmoid activation function, H is the total height of the channel encoded in the vertical direction, W is the total width of the channel encoded in the horizontal direction, and x c (i,j) is the coordinate of the image, is the position weight of the image in the cth channel of the recognition model.

[0083] In some embodiments, the determination module 10 is further configured to perform a product process on the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature.

[0084] In some embodiments, the extraction module 20 is further configured to randomly select one of the convolution kernels in all the convolution kernels in the deformable convolution layer for each group of initial features to extract features from the group of initial features to obtain the intermediate features.

[0085] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0086] The device of the above embodiment is used to implement the corresponding image recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0087] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image recognition method as described in any of the above embodiments is implemented.

[0088] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0089] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0090] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0091] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0092] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0093] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0094] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0095] The electronic device of the above embodiment is used to implement the corresponding image recognition method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0096] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image recognition method described in any of the above embodiments.

[0097] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0098] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image recognition method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0100] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0101] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0102] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for recognizing an image, characterized in that: include: Acquire an image, and input the image into a pre-built recognition model, and determine initial features corresponding to the image through the recognition model; The initial features are grouped according to a predetermined channel dimension, and feature extraction is performed on each group of initial features to obtain intermediate features corresponding to each group of initial features, and all the intermediate features are fused to obtain final features; Based on the final feature, outputting a recognition result corresponding to the image through the recognition model; The recognition model includes a spatial attention mechanism, a channel attention mechanism, and a cross-channel coordinated attention mechanism; Determining the initial features corresponding to the image by the recognition model includes: extracting features from the image by the cross-channel coordinated attention mechanism to obtain a first sub-feature, extracting features from the image by the spatial attention mechanism to obtain a second sub-feature; extracting features from the image by the channel attention mechanism to obtain a third sub-feature; fusing the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial features of the image; The extracting features of the image by the cross-channel coordinated attention mechanism to obtain a first sub-feature includes: determining the first sub-feature by the following formula: Among them, y(p) is the first sub-feature, ψ is used to represent the cross-channel coordinated attention mechanism applied to the offset generated by the deformable convolution to optimize the sampling position, and w k is the weight of the kth position of the convolution kernel in the deformable convolution, p is the image, p k is the predetermined offset of the kth position of the convolution kernel in the deformable convolution, Δp k is the learnable offset of the kth position of the convolution kernel in the deformable convolution, Δm k is the modulation scalar of the kth position of the convolution kernel in the deformable convolution, and x() is used to represent the operation performed on the image; The step of extracting features from the image by using the spatial attention mechanism to obtain a second sub-feature includes: determining the second sub-feature by using the following formula: in, is the second sub-feature, σ1 is the first Sigmoid activation function, is the image feature output in the vertical direction of the cth channel of the recognition model, is the weight of the image feature output in the vertical direction of the cth channel of the recognition model, is the image feature output in the horizontal direction of the cth channel of the recognition model, is the weight of the image feature output in the horizontal direction of the cth channel of the recognition model; The step of extracting features from the image by using the channel attention mechanism to obtain a third sub-feature includes: determining the third sub-feature by using the following formula: Among them, g c is the third sub-feature, σ2 is the second Sigmoid activation function, H is the total height of the channel encoded in the vertical direction, W is the total width of the channel encoded in the horizontal direction, and x c (i,j) is the coordinate of the image, is the position weight of the image in the cth channel of the recognition model; The fusing the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature of the image includes: performing a product process on the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature.

2. The method according to claim 1, characterized in that The recognition model includes a deformable convolutional layer; The extracting of features from each set of initial features to obtain intermediate features corresponding to each set of initial features includes: For each group of initial features, one of the convolution kernels in the deformable convolution layer is randomly selected to perform feature extraction on the group of initial features to obtain the intermediate features.

3. An image recognition device, characterized in that: include: A determination module is configured to acquire an image, input the image into a pre-built recognition model, and determine initial features corresponding to the image through the recognition model; An extraction module is configured to group the initial features according to a predetermined channel dimension, perform feature extraction on each group of initial features, obtain intermediate features corresponding to each group of initial features, and fuse all the intermediate features to obtain final features; an output module, configured to output a recognition result corresponding to the image through the recognition model based on the final feature; The determination module is further configured such that the recognition model includes a spatial attention mechanism, a channel attention mechanism, and a cross-channel coordinated attention mechanism; extracting features from the image through the cross-channel coordinated attention mechanism to obtain a first sub-feature, and extracting features from the image through the spatial attention mechanism to obtain a second sub-feature; Extracting features from the image through the channel attention mechanism to obtain a third sub-feature; fusing the first sub-feature, the second sub-feature and the third sub-feature to obtain an initial feature of the image; The determining module is further configured to determine the first sub-feature by using the following formula: Among them, y(p) is the first sub-feature, ψ is used to represent the cross-channel coordinated attention mechanism applied to the offset generated by the deformable convolution to optimize the sampling position, and w k is the weight of the kth position of the convolution kernel in the deformable convolution, p is the image, p k is the predetermined offset of the kth position of the convolution kernel in the deformable convolution, Δp k is the learnable offset of the kth position of the convolution kernel in the deformable convolution, Δm k is the modulation scalar of the kth position of the convolution kernel in the deformable convolution, and x() is used to represent the operation performed on the image; The determining module is further configured to determine the second sub-feature by using the following formula: in, is the second sub-feature, σ1 is the first Sigmoid activation function, is the image feature output in the vertical direction of the cth channel of the recognition model, is the weight of the image feature output in the vertical direction of the cth channel of the recognition model, is the image feature output in the horizontal direction of the cth channel of the recognition model, is the weight of the image feature output in the horizontal direction of the cth channel of the recognition model; The determining module is further configured to determine the third sub-feature by the following formula: Among them, g c is the third sub-feature, σ2 is the second Sigmoid activation function, H is the total height of the channel encoded in the vertical direction, W is the total width of the channel encoded in the horizontal direction, and x c (i,j) is the coordinate of the image, is the position weight of the image in the cth channel of the recognition model; The determination module is further configured to perform a product process on the first sub-feature, the second sub-feature and the third sub-feature to obtain the initial feature.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 2 is implemented.

5. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 2.

Images