An image analysis method, system and electronic device applied to target detection

Abstract

Embodiments of the present disclosure provide an image analysis method, system and electronic device applied to target detection. Applied to the field of image processing, the method comprises: acquiring a to-be-detected picture, determining a high-resolution feature map and a low-resolution feature map of the to-be-detected picture; determining a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map; determining a key matrix containing relative position encoding of the high-resolution feature map and the low-resolution feature map; determining attention weighted features of the high-resolution feature map and the low-resolution feature map in local heads, global heads, channel feature heads and spatial feature heads; and determining a target detection result map. The present scheme can obtain multi-scale information and improve the detection performance of small targets. The relative position encoding of the feature points can improve the detection accuracy. The multi-head fusion attention mechanism can focus on the key features of small targets. Linear transformation can improve the response speed and accuracy of small target detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image processing, and particularly relates to an image analysis method and system applied to target detection and an electronic device. BACKGROUND

[0002] In modern technical fields such as intelligent transportation, security monitoring, and medical image analysis, target detection is a crucial image analysis task. The core of this technology lies in accurately identifying and locating target objects of interest from complex image data, whether they are vehicles, pedestrians, faces, or lesion regions in medical images.

[0003] A typical image recognition process includes feature extraction, candidate region generation, classification, and regression. First, a convolutional neural network is used to extract features from the input image to obtain a feature map. Then, a region proposal network generates a series of candidate regions that may contain targets. Next, fine feature extraction is performed on each candidate region, and a classifier is used to determine whether it contains a target and the target's class. Finally, a regressor adjusts the position and size of the candidate region to obtain the detection result.

[0004] However, today's image recognition process excels in handling large and medium-sized targets in ordinary resolution images, but when it comes to small targets in high-resolution images, the detection effect is not satisfactory. These small targets are often too small in size and not obvious in features, making it difficult for existing technologies to accurately capture and recognize them. SUMMARY

[0005] To address the shortcomings of existing technologies, the present disclosure provides an image analysis method, system, and device applied to target detection. The present disclosure solves the technical problem that existing technologies are difficult to accurately capture and recognize small targets in high-resolution images due to their small size and non-obvious features, resulting in poor detection results.

[0006] According to a first aspect of the present disclosure, an image analysis method applied to target detection is provided, comprising: obtaining a to-be-detected image, inputting the to-be-detected image into a preset image detection model to obtain a high-resolution feature map and a low-resolution feature map of the to-be-detected image; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points;

[0007] The high-resolution feature map and the low-resolution feature map are transformed according to a preset query weight matrix, a preset key weight matrix, and a preset value weight matrix by the preset image detection model to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map;

[0008] The relative position encodings of each feature point of the high-resolution feature map and the low-resolution feature map are calculated by a preset picture detection model, and the relative position encodings are added to the key matrix of the high-resolution feature map and the low-resolution feature map, so as to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position encodings.

[0009] The similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map are calculated by the local head, the global head, the channel feature head and the spatial feature head of the preset picture detection model according to a preset similarity score calculation formula, the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position encodings, and the similarity scores are multiplied with the value matrix of the high-resolution feature map and the low-resolution feature map, so as to obtain the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head of the high-resolution feature map and the low-resolution feature map.

[0010] The attention weighted features of the local head, the global head, the channel feature head and the spatial feature head are spliced by the preset picture detection model, so as to obtain a spliced feature map, and the spliced feature map is linearly transformed to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

[0011] According to a second aspect of the present disclosure, an image analysis system applied to target detection is provided for executing the method as described in the first aspect, comprising: a detection module for acquiring a picture to be detected, inputting the picture to be detected into a preset picture detection model to obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points;

[0012] A transformation module is configured to transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix by the preset picture detection model, so as to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map.

[0013] A key matrix processing module is configured to calculate the relative position encodings of each feature point of the high-resolution feature map and the low-resolution feature map by the preset picture detection model, and add the relative position encodings to the key matrix of the high-resolution feature map and the low-resolution feature map, so as to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position encodings.

[0014] The attention-weighted feature determination module is configured to calculate similarity scores of feature points in the high-resolution feature map and the low-resolution feature map respectively according to a preset similarity score calculation formula, a key matrix containing relative position encoding of the high-resolution feature map and the low-resolution feature map, and multiply the similarity scores with value matrices of the high-resolution feature map and the low-resolution feature map, to obtain attention-weighted features of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head.

[0015] The target detection result map determination module is configured to splice the attention-weighted features of the local head, the global head, the channel feature head and the spatial feature head by the preset picture detection model to obtain a spliced feature map, and perform linear transformation on the spliced feature map to obtain a target detection result map, wherein the target detection result map comprises target category information and target position bounding box information.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, which comprises a memory and a processor, the memory having a computer program stored thereon, and the processor implements the method as described above when executing the program.

[0017] In the image analysis method, system and device for target detection provided above, the model can obtain multi-scale information by combining the high-resolution feature map and the low-resolution feature map, thereby improving the detection performance of small targets. The detection accuracy can be improved by calculating the relative position encoding of the feature points and adding it to the key matrix. The attention mechanism of the local head, the global head, the channel feature head and the spatial feature head can better focus on the key features of small targets, thereby improving the detection effect of small targets. Linear transformation can improve the response speed and accuracy of small target detection. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 A flowchart of an image analysis method for target detection according to an embodiment of the present disclosure is shown;

[0020] Figure 2 A schematic block diagram of an image analysis system for target detection according to an embodiment of the present disclosure is shown;

[0021] Figure 3 A block diagram of an exemplary electronic device according to embodiments of the disclosure is shown. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of components and steps set forth in these embodiments are not limiting to the scope of the present disclosure unless specifically stated otherwise.

[0023] Those skilled in the art can understand that the terms "first", "second", and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they represent a logical sequence between them. It should also be understood that in the embodiments of the present disclosure, "multiple" can mean two or more, and "at least one" can mean one, two, or more. It should also be understood that for any component, data, or structure mentioned in the embodiments of the present disclosure, unless specifically limited or given the opposite implication by the context, it can be understood as one or more in general. In addition, the term "and / or" in the present disclosure is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects. It should also be understood that the description of various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, they will not be repeated.

[0024] Meanwhile, it should be understood that the sizes of the various portions shown in the drawings are not drawn in accordance with the actual proportional relationship for the sake of convenience in description. The following description of at least one exemplary embodiment is merely illustrative in nature and in no way limiting to the disclosure and its applications or uses. Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but where appropriate, the described techniques, methods, and devices should be considered as part of the specification. It should be noted that similar reference numbers and letters in the following drawings represent similar items, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0025] In order to make the objects, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure.

[0026] Figure 1 A flowchart of an image analysis method for target detection is provided for the embodiments of the present disclosure. The method of the embodiments of the present disclosure aims to achieve accurate detection of large and small targets in pictures.

[0027] S101, a picture to be detected is obtained, and the picture to be detected is input into a preset picture detection model to obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points.

[0028] The picture to be detected can be an image that needs to be processed for target detection. It can be any input image, such as a photo in a natural scene, a video frame, or other visual data containing objects.

[0029] The preset picture detection model can be a convolutional neural network architecture specifically designed for target detection tasks. For example, it can be a YOLOv5 model, which is a fast and efficient model capable of simultaneously predicting the class and bounding box position of multiple objects in a single forward propagation process. The preset picture detection model refers to a detection network that has been set up with specific structure and parameters, and YOLOv5 is an example of such a preset model.

[0030] The high-resolution feature map can be a feature extracted from a shallow network of the picture. Since fewer convolutional layers and less pooling operation are used, it retains more spatial details, i.e., the pixel resolution in the picture is higher. This feature map is usually used to detect smaller targets, because small targets occupy fewer pixels in the image, and the preservation of detailed information enables more accurate positioning and identification of these targets.

[0031] The low-resolution feature map can be a feature extracted from a deep network of the picture. Since more convolutional and pooling layers are used, the resolution of the feature map is reduced, but its features are more abstract and can capture more global or larger-scale semantic information of the object. This makes the low-resolution feature map more effective in large target detection, as it can generalize the overall shape and structure of the target.

[0032] The feature points can be key positions in the feature map, and each feature point contains information representing a certain region or position in the input picture. The feature points correspond to each pixel or cell in the feature map, and these feature points capture certain important attributes of objects in the image, such as edges, textures, or high-level semantic information.

[0033] The picture to be detected can be input into a preset picture detection model, for example, into a YOLOv5 model or a similar convolutional neural network. The Backbone network layer of the model is responsible for extracting feature maps of different levels from the input picture, obtaining high-resolution feature maps (for small target detection) and low-resolution feature maps (for large target detection) respectively. Specifically, the high-resolution feature maps are usually generated in the shallow network layer of the model (close to the input layer). Due to the small step size of the convolution operation and the small number of pooling layers, the feature maps have high spatial resolution and retain more details of the picture. This is very effective for detecting small targets. The low-resolution feature maps are generated in the deep network layer of the model (close to the output layer). After multiple convolution and pooling operations, the spatial resolution is low, but the semantic information is more abundant, and it is more suitable for detecting large targets. In each feature map (whether high-resolution or low-resolution), each pixel or unit can be regarded as a feature point. These feature points contain the local features of the input picture at that position, such as color, edge or object contour.

[0034] The training process of the preset picture detection model can include collecting an image dataset containing various target categories and labeled bounding boxes. Ensure that the dataset contains different scenes, angles, lighting conditions, etc. to improve the generalization ability of the model. Provide corresponding target category labels and position bounding box information for each image, which can be represented as (x min , y min, width, height) format representation. Backbone: Select a suitable convolutional neural network (such as ResNet, EfficientNet, etc.) as the basis for feature extraction, which can extract high-resolution and low-resolution feature maps. Design a channel attention module and a spatial attention module to enhance the feature representation capability of the feature map. Design a feature fusion head to process local, global, channel, and spatial features respectively, and obtain the corresponding attention scores through similarity calculation. Input the picture to be detected, and extract high-resolution feature maps and low-resolution feature maps through the Backbone of the model. Use the preset query weight matrix, key weight matrix, and value weight matrix to transform the feature maps to obtain the query matrix, key matrix, and value matrix. Calculate the relative position encoding of the feature points and add it to the key matrix to consider the relative position relationship between the feature points when calculating the attention. Use the preset similarity score calculation formula to calculate the similarity scores of the high-resolution feature maps and the low-resolution feature maps based on the key matrix containing the relative position encoding. Multiply the similarity scores with the value matrix to obtain the attention fusion features. Define appropriate loss functions, such as cross-entropy loss (for classification tasks) and bounding box regression loss (for position prediction), to optimize the output of the model. Use optimization algorithms such as Adam and SGD for backpropagation to update the model parameters until the loss converges. Evaluate the model performance on the validation set and monitor the model's performance on unseen data to adjust the hyperparameters. Use common evaluation metrics such as mAP (mean Average Precision), IoU (Intersection over Union), etc. to evaluate the detection performance of the model. Based on the evaluation results, further fine-tuning of the model, increasing data augmentation, or using transfer learning techniques may be needed to improve performance.

[0035] On the basis of the above technical solutions, the picture to be detected is input into a preset picture detection model to obtain high-resolution feature maps and low-resolution feature maps of the picture to be detected, including:

[0036] The picture to be detected is input into a preset picture detection model, and the Backbone layer of the preset picture detection model is used to perform convolution and pooling operations on the picture to be detected to obtain initial high-resolution feature maps and initial low-resolution feature maps.

[0037] The initial high-resolution feature maps and the initial low-resolution feature maps are processed by the channel attention component of the Attention Module layer of the preset picture detection model to obtain high-resolution channel attention maps and low-resolution channel attention maps.

[0038] The high-resolution channel attention map and the initial high-resolution feature map are multiplied, and the low-resolution channel attention map and the initial low-resolution feature map are multiplied to obtain an enhanced high-resolution feature map and an enhanced low-resolution feature map.

[0039] The enhanced high-resolution feature map and the enhanced low-resolution feature map are processed by the spatial attention component of the Attention Module layer of the preset picture detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map.

[0040] The enhanced high-resolution feature map and the initial high-resolution feature map are multiplied, and the enhanced low-resolution feature map and the initial low-resolution feature map are multiplied to obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected.

[0041] In this scheme, the Backbone layer can be the main feature extraction part in the preset picture detection model, usually composed of multiple convolutional layers and pooling layers. Its main role is to extract low-level and high-level features from the input image. Convolutional operation is used to detect local features (such as edges, textures, etc.), while pooling operation is used to reduce the spatial dimension of the feature map, keeping important information while reducing computational complexity.

[0042] The Attention Module layer can be a mechanism for the preset picture detection model to focus on important features. By calculating the weights of the input features, the model can pay more attention to key information when performing feature fusion. The Attention mechanism can be divided into different components, such as channel attention and spatial attention, for processing different aspects of features.

[0043] The channel attention component can emphasize important feature channels by calculating the weights of each channel. Different importance is assigned to different feature channels, so that the model can focus on more significant features during feature extraction. For example, channel descriptions can be obtained through global average pooling and global maximum pooling, then channel weights can be generated through a small neural network (such as MLP), and finally the weights can be applied to the original feature map.

[0044] The high-resolution channel attention map can be a feature map generated by the channel attention component, aiming to highlight the importance of each channel in the image. It generates a weight distribution by analyzing the feature response of each channel, emphasizing those channels that are more important for target detection.

[0045] The low-resolution channel attention map is similar to the high-resolution channel attention map, but it is generated for the low-resolution feature map. This map emphasizes key channels in the low-resolution feature map through the channel attention mechanism, aiming to preserve important contextual information and enhance the ability to detect large targets or overall features.

[0046] The enhanced high-resolution feature map can be the result of multiplying the high-resolution channel attention map with the initial high-resolution feature map. This process enhances the high-resolution feature map on key features, improving the model's ability to detect small targets or important local features.

[0047] The enhanced low-resolution feature map can be obtained by multiplying the low-resolution channel attention map with the initial low-resolution feature map. This map provides enhanced contextual information, suitable for detecting large targets or overall features, thus playing a role in the feature fusion stage.

[0048] The high-resolution spatial attention map can be obtained by processing the enhanced high-resolution feature map through the spatial attention component. The spatial attention mechanism focuses on the spatial distribution of the feature map, emphasizing the most relevant areas in the image, thus helping the model better understand the spatial structure and important positions of the image.

[0049] The low-resolution spatial attention map can be similar to the high-resolution spatial attention map, but it is generated based on the enhanced low-resolution feature map. This map emphasizes important spatial areas in the low-resolution feature map, suitable for large target detection and overall image understanding, improving the model's response ability to global features.

[0050] The to-be-detected picture can pass through the Backbone layer of the model and be subjected to feature extraction through multiple convolution layers. Each convolution layer extracts features at different levels, including edge, texture and other information. After convolution operation, pooling (such as maximum pooling or average pooling) is usually performed to reduce the spatial dimension of the feature map and enhance the robustness of the model to small deformation and position change. After convolution and pooling, two feature maps are obtained: an initial high-resolution feature map (retaining more spatial information, suitable for detecting small targets) and an initial low-resolution feature map (containing more abstract features, suitable for detecting large targets). Then the initial high-resolution feature map and the initial low-resolution feature map are processed to generate corresponding attention maps through a channel attention mechanism: a high-resolution channel attention map that emphasizes important channels in the high-resolution feature map, and a low-resolution channel attention map that emphasizes important channels in the low-resolution feature map. Then the high-resolution channel attention map is multiplied element by element with the initial high-resolution feature map to obtain an enhanced high-resolution feature map. This process strengthens the important features in the feature map. Similarly, the low-resolution channel attention map is multiplied element by element with the initial low-resolution feature map to obtain an enhanced low-resolution feature map. Then the enhanced high-resolution feature map and the low-resolution feature map are processed to generate spatial attention maps: a high-resolution spatial attention map that emphasizes important spatial regions in the high-resolution feature map, and a low-resolution spatial attention map that emphasizes important spatial regions in the low-resolution feature map. Finally, the enhanced high-resolution feature map is multiplied element by element with the initial high-resolution feature map to obtain a final high-resolution feature map. This process combines the initial features with the features strengthened by the attention mechanism. Similarly, the enhanced low-resolution feature map is multiplied element by element with the initial low-resolution feature map to obtain a final low-resolution feature map.

[0051] In the present solution, important information of different scales and features can be effectively extracted and enhanced, improving the accuracy and robustness of target detection.

[0052] On the basis of the above technical solution, optionally, the initial high-resolution feature map and the initial low-resolution feature map are processed through a channel attention component of an AttentionModule layer of a preset picture detection model to obtain a high-resolution channel attention map and a low-resolution channel attention map, including:

[0053] The channel attention component of the AttentionModule layer of the preset picture detection model is used to perform global maximum pooling and global average pooling on each channel of the initial high-resolution feature map and the initial low-resolution feature map respectively to obtain an initial high-resolution one-dimensional feature map and an initial low-resolution one-dimensional feature map; wherein the initial high-resolution one-dimensional feature map and the initial low-resolution one-dimensional feature map both contain global maximum pooling values and global average pooling values of each channel.

[0054] The global maximum pooling value and the global average pooling value of the initial high-resolution one-dimensional feature map are summed to obtain a first sum result, and the first sum result is normalized by a Sigmoid activation function to obtain a high-resolution channel attention map.

[0055] The global maximum pooling value and the global average pooling value of the initial low-resolution one-dimensional feature map are summed to obtain a second sum result, and the second sum result is normalized by a Sigmoid activation function to obtain a low-resolution channel attention map.

[0056] In this scheme, the initial high-resolution one-dimensional feature map can be a feature vector obtained by performing global maximum pooling and global average pooling operations on each channel of the initial high-resolution feature map. This one-dimensional feature map represents the significance of each channel in the overall feature map.

[0057] The initial low-resolution one-dimensional feature map can be a one-dimensional feature map obtained by performing the same pooling operation on each channel of the initial low-resolution feature map.

[0058] The global maximum pooling value can be the maximum value of the channel feature map in a certain channel, representing the most significant feature of the channel.

[0059] The global average pooling value can be the average value of all pixel values in a certain channel, representing the overall feature intensity of the channel.

[0060] The first sum result can be the sum of the global maximum pooling value and the global average pooling value in the initial high-resolution one-dimensional feature map, and the result reflects the importance of the channel in feature extraction.

[0061] The Sigmoid activation function can be a commonly used activation function, whose output range is between 0 and 1, and can be represented by the formula . It normalizes the first sum result for subsequent use.

[0062] The second sum result can be the sum of the global maximum pooling value and the global average pooling value in the initial low-resolution one-dimensional feature map, and the result represents the overall significance of the low-resolution feature channel.

[0063] The maximum pooling can be performed on each channel of the initial high-resolution feature map and the initial low-resolution feature map, the maximum value in each channel is selected to generate a maximum pooling part of the high-resolution and low-resolution one-dimensional feature map. The average pooling is performed on the same feature map, the average value of each channel is calculated to generate an average pooling part of the one-dimensional feature map. Then, the global maximum pooling value and the global average pooling value of each channel of the high-resolution feature map are combined to form a one-dimensional feature map. Similarly, the pooling values of each channel of the low-resolution feature map are combined to form another one-dimensional feature map. The global maximum pooling value and the global average pooling value in the initial high-resolution one-dimensional feature map are added to obtain a first summation result. The Sigmoid activation function is applied to the first summation result for normalization to obtain a high-resolution channel attention map, which ranges between 0 and 1. The global maximum pooling value and the global average pooling value in the initial low-resolution one-dimensional feature map are added to obtain a second summation result. The Sigmoid activation function is also applied to the second summation result for normalization to obtain a low-resolution channel attention map.

[0064] In the scheme, the channel attention component can effectively extract the significance of each channel in the feature map, thereby improving the attention of the model to important features and further improving the effect of target detection.

[0065] On the basis of the above technical scheme, optionally, the enhanced high-resolution feature map and the enhanced low-resolution feature map are processed by the spatial attention component of the AttentionModule layer of the preset picture detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map, including:

[0066] The spatial attention component of the AttentionModule layer of the preset picture detection model performs global maximum pooling and global average pooling on the pixel values at the same position of the enhanced high-resolution feature map to obtain a first spatial maximum pooling feature map and a first spatial average pooling feature map;

[0067] The spatial attention component of the AttentionModule layer of the preset picture detection model performs global maximum pooling and global average pooling on the pixel values at the same position of the enhanced low-resolution feature map to obtain a second spatial maximum pooling feature map and a second spatial average pooling feature map;

[0068] The first spatial maximum pooling feature map and the first spatial average pooling feature map are spliced to obtain a first spliced spatial feature map, and the second spatial maximum pooling feature map and the second spatial average pooling feature map are spliced to obtain a second spliced spatial feature map;

[0069] The first spliced spatial feature map and the second spliced spatial feature map are subjected to convolution operation and Sigmoid activation to obtain a high-resolution spatial attention map and a low-resolution spatial attention map.

[0070] In this scheme, the first spatial maximum pooling feature map can be a feature map obtained by performing global maximum pooling on each spatial position of the enhanced high-resolution feature map. It retains the maximum pixel value of each spatial position, highlights the most significant features, and helps the model focus on key areas.

[0071] The first spatial average pooling feature map can be a feature map obtained by performing global average pooling on each spatial position of the enhanced high-resolution feature map. It calculates the average value of each spatial position, providing distribution information of the overall features.

[0072] The second spatial maximum pooling feature map is similar to the first spatial maximum pooling feature map, but it is the result of the enhanced low-resolution feature map. It retains the maximum pixel value of each spatial position in the low-resolution feature map.

[0073] The second spatial average pooling feature map is also the average pooling result of the enhanced low-resolution feature map, reflecting the average pixel value of each spatial position.

[0074] The first spliced spatial feature map can be a feature map obtained by splicing the first spatial maximum pooling feature map and the first spatial average pooling feature map. After splicing, the feature map contains the maximum and average information of the high-resolution features, providing more rich context information for subsequent processing.

[0075] The second spliced spatial feature map can be a feature map obtained by splicing the second spatial maximum pooling feature map and the second spatial average pooling feature map, retaining the maximum and average information of the low-resolution features.

[0076] The enhanced high-resolution feature map can be globally maximum-pooled, each channel is traversed, the maximum value of all pixels in the channel is extracted, and a first spatial maximum-pooled feature map is obtained. The first spatial maximum-pooled feature map is globally average-pooled, the average value of all pixels in each channel is calculated, and a first spatial average-pooled feature map is obtained. The enhanced low-resolution feature map is globally maximum-pooled, and a second spatial maximum-pooled feature map is obtained. The second spatial maximum-pooled feature map is globally average-pooled, and a second spatial average-pooled feature map is obtained. The first spatial maximum-pooled feature map and the first spatial average-pooled feature map are spliced, and a first spliced spatial feature map is obtained. This step combines different pooling information and enhances the representation ability of the features. The second spatial maximum-pooled feature map and the second spatial average-pooled feature map are spliced, and a second spliced spatial feature map is obtained. The first spliced spatial feature map and the second spliced spatial feature map are subjected to convolution operation, and higher-level features are extracted. Then, the features after convolution are activated through a Sigmoid activation function, and a high-resolution spatial attention map and a low-resolution spatial attention map are obtained.

[0077] In this scheme, the spatial attention mechanism is applied to the enhanced feature map, which can effectively highlight important spatial regions and provide more accurate information for subsequent target detection tasks.

[0078] In S102, the high-resolution feature map and the low-resolution feature map are transformed according to a preset query weight matrix, a preset key weight matrix, and a preset value weight matrix through a preset picture detection model, to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map.

[0079] The preset query weight matrix can be used for linear transformation of information of the input feature map (high-resolution and low-resolution feature map), thereby generating a query vector. Matrix multiplication can be performed on the feature map to generate a query matrix for querying related information of feature points. It is usually a learnable weight matrix, and the shape is determined by the dimensions of the input feature map and the dimensions of the output query vector.

[0080] The preset key weight matrix can be used for linear transformation of information in the input feature map into a key vector. The correlation between feature points can be represented by the key vector, so as to calculate the similarity in cooperation with the query matrix. It is also a learnable weight matrix, and the shape is determined by the dimensions of the input feature map and the key matrix.

[0081] The preset value weight matrix can be used for linear transformation of the input feature map into a value vector. A value matrix for representing the feature map can be generated, and the final attention score will act on these value matrices to obtain a fused feature map. It is a learnable weight matrix, and the shape is determined by the dimensions of the input feature map and the dimensions of the output value matrix.

[0082] The query matrix can be a matrix obtained by multiplying the high-resolution or low-resolution feature map with a query weight matrix. It can be used to calculate the similarity between feature points by dot product with the key matrix. The query vector of each feature point represents the importance of the point among other feature points.

[0083] The key matrix can be a matrix obtained by multiplying the feature map with a key weight matrix. It can be used to calculate the relevance or similarity between feature points together with the query matrix. The key vector of each feature point describes its relationship with other feature points.

[0084] The value matrix can be a matrix obtained by multiplying the feature map with a value weight matrix. The value matrix stores the value of each feature point in the feature map, which is output after being weighted by attention weight. The final output is the weighted sum of these value matrices.

[0085] The query matrix, key matrix, and value matrix can be generated by applying linear transformation operations of the preset query weight matrix, preset key weight matrix, and preset value weight matrix to the high-resolution feature map and low-resolution feature map in the Attention module of the picture detection model, respectively. Then the position of each feature point can be encoded and these encoded values can be added to the key matrix. These relative position encodings help the Attention mechanism better understand the spatial relationships between feature points. For each input feature map (high-resolution and low-resolution feature map), query, key, and value calculations can also be performed separately. This operation is done independently for each resolution of the feature map, but when the final attention is fused, these matrices will be combined.

[0086] On the basis of the above technical solutions, the high-resolution feature map and the low-resolution feature map can be transformed according to the preset query weight matrix, the preset key weight matrix, and the preset value weight matrix by the preset picture detection model to obtain the query matrix, the key matrix, and the value matrix of the high-resolution feature map and the low-resolution feature map, including:

[0087] A preset transformation formula is obtained, and the high-resolution feature map and the low-resolution feature map are transformed according to the preset query weight matrix, the preset key weight matrix, the preset value weight matrix, and the preset transformation formula by the preset picture detection model to obtain the query matrix, the key matrix, and the value matrix of the high-resolution feature map and the low-resolution feature map; wherein the preset transformation formula is:

[0088] Q = F·W Q ;

[0089] K = F·W K ;

[0090] V = F·W V ;

[0091] wherein F is a high-resolution feature map / low-resolution feature map; W Q is a preset query weight matrix; W K is a preset key weight matrix; W V is a preset value weight matrix; Q is a query matrix of a high-resolution feature map / low-resolution feature map; K is a key matrix of a high-resolution feature map / low-resolution feature map; and V is a value matrix of a high-resolution feature map / low-resolution feature map.

[0092] In this scheme, the high-resolution feature map and the low-resolution feature map are input feature maps, which can be two-dimensional arrays (tensors) containing multiple channels. Multiplying the image (feature map) with the weight matrix is usually implemented in a deep learning framework (such as TensorFlow, PyTorch, etc.). The feature map is treated as a multi-dimensional tensor, while the weight matrix is a two-dimensional array. The multiplication operation is performed according to the rules of matrix multiplication, ensuring dimension matching to obtain the corresponding query, key and value matrices.

[0093] In this scheme, after the linear transformation of the feature map by the weight matrix, the respective query, key and value information can be effectively extracted, providing a basis for subsequent attention mechanism calculation.

[0094] S103, calculate the relative position encodings of each feature point of the high-resolution feature map and the low-resolution feature map by a preset picture detection model, and add the relative position encodings to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position encodings.

[0095] The relative position encoding can be a technique for adding position information in the Attention mechanism, and the main purpose is to make the model not only pay attention to the value of the feature point, but also pay attention to the relative spatial relationship between the feature points. In the image detection task, the feature points (i.e. the features of the image pixels or regions) not only have their feature values, but also have their positions. In the Attention mechanism, adding relative position encoding can help the model better capture the relative spatial position information between these feature points. Traditional Attention mechanism only calculates similarity through the query matrix and the key matrix of the feature points, without directly considering the position information between the feature points. The relative position encoding can enhance the spatial perception ability of the Attention mechanism, so that it can capture the relative position relationship between each feature point when processing high-resolution and low-resolution feature maps.

[0096] The relative position encoding can be generated for each feature point according to its relative position. Specifically, the relative position encoding can be generated based on spatial offset. The relative position between feature points can be represented by the distance in a two-dimensional plane (e.g., the distance in horizontal and vertical directions). The relative position information of each feature point is defined, for example, the row difference and column difference relative to other feature points. The model can also be trained to automatically learn a set of relative position encoding matrices suitable for the current task. In this way, the values of the position encoding are gradually adjusted during the training process through optimization algorithms such as gradient descent. After generating the relative position encoding, it is added to the key matrix of the high-resolution feature map and the low-resolution feature map. The key matrix is obtained by linear transformation and is used to represent the correlation between feature points. By adding the relative position encoding to the key matrix, the model can capture the position information between feature points. The high-resolution feature map and the low-resolution feature map can be processed separately, and the key matrix of each of them is added with the relative position encoding in the same way. Because the two feature maps capture different levels of image information (high resolution for small targets and low resolution for large targets), they need to be processed separately. By adding the relative position encoding, the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position encoding is obtained. These matrices can be used for Attention calculation next, helping the model better understand the spatial relationship between feature points.

[0097] On the basis of the above technical solutions, optionally, the relative position encoding of each feature point of the high-resolution feature map and the low-resolution feature map is calculated through a preset picture detection model, and the relative position encoding is added to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position encoding, comprising:

[0098] A preset relative position encoding calculation formula is obtained, and the coordinate information of each feature point of the high-resolution feature map and the low-resolution feature map is determined.

[0099] The relative position encoding of each feature point of the high-resolution feature map and the low-resolution feature map is calculated through a preset picture detection model according to the preset relative position encoding calculation formula, the coordinate information of each feature point of the high-resolution feature map and the low-resolution feature map, wherein the preset relative position encoding calculation formula is:

[0100] R(i, j) = f(p i -p j );

[0101] wherein R(i, j) is the relative position encoding of the i-th feature point and the j-th feature point of the high-resolution feature map / low-resolution feature map; p iis the coordinate information of the i-th feature point of the high-resolution feature map / low-resolution feature map; p j is the coordinate information of the j-th feature point of the high-resolution feature map / low-resolution feature map; f is a function for calculating relative position encoding;

[0102] The relative position encodings of the high-resolution feature map and the low-resolution feature map are added to the key matrix to obtain a key matrix containing relative position encodings of the high-resolution feature map and the low-resolution feature map.

[0103] In this scheme, the coordinate information can refer to the position of each feature point in the feature map. For the high-resolution feature map and the low-resolution feature map, the coordinates of each feature point are usually represented in two dimensions, such as (i, j), where i represents the row coordinate and j represents the column coordinate.

[0104] For the high-resolution feature map and the low-resolution feature map, the coordinate information of the feature points can be represented as all possible combinations of row and column coordinates, and for each pair of feature points (p i , p j ), the coordinate difference p i -p j is calculated, a function for calculating relative position encoding is applied to obtain the relative position encoding of each feature point, and then the calculated relative position encoding is directly added to the corresponding element in the key matrix to form a new key matrix K'. This step enhances the sensitivity of the model to the relative positions between feature points.

[0105] In this scheme, the relative position encodings of the feature points can be calculated from the coordinate information of the feature map and added to the key matrix, thereby enhancing the understanding of the model to the spatial relationship.

[0106] S104, according to a preset similarity score calculation formula, a high-resolution feature map and a low-resolution feature map containing relative position encodings of the key matrix, the similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map are calculated by the local head, the global head, the channel feature head and the spatial feature head of the preset picture detection model, and the similarity scores are multiplied by the value matrix of the high-resolution feature map and the low-resolution feature map to obtain the attention fusion feature map of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head.

[0107] The local head, global head, channel feature head, and spatial feature head are different focal points in the multi-head attention mechanism. Each head represents information extraction from different angles of input features, focusing on specific image features or spatial relationships. Among them, the local head can focus on local feature extraction, mainly focusing on adjacent or close feature points, and is usually used for detecting details or small objects. This is very useful for processing high-resolution feature maps.

[0108] The global head can focus on global information extraction, capturing the relationship between distant feature points in the image, helping to understand the global context information. It is particularly useful for low-resolution feature maps and is used for detecting large targets or global context.

[0109] The channel feature head can focus on the channel dimension of each feature map, mainly used to capture the relationship between different channels (usually related to color or other feature dimensions) in the feature map.

[0110] The spatial feature head can focus on the relationship in the image spatial position, capturing the relative position and relationship of each feature point in the two-dimensional space, and is particularly useful for extracting spatial structure information.

[0111] The similarity score calculation formula can be a mathematical formula for measuring the correlation (or similarity) between feature points.

[0112] The similarity score can be a measure of the similarity between different feature points, representing the strength of the association between feature points. After calculating the similarity score, the model can adjust the importance of each feature point based on these scores, and then use this information in the target detection task. Similarity scores are usually used to weight other feature matrix (such as value matrix) operations to highlight important feature points. Specifically, a high similarity score indicates that the relationship between two feature points is strong. A low similarity score indicates that the relationship between two feature points is weak.

[0113] Attention weighted features can be a feature map obtained by calculating the similarity score between feature points (the core idea of attention mechanism), and then weighting and combining the similarity score with the feature value in the feature map. This weighting method can highlight important features related to the target while suppressing irrelevant or unimportant features.

[0114] The key matrix containing relative position encoding of the high-resolution feature map and the low-resolution feature map can be substituted into a preset similarity score calculation formula in the local head, the global head, the channel feature head and the spatial feature head, and the similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map are independently calculated. Specifically, although the similarity score calculation formula of each head is the same, the input features and the attention points of each head are different, and therefore it is necessary to calculate the similarity respectively. The calculated similarity scores are used to weight the features in the value matrix. The higher the similarity, the closer the relationship between the query feature and the key feature, and therefore the greater the weight of the feature. Then, the matrix multiplication of the similarity score matrix and the value matrix is independently performed in each head to obtain the weighted feature values. Through the multiplication of the similarity score matrix and the value matrix, the final output is the attention weighted features of each feature point in the local, global, channel and spatial dimensions. These features reflect the attention distribution of different heads on the input image features. Specifically, for different heads (local head, global head, channel feature head and spatial feature head), the attention weighted features under each head are calculated respectively. Different heads focus on different aspects of information, in which the local head focuses on the similarity in the local region. The global head focuses on the relationship between the feature points in the whole image range. The channel feature head focuses on the similarity between different feature channels. The spatial feature head focuses on the relationship between the spatial positions.

[0115] On the basis of the above technical solutions, optionally, the preset similarity score calculation formula is:

[0116]

[0117] wherein A(Q, K', V) is the similarity score of each feature point in the high-resolution feature map / low-resolution feature map; Q is the query matrix of the high-resolution feature map / low-resolution feature map; K' is the key matrix containing relative position encoding of the high-resolution feature map / low-resolution feature map; K' T is the transpose of the key matrix containing relative position encoding of the high-resolution feature map / low-resolution feature map; V is the value matrix of the high-resolution feature map / low-resolution feature map; dk is the dimension of the key matrix of the high-resolution feature map / low-resolution feature map; QK' T is the similarity score matrix, which is the dot product of the query matrix of the high-resolution feature map / low-resolution feature map and the transpose of the key matrix containing relative position encoding.

[0118] In the present scheme, the key matrix containing relative position encoding of the high-resolution feature map and the low-resolution feature map, the corresponding query matrix and the value matrix can be prepared, and then K' T (the transpose of the key matrix) is calculated, and the matrix multiplication QK'T is performed, and the similarity score matrix is divided by The scaling is performed to maintain numerical stability, and a softmax function is applied to the similarity score matrix to obtain the attention weight of each feature point. The softmax normalizes the similarity score of each feature point into a probability distribution. Finally, the attention weight is multiplied with the value matrix V to obtain the weighted feature.

[0119] In S105, the attention fusion feature maps of the local head, the global head, the channel feature head, and the spatial feature head are spliced by a preset picture detection model to obtain a spliced feature map, and the spliced feature map is linearly transformed to obtain a target detection result map; wherein the target detection result map includes target class information and target position bounding box information.

[0120] The spliced feature map can be a unified feature map formed by splicing different types of attention weighted features (local head, global head, channel feature head, and spatial feature head) along a certain dimension (usually the feature channel dimension). Each head focuses on different aspects of features, so their weighted features can be complementary. By splicing, these information is fused together to obtain a comprehensive feature map containing more dimensional and hierarchical information.

[0121] The target detection result map can be a map containing target object class prediction information and position bounding box information by linearly transforming (usually full connection layer operation) the spliced feature map.

[0122] The target class information can represent the class predicted by the model for each detected object in the image. For example, there can be multiple objects (such as cars, pedestrians, buildings, etc.) in the detection scene, and the model will predict which class each object belongs to according to the image features.

[0123] The target position bounding box information can describe the specific position of the detected object in the image.

[0124] The spliced feature map can be obtained by splicing the attention weighted features of the local head, the global head, the channel feature head, and the spatial feature head in the feature channel dimension. The target detection result map can be obtained by linearly transforming the spliced feature map, which can be realized by a full connection layer (or a convolution layer). Specifically, to perform linear transformation, a weight matrix and a bias vector can be defined, and the calculation process of the weight matrix is as follows:

[0125] W linear =C concat ×(N classes +4);

[0126] wherein W linear is the weight matrix; C concat is the number of feature channels of the spliced feature map; Nclasses is the number of target categories; 4 is used for the bounding box parameters, which can be (x min , y min , width, height).

[0127] The bias vector calculation process is as follows:

[0128] b linear = N classes + 4.

[0129] The linear transformation is to multiply the spliced feature map with the weight matrix and add the bias, and the formula is as follows:

[0130] F output = F concat · W linear + b linear .

[0131] Wherein, F output is the target detection result map; F concat is the spliced feature map.

[0132] Here, the operation is usually performed on the last dimension, that is, the features of each position (i, j) are processed, and specifically, for each position (i, j), the features of the spliced feature map F concat (i, j, c) are multiplied by the weight matrix W linear to obtain an output of size (N classes + 4). After performing the linear transformation, the size of the obtained result F output is:

[0133] height x width x (N classes + 4)

[0134] Wherein the output of each position contains target category information: the first N classes channels contain the class probability distribution of each target. Bounding box information: the last 4 channels contain the bounding box parameters of the detected target.

[0135] In the embodiments of the application, by combining the high-resolution feature map and the low-resolution feature map, the model can obtain multi-scale information and improve the detection performance of small targets. By calculating the relative position encoding of the feature points and adding it to the key matrix, the detection accuracy can be improved. By using the attention mechanism of the local head, the global head, the channel feature head and the spatial feature head, the key features of small targets can be better focused, and the detection effect of small targets can be improved. The linear transformation can improve the response speed and accuracy of small target detection.

[0136] On the basis of the above technical solution, optionally, after obtaining the target detection result image, the method further comprises:

[0137] transmitting the target detection result image to a control center for the staff of the control center to understand the on-site situation according to the target detection result image.

[0138] In this solution, the control center can refer to a place or an organization that centrally manages and monitors, and is usually used to process, analyze and respond to information from different sources. In the context of target detection, the control center may be responsible for monitoring the on-site situation, making decisions, coordinating resources, and ensuring the safety and efficiency of the site.

[0139] The target detection result image (usually an image or a video frame) can be formatted into a format suitable for transmission (such as JPEG, PNG, or video format). Then use a network protocol (such as HTTP, FTP or WebSocket) to transmit the target detection result image to the server or cloud platform of the control center.

[0140] Figure 2 An image analysis system for target detection is provided. The system comprises:

[0141] The detection module 201 is configured to obtain a to-be-detected picture, input the to-be-detected picture into a preset picture detection model, and obtain a high-resolution feature map and a low-resolution feature map of the to-be-detected picture. The high-resolution feature map and the low-resolution feature map each comprise at least two feature points.

[0142] The transformation module 202 is configured to transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through the preset picture detection model, to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map.

[0143] The key matrix processing module 203 is configured to calculate relative position encodings of the feature points of the high-resolution feature map and the low-resolution feature map through the preset picture detection model, and add the relative position encodings to the key matrix of the high-resolution feature map and the low-resolution feature map, to obtain a key matrix containing the relative position encodings of the high-resolution feature map and the low-resolution feature map.

[0144] The attention weighted feature determination module 204 is configured to calculate similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map according to a preset similarity score calculation formula, a high-resolution feature map and a key matrix containing relative position encoding of the low-resolution feature map by using the local head, the global head, the channel feature head and the spatial feature head of the preset picture detection model, and multiply the similarity scores with value matrices of the high-resolution feature map and the low-resolution feature map to obtain attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head.

[0145] The target detection result map determination module 205 is configured to splice the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head by using the preset picture detection model to obtain a spliced feature map, and perform linear transformation on the spliced feature map to obtain a target detection result map, wherein the target detection result map comprises target category information and target position bounding box information.

[0146] In the embodiments of the present application, by combining the high-resolution feature map and the low-resolution feature map, the model can obtain multi-scale information and improve the detection performance of small targets. By calculating the relative position encoding of the feature points and adding it to the key matrix, the detection accuracy can be improved. By using the attention mechanism of the local head, the global head, the channel feature head and the spatial feature head, the key features of small targets can be better focused, and the detection effect of small targets can be improved. Linear transformation can improve the response speed and accuracy of small target detection.

[0147] Figure 3 A schematic block diagram of an electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit the implementations of the present disclosure described and / or claimed in this document.

[0148] The electronic device 300 includes a computing unit 301 that can perform various appropriate actions and processes in accordance with a computer program stored in a ROM 302 or loaded from a storage unit 308 into a RAM 303. Various programs and data required for the operation of the electronic device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0149] The plurality of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a magnetic disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0150] The computing unit 301 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 301 performs various methods and processes described above, such as the image analysis method for object detection. For example, in some embodiments, the image analysis method for object detection can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded onto the RAM 303 and executed by the computing unit 301, one or more steps of the image analysis method for object detection described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the image analysis method for object detection by other any appropriate means, such as by means of firmware.

[0151] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0152] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage medium can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0155] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0156] The computer system can include clients and servers. This relationship can be. The servers are typically remote from the clients with the interactions between them occurring over a communication network. The relationship between a client and a server is one of client-server. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0157] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, and the present disclosure is not limited herein.

[0158] The specific embodiments described above are not intended to be limiting. One of skill in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments described above without departing from the scope of the disclosure. Any modifications, combinations, sub-combinations, and alternatives falling within the spirit and principles of the disclosure should be considered.

Claims

1. An image analysis method applied to object detection, characterized in that, The method comprises: acquiring a to-be-detected picture, inputting the to-be-detected picture into a preset picture detection model, and obtaining a high-resolution feature map and a low-resolution feature map of the to-be-detected picture; wherein the high-resolution feature map and the low-resolution feature map each comprise at least two feature points; transforming the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through the preset picture detection model, and obtaining a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map; calculating relative position encodings of each feature point of the high-resolution feature map and the low-resolution feature map through the preset picture detection model, and adding the relative position encodings to the key matrix of the high-resolution feature map and the low-resolution feature map, to obtain a key matrix containing relative position encodings of the high-resolution feature map and the low-resolution feature map; calculating similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map according to a preset similarity score calculation formula, the key matrix containing relative position encodings of the high-resolution feature map and the low-resolution feature map, through a local head, a global head, a channel feature head and a spatial feature head of the preset picture detection model, and multiplying the similarity scores with the value matrix of the high-resolution feature map and the low-resolution feature map, to obtain attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head; splicing the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head through the preset picture detection model to obtain a spliced feature map, and performing linear transformation on the spliced feature map to obtain a target detection result map; wherein the target detection result map comprises target category information and target position bounding box information.

2. The method of claim 1, wherein, Wherein, inputting the to-be-detected picture into a preset picture detection model to obtain a high-resolution feature map and a low-resolution feature map of the to-be-detected picture comprises: inputting the to-be-detected picture into the preset picture detection model, and performing convolution and pooling operations on the to-be-detected picture through a Backbone layer of the preset picture detection model to obtain an initial high-resolution feature map and an initial low-resolution feature map; processing the initial high-resolution feature map and the initial low-resolution feature map through a channel attention component of an Attention Module layer of the preset picture detection model to obtain a high-resolution channel attention map and a low-resolution channel attention map; multiplying the high-resolution channel attention map with the initial high-resolution feature map, and multiplying the low-resolution channel attention map with the initial low-resolution feature map to obtain an enhanced high-resolution feature map and an enhanced low-resolution feature map; processing the enhanced high-resolution feature map and the enhanced low-resolution feature map through a spatial attention component of the Attention Module layer of the preset picture detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map; The enhanced high-resolution feature map is multiplied by the initial high-resolution feature map, and the enhanced low-resolution feature map is multiplied by the initial low-resolution feature map, to obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected.

3. The method of claim 2, wherein, Wherein, The initial high-resolution feature map and the initial low-resolution feature map are processed through the channel attention component of the Attention Module layer of the preset picture detection model, to obtain a high-resolution channel attention map and a low-resolution channel attention map, including: The global maximum pooling and the global average pooling are respectively performed on each channel of the initial high-resolution feature map and the initial low-resolution feature map through the channel attention component of the Attention Module layer of the preset picture detection model, to obtain an initial high-resolution one-dimensional feature map and an initial low-resolution one-dimensional feature map; wherein the initial high-resolution one-dimensional feature map and the initial low-resolution one-dimensional feature map both contain the global maximum pooling value and the global average pooling value of each channel; The global maximum pooling value and the global average pooling value of the initial high-resolution one-dimensional feature map are summed to obtain a first sum result, and the first sum result is normalized through a Sigmoid activation function to obtain a high-resolution channel attention map; The global maximum pooling value and the global average pooling value of the initial low-resolution one-dimensional feature map are summed to obtain a second sum result, and the second sum result is normalized through a Sigmoid activation function to obtain a low-resolution channel attention map.

4. The method of claim 2, wherein, Wherein, The enhanced high-resolution feature map and the enhanced low-resolution feature map are processed through the spatial attention component of the Attention Module layer of the preset picture detection model, to obtain a high-resolution spatial attention map and a low-resolution spatial attention map, including: The global maximum pooling and the global average pooling are performed on the pixel values at the same position of the enhanced high-resolution feature map through the spatial attention component of the Attention Module layer of the preset picture detection model, to obtain a first spatial maximum pooling feature map and a first spatial average pooling feature map; The global maximum pooling and the global average pooling are performed on the pixel values at the same position of the enhanced low-resolution feature map through the spatial attention component of the Attention Module layer of the preset picture detection model, to obtain a second spatial maximum pooling feature map and a second spatial average pooling feature map; The first spatial maximum pooling feature map and the first spatial average pooling feature map are spliced to obtain a first spliced spatial feature map, and the second spatial maximum pooling feature map and the second spatial average pooling feature map are spliced to obtain a second spliced spatial feature map; The first spliced spatial feature map and the second spliced spatial feature map are subjected to convolution operation and Sigmoid activation to obtain a high-resolution spatial attention map and a low-resolution spatial attention map.

5. The method of claim 1, wherein, Wherein, The high-resolution feature map and the low-resolution feature map are transformed according to the preset query weight matrix, the preset key weight matrix, and the preset value weight matrix through the preset picture detection model, to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map, including: A preset transformation formula is obtained, and the high-resolution feature map and the low-resolution feature map are transformed according to the preset query weight matrix, the preset key weight matrix, the preset value weight matrix, and the preset transformation formula through the preset picture detection model, to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map; wherein the preset transformation formula is: wherein, is a high resolution feature map / low resolution feature map; is a preset query weight matrix; is a preset key weight matrix; is a preset value weight matrix; is a query matrix of the high resolution feature map / low resolution feature map; is a key matrix of the high resolution feature map / low resolution feature map; is a value matrix of the high resolution feature map / low resolution feature map.

6. The method of claim 1, wherein, wherein, The relative position encodings of the feature points of the high-resolution feature map and the low-resolution feature map are calculated through the preset picture detection model, and the relative position encodings are added to the key matrix of the high-resolution feature map and the low-resolution feature map, to obtain a key matrix containing the relative position encodings of the high-resolution feature map and the low-resolution feature map, including: A preset relative position encoding calculation formula is obtained, and coordinate information of the feature points of the high-resolution feature map and the low-resolution feature map is determined; The relative position encodings of the feature points of the high-resolution feature map and the low-resolution feature map are calculated according to the preset relative position encoding calculation formula and the coordinate information of the feature points of the high-resolution feature map and the low-resolution feature map through the preset picture detection model; wherein the preset relative position encoding calculation formula is: wherein, is the relative position encoding of the i-th feature point and the j-th feature point of the high-resolution feature map / low-resolution feature map; is the coordinate information of the i-th feature point of the high-resolution feature map / low-resolution feature map; is the coordinate information of the j-th feature point of the high-resolution feature map / low-resolution feature map; is a function used to calculate the relative position encoding; The relative position encodings of the high-resolution feature map and the low-resolution feature map are added to the key matrix, to obtain a key matrix containing the relative position encodings of the high-resolution feature map and the low-resolution feature map.

7. The method of claim 1, wherein, wherein, The preset similarity score calculation formula is: wherein, is a similarity score of each feature point in the high-resolution feature map / low-resolution feature map; Q is a query matrix of the high-resolution feature map / low-resolution feature map; is a key matrix of the high-resolution feature map / low-resolution feature map containing relative position encoding; is a transpose of the key matrix of the high-resolution feature map / low-resolution feature map containing relative position encoding; V is a value matrix of the high-resolution feature map / low-resolution feature map; is a dimension of the key matrix of the high-resolution feature map / low-resolution feature map; is a similarity score matrix, which is a dot product of the query matrix of the high-resolution feature map / low-resolution feature map and the transpose of the key matrix containing relative position encoding.

8. The method of claim 1, wherein, wherein, After obtaining the target detection result map, the method further includes: The target detection result map is transmitted to a control center, for a staff of the control center to understand a scene condition according to the target detection result map.

9. An image analysis system for object detection for performing the method according to any one of claims 1 to 8, characterized in that The system includes: A detection module is configured to obtain a to-be-detected picture, input the to-be-detected picture into a preset picture detection model, and obtain a high-resolution feature map and a low-resolution feature map of the to-be-detected picture; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points. A transformation module is configured to transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix, and a preset value weight matrix through the preset picture detection model, to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map. A key matrix processing module is configured to calculate relative position encodings of the feature points of the high-resolution feature map and the low-resolution feature map through the preset picture detection model, and add the relative position encodings to the key matrix of the high-resolution feature map and the low-resolution feature map, to obtain a key matrix containing the relative position encodings of the high-resolution feature map and the low-resolution feature map. The attention weighted feature determination module is configured to calculate similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map according to a preset similarity score calculation formula, a key matrix containing relative position encoding of the high-resolution feature map and the low-resolution feature map, and local head, global head, channel feature head and spatial feature head of the preset picture detection model, and multiply the similarity scores with value matrices of the high-resolution feature map and the low-resolution feature map to obtain attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head. The target detection result map determination module is configured to splice the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head by the preset picture detection model to obtain a spliced feature map, and perform linear transformation on the spliced feature map to obtain a target detection result map, wherein the target detection result map comprises target category information and target position bounding box information. 10.An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

Images