Image analysis method and system applied to target detection and electronic equipment

Abstract

The embodiment of the invention provides an image analysis method and system applied to target detection and electronic equipment. The method is applied to the field of image processing, and comprises the following steps: acquiring a to-be-detected picture, and determining a high-resolution feature map and a low-resolution feature map of the to-be-detected picture; determining a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map; key matrixes containing relative position codes of the high-resolution feature map and the low-resolution feature map are determined; determining attention weighted features of the high-resolution feature map and the low-resolution feature map at a local head, a global head, a channel feature head and a spatial feature head; and determining a target detection result graph. According to the scheme, multi-scale information can be obtained, and the small target detection performance is improved. The relative position codes of the feature points are calculated, and the detection accuracy can be improved. A multi-head fusion attention mechanism can focus on key features of a small target. The linear transformation can improve the response speed and precision of small target detection.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image analysis method, system and electronic equipment applied to target detection. Background Art

[0002] In modern technology fields such as intelligent transportation, security monitoring, and medical image analysis, object detection is a crucial image analysis task. The core of this technology is to accurately identify and locate the target object of interest from complex image data, whether it is a vehicle, pedestrian, face, or lesion area in medical images.

[0003] The image recognition process includes feature extraction, candidate region generation, classification and regression. First, the convolutional neural network is used to extract features from the input image to obtain a feature map. Next, a series of candidate regions are generated through the region proposal network, which may contain targets. Then, fine feature extraction is performed on each candidate region, and a classifier is used to determine whether it contains a target and the category of the target. Finally, the position and size of the candidate region are adjusted through the regressor to obtain the detection result.

[0004] However, although today’s image recognition processing performs well in processing large and medium-sized objects in images with normal resolution, it is not as good as expected when it comes to detecting small objects in high-resolution images. These small objects are often too small and have unclear features to be accurately captured and recognized by existing technologies. Summary of the invention

[0005] In order to solve the shortcomings of the prior art, the present disclosure provides an image analysis method, system and device for target detection. The present disclosure solves the technical problem that when facing small targets in high-resolution images, these small targets are often difficult to accurately capture and identify due to their small size and unclear features, resulting in poor detection effect.

[0006] According to a first aspect of the present disclosure, there is provided an image analysis method for target detection, comprising: obtaining a picture to be detected, inputting the picture to be detected into a preset picture detection model, and obtaining a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points;

[0007] The high-resolution feature map and the low-resolution feature map are transformed according to a preset query weight matrix, a preset key weight matrix, and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map;

[0008] Calculate the relative position code of each feature point of the high-resolution feature map and the low-resolution feature map by using a preset image detection model, and add the relative position code to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position code;

[0009] The local head, global head, channel feature head and spatial feature head of the preset image detection model are used to calculate the similarity scores of the feature points in the high-resolution feature map and the low-resolution feature map according to the preset similarity score calculation formula, the key matrix containing the relative position encoding of the high-resolution feature map and the low-resolution feature map, and the similarity scores are multiplied by the value matrices of the high-resolution feature map and the low-resolution feature map to obtain the attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head;

[0010] The attention weighted features of the local head, the global head, the channel feature head and the spatial feature head are spliced ​​through a preset image detection model to obtain a spliced ​​feature map, and the spliced ​​feature map is linearly transformed to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

[0011] According to a second aspect of the present disclosure, there is provided an image analysis system for target detection, which is used to execute the method according to the first aspect, including: a detection module, which is used to obtain a picture to be detected, input the picture to be detected into a preset picture detection model, and obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points;

[0012] A transformation module, used to transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map;

[0013] A key matrix processing module, used to calculate the relative position code of each feature point of the high-resolution feature map and the low-resolution feature map through a preset image detection model, and add the relative position code to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position code;

[0014] An attention weighted feature determination module is used to calculate the similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map respectively through the local head, global head, channel feature head and spatial feature head of the preset image detection model according to the preset similarity score calculation formula, the key matrix containing the relative position encoding of the high-resolution feature map and the low-resolution feature map, and multiply the similarity scores with the value matrices of the high-resolution feature map and the low-resolution feature map to obtain the attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, global head, channel feature head and spatial feature head;

[0015] The target detection result map determination module is used to splice the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head through a preset image detection model to obtain a spliced ​​feature map, and linearly transform the spliced ​​feature map to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0017] In an image analysis method, system and device for target detection provided above, the disclosed embodiment combines high-resolution feature maps and low-resolution feature maps, and the model can obtain multi-scale information and improve the detection performance of small targets. By calculating the relative position encoding of feature points and adding them to the key matrix, the detection accuracy can be improved. The attention mechanism of local head, global head, channel feature head and spatial feature head can better focus on the key features of small targets and improve the detection effect of small targets. Linear transformation can improve the response speed and accuracy of small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of a process of an image analysis method applied to target detection according to an embodiment of the present disclosure is shown;

[0020] Figure 2 A schematic block diagram of an image analysis system applied to target detection according to an embodiment of the present disclosure is shown;

[0021] Figure 3 A block diagram of an exemplary electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0023] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor represent the necessary logical order between them. It should also be understood that in the embodiments of the present disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, in the absence of explicit limitation or contrary revelation given in the context, it can generally be understood as one or more. In addition, the term "and / or" in the present disclosure is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are an "or" relationship. It should also be understood that the description of each embodiment in the present disclosure emphasizes the differences between the embodiments, and the same or similar parts can refer to each other. For the sake of brevity, they will not be repeated one by one.

[0024] At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. The techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices should be considered part of the specification. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] Figure 1 A schematic diagram of a flow chart of an image analysis method for target detection provided in an embodiment of the present disclosure. The method of the embodiment of the present disclosure aims to achieve accurate detection of large and small targets in an image.

[0027] S101, obtaining a picture to be detected, inputting the picture to be detected into a preset picture detection model, and obtaining a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points.

[0028] The image to be detected can be an image that needs to be processed for target detection. It can be any input image, such as a photo in a natural scene, a video frame, or other visual data containing an object.

[0029] The preset image detection model can be a convolutional neural network architecture specifically designed for object detection tasks. For example, it can be the YOLOv5 model, which is a fast and efficient model that can simultaneously predict the categories and bounding box locations of multiple objects in a single forward propagation process. The preset image detection model refers to a pre-set detection network with a specific structure and parameters. YOLOv5 is an example of such a preset model.

[0030] A high-resolution feature map can be a feature extracted from a shallow network of an image. Since it passes through fewer convolutional layers and fewer pooling operations, it retains more spatial details, that is, the pixel resolution in the image is higher. This feature map is usually used to detect smaller objects, because small objects occupy fewer pixels in the image, and the retention of detail information can more accurately locate and identify these objects.

[0031] Low-resolution feature maps can be features extracted from a deeper network of an image. As more convolution and pooling layers are passed through, the resolution of the feature map decreases, but its features are more abstract and can better capture the global or larger-scale semantic information of the object. This makes low-resolution feature maps more effective in large object detection because they can summarize the overall shape and structure of the target.

[0032] Feature points can be key locations in a feature map, and the information contained in each feature point represents the characteristics of a certain area or location in the input image. Feature points correspond to each pixel or unit in the feature map, and these feature points capture some important properties of objects in the image, such as edges, textures, or high-level semantic information.

[0033] The image to be detected can be input into a preset image detection model, such as a YOLOv5 model or a similar convolutional neural network. The backbone network layer of the model is responsible for extracting feature maps of different levels from the input image, and obtaining high-resolution feature maps (for small target detection) and low-resolution feature maps (for large target detection). Specifically, high-resolution feature maps are usually generated in the shallow network of the model (the network layer close to the input layer). Due to the fewer steps and pooling layers of the convolution operation, its feature map has a higher spatial resolution and retains more details of the image. This is very effective for detecting small targets. Low-resolution feature maps are generated in the deep network of the model (the network layer close to the output layer). After multiple convolution and pooling operations, its spatial resolution is lower, but the semantic information is richer, which is more suitable for detecting large targets. In each feature map (whether high-resolution or low-resolution), each pixel or unit can be regarded as a feature point. These feature points contain local features of the input image at that position, such as color, edge or object outline.

[0034] The training process of the preset image detection model can include collecting a dataset of images containing various object categories and annotated bounding boxes. Make sure the dataset contains different scenes, angles, lighting conditions, etc. to improve the generalization ability of the model. Provide the corresponding object category label and location bounding box information for each image, which can be used (x min ,y min, width, height) format. Backbone: Select a suitable convolutional neural network (such as ResNet, EfficientNet, etc.) as the basis for feature extraction, which can extract high-resolution and low-resolution feature maps. Design channel attention modules and spatial attention modules to enhance the feature representation ability of feature maps. Design feature fusion heads to process local, global, channel and spatial features respectively, and obtain corresponding attention scores through similarity calculation. Input the image to be detected, and extract high-resolution feature maps and low-resolution feature maps through the backbone of the model. Transform the feature maps using the preset query weight matrix, key weight matrix and value weight matrix to obtain the query matrix, key matrix and value matrix. Calculate the relative position encoding of the feature points and add it to the key matrix so that the relative position relationship between the feature points is considered when calculating attention. Use the preset similarity score calculation formula to calculate the similarity scores of the high-resolution feature map and the low-resolution feature map respectively according to the key matrix containing the relative position encoding. Multiply the similarity score with the value matrix to obtain the attention fusion feature. Define a suitable loss function, such as cross entropy loss (for classification tasks) and bounding box regression loss (for position prediction) to optimize the output of the model. Use an optimization algorithm (such as Adam, SGD) to perform back propagation and update the model parameters until the loss converges. Evaluate the model performance on the validation set, monitor the model's performance on unseen data, and adjust hyperparameters. Use common evaluation metrics such as mAP (meanAverage Precision) and IoU (Intersection over Union) to evaluate the detection performance of the model. Based on the evaluation results, it may be necessary to further fine-tune the model, add data augmentation, or use techniques such as transfer learning to improve performance.

[0035] On the basis of the above technical solution, optionally, the image to be detected is input into a preset image detection model to obtain a high-resolution feature map and a low-resolution feature map of the image to be detected, including:

[0036] Input the image to be detected into a preset image detection model, and perform convolution and pooling operations on the image to be detected through the Backbone layer of the preset image detection model to obtain an initial high-resolution feature map and an initial low-resolution feature map;

[0037] The initial high-resolution feature map and the initial low-resolution feature map are processed by a channel attention component of the Attention Module layer of a preset image detection model to obtain a high-resolution channel attention map and a low-resolution channel attention map;

[0038] Multiplying the high-resolution channel attention map with the initial high-resolution feature map, and multiplying the low-resolution channel attention map with the initial low-resolution feature map to obtain an enhanced high-resolution feature map and an enhanced low-resolution feature map;

[0039] The enhanced high-resolution feature map and the enhanced low-resolution feature map are processed through the spatial attention component of the Attention Module layer of the preset image detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map;

[0040] The enhanced high-resolution feature map is multiplied by the initial high-resolution feature map, and the enhanced low-resolution feature map is multiplied by the initial low-resolution feature map to obtain a high-resolution feature map and a low-resolution feature map of the image to be detected.

[0041] In this solution, the Backbone layer can be the main feature extraction part in the preset image detection model, which is usually composed of multiple convolutional layers and pooling layers. Its main function is to extract low-level and high-level features from the input image. The convolution operation is used to detect local features (such as edges, textures, etc.), while the pooling operation is used to reduce the spatial dimension of the feature map, retaining important information while reducing computational complexity.

[0042] The Attention Module layer can be a preset mechanism for the image detection model to focus on important features. By calculating the weights of the input features, the model can pay more attention to key information when performing feature fusion. The Attention mechanism can be divided into different components, such as channel attention and spatial attention, which are used to process different aspects of features respectively.

[0043] The channel attention component can emphasize important feature channels by calculating the weights of each channel. Assigning different importance to different feature channels allows the model to focus on more significant features during feature extraction. For example, channel descriptions can be obtained through global average pooling and global maximum pooling, and then channel weights can be generated through a small neural network (such as MLP), and finally the weights can be applied to the original feature map.

[0044] The high-resolution channel attention map can be a feature map generated by the channel attention component, which aims to highlight the importance of each channel in the image. It generates a weight distribution by analyzing the feature response of each channel, emphasizing those channels that are more important for object detection.

[0045] The low-resolution channel attention map is similar to the high-resolution channel attention map, but it is generated for the low-resolution feature map. The map emphasizes the key channels in the low-resolution feature map through the channel attention mechanism, aiming to preserve important contextual information and enhance the detection ability of large objects or overall features.

[0046] The enhanced high-resolution feature map can be the result of multiplying the high-resolution channel attention map with the initial high-resolution feature map. This process enhances the high-resolution feature map in key features and improves the model's ability to detect small objects or important local features.

[0047] The enhanced low-resolution feature map can be obtained by multiplying the low-resolution channel attention map with the initial low-resolution feature map. This map provides enhanced contextual information, which is suitable for detecting large objects or global features, thus playing a role in the feature fusion stage.

[0048] The high-resolution spatial attention map can be obtained by processing the enhanced high-resolution feature map through the spatial attention component. The spatial attention mechanism focuses on the spatial distribution of the feature map and emphasizes the most relevant areas in the image, thereby helping the model better understand the spatial structure and important locations of the image.

[0049] The low-resolution spatial attention map can be similar to the high-resolution spatial attention map, but it is generated based on the enhanced low-resolution feature map. This map emphasizes the important spatial areas in the low-resolution feature map, which is suitable for large object detection and overall image understanding, and improves the model's responsiveness to global features.

[0050] The image to be detected can pass through the backbone layer of the model and extract features through multiple convolutional layers. Each convolutional layer extracts features at different levels, including edge, texture and other information. After the convolution operation, pooling (such as maximum pooling or average pooling) is usually performed to reduce the spatial dimension of the feature map and enhance the robustness of the model to small deformations and position changes. After convolution and pooling, two feature maps are obtained: the initial high-resolution feature map (retains more spatial information, suitable for detecting small targets) and the initial low-resolution feature map (contains more abstract features, suitable for detecting large targets). Then the initial high-resolution feature map and the initial low-resolution feature map are processed, and the corresponding attention maps are generated through the channel attention mechanism: High-resolution channel attention map: emphasizes the important channels in the high-resolution feature map. Low-resolution channel attention map: emphasizes the important channels in the low-resolution feature map. Then the high-resolution channel attention map is element-wise multiplied with the initial high-resolution feature map to obtain the enhanced high-resolution feature map. This process strengthens the important features in the feature map. Similarly, the low-resolution channel attention map is element-wise multiplied with the initial low-resolution feature map to obtain the enhanced low-resolution feature map. The enhanced high-resolution feature map and low-resolution feature map are then processed to generate a spatial attention map: High-resolution spatial attention map: emphasizes important spatial regions in the high-resolution feature map. Low-resolution spatial attention map: emphasizes important spatial regions in the low-resolution feature map. Finally, the enhanced high-resolution feature map is element-wise multiplied with the initial high-resolution feature map to obtain the final high-resolution feature map. This process combines the initial features with the features enhanced by the attention mechanism. Similarly, the enhanced low-resolution feature map is element-wise multiplied with the initial low-resolution feature map to obtain the final low-resolution feature map.

[0051] In this scheme, important information of different scales and features can be effectively extracted and enhanced, improving the accuracy and robustness of target detection.

[0052] On the basis of the above technical solution, optionally, the initial high-resolution feature map and the initial low-resolution feature map are processed by the channel attention component of the AttentionModule layer of the preset image detection model to obtain a high-resolution channel attention map and a low-resolution channel attention map, including:

[0053] Through the channel attention component of the Attention Module layer of the preset image detection model, global maximum pooling and global average pooling are performed on each channel of the initial high-resolution feature map and the initial low-resolution feature map, respectively, to obtain an initial high-resolution one-dimensional feature map and an initial low-resolution one-dimensional feature map; wherein the initial high-resolution one-dimensional feature map and the initial low-resolution one-dimensional feature map both contain the global maximum pooling value and the global average pooling value of each channel;

[0054] The global maximum pooling value and the global average pooling value of the initial high-resolution one-dimensional feature map are summed to obtain a first summation result, and the first summation result is normalized by a Sigmoid activation function to obtain a high-resolution channel attention map;

[0055] The global maximum pooling value and the global average pooling value of the initial low-resolution one-dimensional feature map are summed to obtain a second summation result, and the second summation result is normalized by a Sigmoid activation function to obtain a low-resolution channel attention map.

[0056] In this solution, the initial high-resolution one-dimensional feature map can be a feature vector obtained by performing global maximum pooling and global average pooling operations on each channel of the initial high-resolution feature map. This one-dimensional feature map represents the significance of each channel in the overall feature map.

[0057] The initial low-resolution one-dimensional feature map may be a one-dimensional feature map obtained by performing the same pooling operation on each channel of the initial low-resolution feature map.

[0058] The global maximum pooling value can be the maximum value of the feature map of a certain channel, which represents the most significant feature of the channel.

[0059] The global average pooling value can be the average value of all pixel values ​​in a channel, which represents the overall feature strength of the channel.

[0060] The first summation result may be the sum of the global maximum pooling value and the global average pooling value in the initial high-resolution one-dimensional feature map, and the result obtained reflects the importance of the channel in feature extraction.

[0061] The Sigmoid activation function can be a commonly used activation function, whose output range is between 0 and 1, which can be expressed as It normalizes the first summation result for subsequent use.

[0062] The second summation result may be the sum of the global maximum pooling value and the global average pooling value in the initial low-resolution one-dimensional feature map, and the result obtained represents the overall significance of the low-resolution feature channel.

[0063] Each channel of the initial high-resolution feature map and the initial low-resolution feature map can be subjected to maximum pooling, and the maximum value in each channel can be selected to generate the maximum pooling part of the high-resolution and low-resolution one-dimensional feature maps. The same feature map is subjected to average pooling, and the average value of each channel is calculated to generate the average pooling part of the one-dimensional feature map. Then the global maximum pooling value and the global average pooling value of each channel of the high-resolution feature map are combined to form a one-dimensional feature map. Similarly, the pooling values ​​of each channel of the low-resolution feature map are combined to form another one-dimensional feature map. The global maximum pooling value and the global average pooling value in the initial high-resolution one-dimensional feature map are added to obtain a first summation result. The first summation result is normalized by applying the Sigmoid activation function to obtain a high-resolution channel attention map, which ranges from 0 to 1. The global maximum pooling value and the global average pooling value in the initial low-resolution one-dimensional feature map are added to obtain a second summation result. The second summation result is also normalized by applying the Sigmoid activation function to obtain a low-resolution channel attention map.

[0064] In this scheme, the channel attention component can effectively extract the saliency of each channel in the feature map, thereby improving the model's attention to important features and further improving the effect of target detection.

[0065] On the basis of the above technical solution, optionally, the enhanced high-resolution feature map and the enhanced low-resolution feature map are processed by the spatial attention component of the AttentionModule layer of the preset image detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map, including:

[0066] Through the spatial attention component of the Attention Module layer of the preset image detection model, global maximum pooling and global average pooling are performed on the pixel values ​​at the same position of the enhanced high-resolution feature map to obtain the first spatial maximum pooling feature map and the first spatial average pooling feature map;

[0067] Through the spatial attention component of the Attention Module layer of the preset image detection model, global maximum pooling and global average pooling are performed on the pixel values ​​at the same position of the enhanced low-resolution feature map to obtain a second spatial maximum pooling feature map and a second spatial average pooling feature map;

[0068] The first spatial maximum pooling feature map and the first spatial average pooling feature map are spliced ​​to obtain a first spliced ​​spatial feature map, and the second spatial maximum pooling feature map and the second spatial average pooling feature map are spliced ​​to obtain a second spliced ​​spatial feature map;

[0069] Convolution operation and Sigmoid activation are performed on the first spliced ​​spatial feature map and the second spliced ​​spatial feature map to obtain a high-resolution spatial attention map and a low-resolution spatial attention map.

[0070] In this solution, the first spatial maximum pooling feature map can be a feature map obtained by performing global maximum pooling on each spatial position of the enhanced high-resolution feature map. It retains the maximum pixel value of each spatial position, highlights the most significant features, and helps the model focus on key areas.

[0071] The first spatial average pooling feature map may be a feature map obtained by performing global average pooling on each spatial position of the enhanced high-resolution feature map. It calculates the average value of each spatial position and provides distribution information of the overall feature.

[0072] The second spatial maximum pooling feature map is similar to the first spatial maximum pooling feature map, but it is the result of enhancing the low-resolution feature map. It retains the maximum pixel value at each spatial position in the low-resolution feature map.

[0073] The second spatial average pooling feature map is also the average pooling result of the enhanced low-resolution feature map, reflecting the average pixel value at each spatial position.

[0074] The first spliced ​​spatial feature map may be a feature map obtained by splicing the first spatial maximum pooling feature map and the first spatial average pooling feature map. After splicing, the feature map contains the maximum and average information of the high-resolution features, providing richer context information for subsequent processing.

[0075] The second spliced ​​spatial feature map may be a feature map obtained by splicing the second spatial maximum pooling feature map and the second spatial average pooling feature map, retaining the maximum and average information of the low-resolution features.

[0076] The enhanced high-resolution feature map can be subjected to global maximum pooling, traversing each channel, extracting the maximum value of all pixels in the channel, and obtaining the first spatial maximum pooling feature map. Global average pooling is performed on it, and the average value of all pixels in each channel is calculated to obtain the first spatial average pooling feature map. Global maximum pooling is performed on the enhanced low-resolution feature map to obtain the second spatial maximum pooling feature map. Global average pooling is performed on it to obtain the second spatial average pooling feature map. The first spatial maximum pooling feature map and the first spatial average pooling feature map are spliced ​​to obtain the first spliced ​​spatial feature map. This step combines different pooling information and enhances the representation ability of the feature. The second spatial maximum pooling feature map and the second spatial average pooling feature map are spliced ​​to obtain the second spliced ​​spatial feature map. Convolution operation is performed on the first spliced ​​spatial feature map and the second spliced ​​spatial feature map to extract higher-level features. Then the convolved features are activated by the Sigmoid activation function to obtain a high-resolution spatial attention map and a low-resolution spatial attention map.

[0077] In this scheme, the spatial attention mechanism is applied to the enhanced feature map, which can effectively highlight the important spatial areas and provide more accurate information for subsequent object detection tasks.

[0078] S102, transforming the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map.

[0079] The preset query weight matrix can be used to linearly transform the information of the input feature map (high-resolution and low-resolution feature maps) to generate a query vector. The feature map can be matrix multiplied to generate a query matrix for querying information related to feature points. It is usually a learnable weight matrix, and its shape is determined by the dimensions of the input feature map and the dimensions of the output query vector.

[0080] The preset key weight matrix can be used to linearly transform the information in the input feature map into a key vector. The key vector can be used to represent the correlation between feature points, so as to calculate the similarity with the query matrix. It is also a learnable weight matrix, and its shape is determined by the dimensions of the input feature map and the key matrix.

[0081] The preset value weight matrix can be used to linearly transform the input feature map into a value vector. A value matrix can be generated to represent the feature map, and the final attention score will act on these value matrices to obtain the fused feature map. It is a learnable weight matrix, and its shape is determined by the dimensions of the input feature map and the output value matrix.

[0082] The query matrix can be a matrix obtained by multiplying the high-resolution or low-resolution feature map with the query weight matrix. It can be used to perform dot products with the key matrix to calculate the similarity between feature points. The query vector of each feature point represents the importance of the point among other feature points.

[0083] The key matrix can be a matrix obtained by multiplying the feature map with the key weight matrix. It can be used together with the query matrix to calculate the correlation or similarity between feature points. The key vector of each feature point describes its relationship with other feature points.

[0084] The value matrix can be a matrix obtained by multiplying the feature map with the value weight matrix. The value matrix stores the value of each feature point in the feature map and is output after being weighted by the attention weight. The final output is the weighted sum of these value matrices.

[0085] Through the Attention module in the image detection model, the linear transformation operations of the preset query weight matrix, the preset key weight matrix, and the preset value weight matrix can be applied to the high-resolution feature map and the low-resolution feature map respectively to generate the query matrix, key matrix, and value matrix. Then the position of each feature point can be encoded and these encoded values ​​are added to the key matrix. These relative position encodings help the Attention mechanism better understand the spatial relationship between feature points. For each input feature map (high-resolution and low-resolution feature map), the query, key, and value calculations can also be performed separately. This operation is performed independently for the feature map of each resolution, but these matrices will be combined in the final attention fusion.

[0086] On the basis of the above technical solution, optionally, the high-resolution feature map and the low-resolution feature map are transformed according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map, including:

[0087] Obtain a preset transformation formula, and transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix, a preset value weight matrix, and a preset transformation formula through a preset image detection model to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map; wherein the preset transformation formula is:

[0088] Q=F·W Q ;

[0089] K=F·W K ;

[0090] V=F·W V ;

[0091] Among them, F is the high-resolution feature map / low-resolution feature map; W Q is the preset query weight matrix; W K is the preset key weight matrix; W V is the preset value weight matrix; Q is the query matrix of high-resolution feature map / low-resolution feature map; K is the key matrix of high-resolution feature map / low-resolution feature map; V is the value matrix of high-resolution feature map / low-resolution feature map.

[0092] In this solution, the high-resolution feature map and the low-resolution feature map are the input feature maps, which can be two-dimensional arrays (tensors) containing multiple channels. Multiplying an image (feature map) with a weight matrix is ​​usually implemented in deep learning frameworks such as TensorFlow, PyTorch, etc. The feature map is considered a multidimensional tensor, while the weight matrix is ​​a two-dimensional array. The multiplication operation is performed according to the rules of matrix multiplication to ensure that the dimensions match to obtain the corresponding query, key, and value matrices.

[0093] In this scheme, after the feature map undergoes linear transformation of the weight matrix, the corresponding query, key, and value information can be effectively extracted, providing a basis for subsequent attention mechanism calculations.

[0094] S103, calculate the relative position code of each feature point of the high-resolution feature map and the low-resolution feature map through a preset image detection model, and add the relative position code to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position code.

[0095] Relative position encoding can be a technique for adding position information to the Attention mechanism. Its main purpose is to make the model pay attention not only to the value of the feature point, but also to the relative spatial relationship between the feature points. In the image detection task, the feature point (i.e., the feature of the image pixel or area) has not only its feature value, but also its position. In the Attention mechanism, adding relative position encoding can help the model better capture the relative spatial position information between these feature points. The traditional Attention mechanism only calculates the similarity through the query matrix and key matrix of the feature point, without directly considering the position information between the feature points. Relative position encoding can enhance the spatial perception ability of the Attention mechanism, so that it can capture the relative position relationship between each feature point when processing high-resolution and low-resolution feature maps.

[0096] A code can be generated for each feature point according to its relative position. Specifically, a relative position code can be generated based on spatial offset. The relative position between feature points can be represented by a distance on a two-dimensional plane (e.g., horizontal and vertical distances). Relative position information is defined for each feature point, such as row and column differences relative to other feature points. The model can also be trained to automatically learn a set of relative position code matrices suitable for the current task. This method gradually adjusts the value of the position code during the training process through optimization algorithms such as gradient descent. After the relative position code is generated, it is added to the key matrix of the high-resolution feature map and the low-resolution feature map. The key matrix is ​​obtained by linear transformation and is used to characterize the correlation between feature points. By adding the relative position code to the key matrix, the model can capture the position information between feature points. The high-resolution feature map and the low-resolution feature map can be processed separately, and their key matrices are added with relative position codes in the same way. Because these two feature maps capture different levels of image information (high resolution for small targets and low resolution for large targets), they need to be processed separately. By adding relative position encoding, key matrices containing relative position encoding of high-resolution feature maps and low-resolution feature maps are obtained. These matrices can then be used for Attention calculation to help the model better understand the spatial relationship between feature points.

[0097] On the basis of the above technical solution, optionally, the relative position codes of the feature points of the high-resolution feature map and the low-resolution feature map are calculated by a preset picture detection model, and the relative position codes are added to the key matrices of the high-resolution feature map and the low-resolution feature map to obtain the key matrices of the high-resolution feature map and the low-resolution feature map containing the relative position codes, including:

[0098] Obtain a preset relative position encoding calculation formula to determine the coordinate information of each feature point of the high-resolution feature map and the low-resolution feature map;

[0099] The relative position coding of each feature point of the high-resolution feature map and the low-resolution feature map is calculated by a preset picture detection model according to the preset relative position coding calculation formula, the coordinate information of each feature point of the high-resolution feature map and the low-resolution feature map; wherein the preset relative position coding calculation formula is:

[0100] R(i, j) = f(p i -p j );

[0101] Among them, R(i, j) is the relative position code of the i-th feature point and the j-th feature point of the high-resolution feature map / low-resolution feature map; p iis the coordinate information of the i-th feature point of the high-resolution feature map / low-resolution feature map; p j is the coordinate information of the jth feature point of the high-resolution feature map / low-resolution feature map; f is a function used to calculate the relative position encoding;

[0102] The relative position codes of the high-resolution feature map and the low-resolution feature map are added to the key matrix to obtain the key matrix containing the relative position codes of the high-resolution feature map and the low-resolution feature map.

[0103] In this solution, the coordinate information may refer to the position of each feature point in the feature map. For high-resolution feature maps and low-resolution feature maps, the coordinates of each feature point are usually represented in a two-dimensional form, such as (i, j), where i represents the row coordinate and j represents the column coordinate.

[0104] For high-resolution feature maps and low-resolution feature maps, the coordinate information of the feature points can be represented as all possible combinations of row and column coordinates. For each pair of feature points (p i , p j ), calculate their coordinate difference p i -p j , apply the function of calculating relative position coding to obtain the relative position coding of each feature point, and then add the calculated relative position coding directly to the corresponding element in the key matrix to form a new key matrix K'. This step will enhance the model's sensitivity to the relative position between feature points.

[0105] In this scheme, the relative position encoding of each feature point can be calculated from the coordinate information of the feature map and added to the key matrix, thereby enhancing the model's understanding of spatial relationships.

[0106] S104, calculate the similarity scores of the feature points in the high-resolution feature map and the low-resolution feature map respectively according to the preset similarity score calculation formula, the key matrix containing the relative position encoding of the high-resolution feature map and the low-resolution feature map through the local head, global head, channel feature head and spatial feature head of the preset image detection model, and multiply the similarity scores with the value matrices of the high-resolution feature map and the low-resolution feature map to obtain the attention fusion feature maps of the high-resolution feature map and the low-resolution feature map in the local head, global head, channel feature head and spatial feature head.

[0107] The local head, global head, channel feature head, and spatial feature head are different focuses in the multi-head attention mechanism. Each head represents information extraction from different angles of the input features, focusing on specific image features or spatial relationships. Among them, the local head can focus on feature extraction in the local area, mainly focusing on adjacent or close feature points, and is usually used to detect details or small objects. This is very useful for processing high-resolution feature maps.

[0108] The global head can focus on global information extraction, capture the relationship between long-distance feature points in the image, and help understand the global context information. It is especially useful for low-resolution feature maps for detecting large objects or full-image context.

[0109] The channel feature head can focus on the channel dimension of each feature map, which is mainly used to capture the relationship between different channels (usually related to color or other feature dimensions) in the feature map.

[0110] The spatial feature head can focus on the relationship between the spatial positions of the image, capture the relative position and relationship of each feature point in the two-dimensional space, and is especially used to extract spatial structure information.

[0111] The similarity score calculation formula may be a mathematical formula for measuring the correlation (or similarity) between feature points.

[0112] The similarity score can be a measure of the similarity between different feature points, indicating the strength of the association between feature points. After calculating the similarity scores, the model can adjust the importance of each feature point based on these scores, and then use this information in the object detection task. The similarity score is often used to weight other feature matrices (such as the value matrix) to highlight important feature points. Specifically, a high similarity score indicates that the relationship between the two feature points is strong. A low similarity score indicates that the relationship between the two feature points is weak.

[0113] Attention weighted features can be obtained by calculating the similarity scores between feature points (the core idea of ​​the attention mechanism), and then weighting the similarity scores with the feature values ​​in the feature map to obtain the feature map. This weighting method can highlight important features related to the target while suppressing irrelevant or insignificant features.

[0114] The key matrix containing relative position encoding of the high-resolution feature map and the low-resolution feature map can be substituted into the preset similarity score calculation formula in the local head, the global head, the channel feature head and the spatial feature head, and the similarity score of each feature point in the high-resolution feature map and the low-resolution feature map can be calculated independently. Specifically, although the similarity score calculation formula of each head is the same, the input features and focus points of each head are different, so it is necessary to calculate the similarity separately. The calculated similarity score is used to weight the features in the value matrix. The higher the similarity, the closer the relationship between the query feature and the key feature, so the greater the weighting of the feature. Then, the matrix multiplication of the similarity score and the value matrix is ​​performed independently on each head to obtain the weighted eigenvalue. By multiplying the similarity score matrix with the value matrix, the final output is the attention weighted features of each feature point in the local, global, channel and spatial dimensions. These features reflect the attention distribution of different heads to the input image features. Specifically, for different heads (local head, global head, channel feature head, spatial feature head), the attention weighted features under each head are calculated separately. Different heads focus on different aspects of information. The local head focuses on the similarity in the local area. The global head focuses on the relationship between feature points in the entire image. The channel feature head focuses on the similarity between different feature channels. The spatial feature head focuses on the relationship between spatial positions.

[0115] Based on the above technical solution, optionally, the preset similarity score calculation formula is:

[0116]

[0117] Among them, A(Q, K′, V) is the similarity score of each feature point in the high-resolution feature map / low-resolution feature map; Q is the query matrix of the high-resolution feature map / low-resolution feature map; K' is the key matrix containing the relative position encoding of the high-resolution feature map / low-resolution feature map; K' T is the transpose of the key matrix containing the relative position encoding of the high-resolution feature map / low-resolution feature map; V is the value matrix of the high-resolution feature map / low-resolution feature map; dk is the dimension of the key matrix of the high-resolution feature map / low-resolution feature map; QK' T is the similarity score matrix, which is the dot product of the query matrix of the high-resolution feature map / low-resolution feature map and the transposed key matrix containing the relative position encoding.

[0118] In this scheme, a key matrix containing relative position encoding of high-resolution feature maps and low-resolution feature maps and a corresponding query matrix and value matrix can be prepared, and then K' can be calculated. T (transpose of the key matrix), perform matrix multiplication QK'T, and divide the similarity score matrix by Scaling is performed to maintain numerical stability, and a softmax function is applied to the similarity score matrix to obtain the attention weight of each feature point. Softmax normalizes the similarity score of each feature point output into a probability distribution. Finally, the attention weight is multiplied by the value matrix V to obtain the weighted feature.

[0119] S105, splicing the attention fusion feature maps of the local head, the global head, the channel feature head and the spatial feature head through a preset image detection model to obtain a spliced ​​feature map, and linearly transforming the spliced ​​feature map to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

[0120] The concatenated feature map can be a unified feature map formed by concatenating different types of attention weighted features (local head, global head, channel feature head, spatial feature head) along a certain dimension (usually the dimension of the feature channel). Each head focuses on different aspects of features, so their weighted features can complement each other. By concatenating these information together, a comprehensive feature map is obtained, which contains more dimensional and hierarchical information.

[0121] The target detection result map can be obtained by performing a linear transformation on the concatenated feature map (usually the operation of a fully connected layer), and then outputting a map containing the category prediction information and position bounding box information of the target object.

[0122] The target category information can represent the category predicted by the model for each detected object in the image. For example, there may be multiple objects in the detection scene (such as cars, pedestrians, buildings, etc.), and the model will predict which category each object belongs to based on the image features.

[0123] The target location bounding box information can describe the specific location of the detected object in the image.

[0124] The attention weighted features of the local head, global head, channel feature head and spatial feature head can be spliced ​​in the feature channel dimension to obtain a spliced ​​feature map, and the spliced ​​feature map is linearly transformed to obtain the target detection result map, which can be achieved through a fully connected layer (or convolutional layer). Specifically, in order to perform linear transformation, a weight matrix and a bias vector can be defined. The weight matrix calculation process is:

[0125] W linear =C concat ×(N classes +4);

[0126] Among them, W linear is the weight matrix; C concat N is the number of feature channels of the concatenated feature map;classes is the number of target categories; 4 is used for the bounding box parameter, which can be (x min ,y min , width, height).

[0127] The bias vector calculation process is:

[0128] b linear =N classes +4;

[0129] The linear transformation is to multiply the concatenated feature map by the weight matrix and add the bias. The formula is as follows:

[0130] F output =F concat ·W linear +b linear ;

[0131] Among them, F output is the target detection result diagram; F concat It is the concatenated feature map.

[0132] Here, the operation is usually performed on the last dimension, that is, the features of each position (i, j) are processed. Specifically, for each position (i, j), the concatenated feature map F concat The features of (i, j,;) are combined with the weight matrix W linear Perform matrix multiplication to obtain a matrix of size (N classes +4) output. After performing the linear transformation, the result obtained is F output The dimensions are:

[0133] height×width×(N classes +4)

[0134] The output of each position contains the target category information: classes The first 4 channels contain the class probability distribution of each object. Bounding box information: The last 4 channels contain the bounding box parameters of the detected object.

[0135] In the embodiment of the present application, by combining high-resolution feature maps and low-resolution feature maps, the model can obtain multi-scale information and improve the detection performance of small targets. By calculating the relative position encoding of feature points and adding them to the key matrix, the detection accuracy can be improved. The attention mechanism of local head, global head, channel feature head and spatial feature head can better focus on the key features of small targets and improve the detection effect of small targets. Linear transformation can improve the response speed and accuracy of small target detection.

[0136] Based on the above technical solution, optionally, after obtaining the target detection result map, the method further includes:

[0137] The target detection result map is transmitted to the control center so that the staff of the control center can understand the on-site situation according to the target detection result map.

[0138] In this scenario, a control center can refer to a centralized management and monitoring location or organization, usually used to process, analyze, and respond to information from different sources. In the context of target detection, the control center may be responsible for monitoring the on-site situation, making decisions, coordinating resources, and ensuring the safety and efficiency of the site.

[0139] The target detection result map (usually an image or video frame) can be formatted into a format suitable for transmission (such as JPEG, PNG, or video format). Then, a network protocol (such as HTTP, FTP, or WebSocket) is used to transmit the target detection result map to a server or cloud platform in the control center.

[0140] Figure 2 A schematic block diagram of an image analysis system for target detection provided in an embodiment of the present disclosure. The system includes:

[0141] The detection module 201 is used to obtain a picture to be detected, input the picture to be detected into a preset picture detection model, and obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points;

[0142] A transformation module 202 is used to transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map;

[0143] The key matrix processing module 203 is used to calculate the relative position code of each feature point of the high-resolution feature map and the low-resolution feature map through a preset picture detection model, and add the relative position code to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position code;

[0144] The attention weighted feature determination module 204 is used to calculate the similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map respectively through the local head, global head, channel feature head and spatial feature head of the preset image detection model according to the preset similarity score calculation formula, the key matrix containing the relative position encoding of the high-resolution feature map and the low-resolution feature map, and multiply the similarity scores with the value matrices of the high-resolution feature map and the low-resolution feature map to obtain the attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, global head, channel feature head and spatial feature head;

[0145] The target detection result map determination module 205 is used to splice the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head through a preset image detection model to obtain a spliced ​​feature map, and linearly transform the spliced ​​feature map to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

[0146] In the embodiment of the present application, by combining high-resolution feature maps and low-resolution feature maps, the model can obtain multi-scale information and improve the detection performance of small targets. By calculating the relative position encoding of feature points and adding them to the key matrix, the detection accuracy can be improved. The attention mechanism of local head, global head, channel feature head and spatial feature head can better focus on the key features of small targets and improve the detection effect of small targets. Linear transformation can improve the response speed and accuracy of small target detection.

[0147] Figure 3 A schematic block diagram of an electronic device 300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0148] The electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a ROM 302 or a computer program loaded from a storage unit 308 into a RAM 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0149] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0150] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the image analysis method applied to target detection. For example, in some embodiments, the image analysis method applied to target detection may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the image analysis method applied to target detection described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the image analysis method applied to target detection in any other appropriate manner (for example, by means of firmware).

[0151] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0156] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0157] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0158] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image analysis method for target detection, characterized in that: The method comprises: Acquire a picture to be detected, input the picture to be detected into a preset picture detection model, and obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points; The high-resolution feature map and the low-resolution feature map are transformed according to a preset query weight matrix, a preset key weight matrix, and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map; Calculate the relative position code of each feature point of the high-resolution feature map and the low-resolution feature map by using a preset image detection model, and add the relative position code to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position code; The local head, global head, channel feature head and spatial feature head of the preset image detection model are used to calculate the similarity scores of the feature points in the high-resolution feature map and the low-resolution feature map according to the preset similarity score calculation formula, the key matrix containing the relative position encoding of the high-resolution feature map and the low-resolution feature map, and the similarity scores are multiplied by the value matrices of the high-resolution feature map and the low-resolution feature map to obtain the attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, the global head, the channel feature head and the spatial feature head; The attention weighted features of the local head, the global head, the channel feature head and the spatial feature head are spliced ​​through a preset image detection model to obtain a spliced ​​feature map, and the spliced ​​feature map is linearly transformed to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

2. The method according to claim 1, characterized in that in, Inputting the image to be detected into a preset image detection model to obtain a high-resolution feature map and a low-resolution feature map of the image to be detected, including: Input the image to be detected into a preset image detection model, and perform convolution and pooling operations on the image to be detected through the Backbone layer of the preset image detection model to obtain an initial high-resolution feature map and an initial low-resolution feature map; The initial high-resolution feature map and the initial low-resolution feature map are processed by a channel attention component of the Attention Module layer of a preset image detection model to obtain a high-resolution channel attention map and a low-resolution channel attention map; Multiplying the high-resolution channel attention map with the initial high-resolution feature map, and multiplying the low-resolution channel attention map with the initial low-resolution feature map to obtain an enhanced high-resolution feature map and an enhanced low-resolution feature map; The enhanced high-resolution feature map and the enhanced low-resolution feature map are processed through the spatial attention component of the Attention Module layer of the preset image detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map; The enhanced high-resolution feature map is multiplied by the initial high-resolution feature map, and the enhanced low-resolution feature map is multiplied by the initial low-resolution feature map to obtain a high-resolution feature map and a low-resolution feature map of the image to be detected.

3. The method according to claim 2, characterized in that in, The initial high-resolution feature map and the initial low-resolution feature map are processed by the channel attention component of the Attention Module layer of the preset image detection model to obtain a high-resolution channel attention map and a low-resolution channel attention map, including: Through the channel attention component of the Attention Module layer of the preset image detection model, global maximum pooling and global average pooling are performed on each channel of the initial high-resolution feature map and the initial low-resolution feature map, respectively, to obtain an initial high-resolution one-dimensional feature map and an initial low-resolution one-dimensional feature map; wherein the initial high-resolution one-dimensional feature map and the initial low-resolution one-dimensional feature map both contain the global maximum pooling value and the global average pooling value of each channel; The global maximum pooling value and the global average pooling value of the initial high-resolution one-dimensional feature map are summed to obtain a first summation result, and the first summation result is normalized by a Sigmoid activation function to obtain a high-resolution channel attention map; The global maximum pooling value and the global average pooling value of the initial low-resolution one-dimensional feature map are summed to obtain a second summation result, and the second summation result is normalized by a Sigmoid activation function to obtain a low-resolution channel attention map.

4. The method according to claim 2, characterized in that: in, The enhanced high-resolution feature map and the enhanced low-resolution feature map are processed through the spatial attention component of the Attention Module layer of the preset image detection model to obtain a high-resolution spatial attention map and a low-resolution spatial attention map, including: Through the spatial attention component of the Attention Module layer of the preset image detection model, global maximum pooling and global average pooling are performed on the pixel values ​​at the same position of the enhanced high-resolution feature map to obtain a first spatial maximum pooling feature map and a first spatial average pooling feature map; Through the spatial attention component of the Attention Module layer of the preset image detection model, global maximum pooling and global average pooling are performed on the pixel values ​​at the same position of the enhanced low-resolution feature map to obtain a second spatial maximum pooling feature map and a second spatial average pooling feature map; The first spatial maximum pooling feature map and the first spatial average pooling feature map are spliced ​​to obtain a first spliced ​​spatial feature map, and the second spatial maximum pooling feature map and the second spatial average pooling feature map are spliced ​​to obtain a second spliced ​​spatial feature map; Convolution operation and Sigmoid activation are performed on the first spliced ​​spatial feature map and the second spliced ​​spatial feature map to obtain a high-resolution spatial attention map and a low-resolution spatial attention map.

5. The method according to claim 1, characterized in that in, The high-resolution feature map and the low-resolution feature map are transformed according to a preset query weight matrix, a preset key weight matrix, and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map, including: Obtain a preset transformation formula, and transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix, a preset value weight matrix, and a preset transformation formula through a preset image detection model to obtain a query matrix, a key matrix, and a value matrix of the high-resolution feature map and the low-resolution feature map; wherein the preset transformation formula is: Q=F·W Q ; K=F·W K ; V=F·W V ; Among them, F is the high-resolution feature map / low-resolution feature map; W Q is the preset query weight matrix; W K is the preset key weight matrix; W V is the preset value weight matrix; Q is the query matrix of high-resolution feature map / low-resolution feature map; K is the key matrix of high-resolution feature map / low-resolution feature map; V is the value matrix of high-resolution feature map / low-resolution feature map.

6. The method according to claim 1, characterized in that in, The relative position codes of the feature points of the high-resolution feature map and the low-resolution feature map are calculated by a preset image detection model, and the relative position codes are added to the key matrices of the high-resolution feature map and the low-resolution feature map to obtain the key matrices of the high-resolution feature map and the low-resolution feature map containing the relative position codes, including: Obtain a preset relative position encoding calculation formula to determine the coordinate information of each feature point of the high-resolution feature map and the low-resolution feature map; The relative position coding of each feature point of the high-resolution feature map and the low-resolution feature map is calculated by a preset picture detection model according to the preset relative position coding calculation formula, the coordinate information of each feature point of the high-resolution feature map and the low-resolution feature map; wherein the preset relative position coding calculation formula is: R(i,j)=f(p i -p j ); Among them, R(i, j) is the relative position code of the i-th feature point and the j-th feature point of the high-resolution feature map / low-resolution feature map; p i is the coordinate information of the i-th feature point of the high-resolution feature map / low-resolution feature map; p j is the coordinate information of the jth feature point of the high-resolution feature map / low-resolution feature map; f is a function used to calculate the relative position encoding; The relative position codes of the high-resolution feature map and the low-resolution feature map are added to the key matrix to obtain the key matrix containing the relative position codes of the high-resolution feature map and the low-resolution feature map.

7. The method according to claim 1, characterized in that in, The preset similarity score calculation formula is: Among them, A(Q, K′, V) is the similarity score of each feature point in the high-resolution feature map / low-resolution feature map; Q is the query matrix of the high-resolution feature map / low-resolution feature map; K' is the key matrix containing the relative position encoding of the high-resolution feature map / low-resolution feature map; K' T is the transpose of the key matrix containing the relative position encoding of the high-resolution feature map / low-resolution feature map; V is the value matrix of the high-resolution feature map / low-resolution feature map; d k is the dimension of the key matrix of the high-resolution feature map / low-resolution feature map; QK' T is the similarity score matrix, which is the dot product of the query matrix of the high-resolution feature map / low-resolution feature map and the transposed key matrix containing the relative position encoding.

8. The method according to claim 1, characterized in that in, After obtaining the fused feature map, the method further includes: The fused feature map is transmitted to a control center so that the staff of the control center can understand the on-site situation according to the fused feature map.

9. An image analysis system for target detection, used to execute the method according to any one of claims 1 to 8, characterized in that: The system comprises: A detection module, used to obtain a picture to be detected, input the picture to be detected into a preset picture detection model, and obtain a high-resolution feature map and a low-resolution feature map of the picture to be detected; wherein the high-resolution feature map and the low-resolution feature map each include at least two feature points; A transformation module, used to transform the high-resolution feature map and the low-resolution feature map according to a preset query weight matrix, a preset key weight matrix and a preset value weight matrix through a preset image detection model to obtain a query matrix, a key matrix and a value matrix of the high-resolution feature map and the low-resolution feature map; A key matrix processing module, used to calculate the relative position code of each feature point of the high-resolution feature map and the low-resolution feature map through a preset image detection model, and add the relative position code to the key matrix of the high-resolution feature map and the low-resolution feature map to obtain the key matrix of the high-resolution feature map and the low-resolution feature map containing the relative position code; An attention weighted feature determination module is used to calculate the similarity scores of each feature point in the high-resolution feature map and the low-resolution feature map respectively through the local head, global head, channel feature head and spatial feature head of the preset image detection model according to the preset similarity score calculation formula, the key matrix containing the relative position encoding of the high-resolution feature map and the low-resolution feature map, and multiply the similarity scores with the value matrices of the high-resolution feature map and the low-resolution feature map to obtain the attention weighted features of the high-resolution feature map and the low-resolution feature map in the local head, global head, channel feature head and spatial feature head; The target detection result map determination module is used to splice the attention weighted features of the local head, the global head, the channel feature head and the spatial feature head through a preset image detection model to obtain a spliced ​​feature map, and linearly transform the spliced ​​feature map to obtain a target detection result map; wherein the target detection result map includes target category information and target position bounding box information.

10. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

Images