Detection result determination method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a detection result determination method and device, a storage medium and electronic equipment, and the method comprises the steps: carrying out the feature extraction of a target image through a plurality of feature extraction networks, so as to obtain a plurality of levels of first image features of the target image; performing feature fusion on the first image features of the plurality of levels to obtain second image features of a plurality of scales; and determining detection results of the target image on different scales according to the second image features of the plurality of scales, and determining a target detection result of the target image according to a plurality of detection results.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a method and apparatus for determining detection results, a storage medium, and an electronic device. Background Technology

[0002] Currently, mainstream technologies for data center monitoring systems rely on fixed sensors and periodic manual inspections. Sensors are primarily used to monitor environmental parameters such as temperature and humidity, while manual inspections focus on physical checks and condition assessments of equipment. However, this manual-based detection method is inefficient and struggles to achieve continuous, uninterrupted monitoring. While sensor detection can provide real-time environmental data, it misses detailed anomalies in equipment. For example, in the initial stages of a fire, traditional temperature sensors and smoke detectors may produce false alarms or missed alarms in certain situations, especially in the early stages when smoke concentration is low or temperature changes are not significant. This severely limits the practicality and reliability of data center security monitoring systems.

[0003] There is currently no effective solution to the problem of low accuracy in sensor-based detection methods in related technologies.

[0004] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0005] This application provides a method and apparatus for determining detection results, a storage medium, and an electronic device to at least solve the problem of low accuracy in sensor-based detection methods in related technologies.

[0006] According to one embodiment of this application, a method for determining detection results is provided, comprising: extracting features from a target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels; performing feature fusion on the first image features at multiple levels to obtain second image features at multiple scales; determining the detection results of the target image at different scales based on the second image features at multiple scales; and determining the target detection result of the target image based on the multiple detection results.

[0007] In one exemplary embodiment, feature extraction is performed on a target image using multiple feature extraction networks to obtain first image features at multiple levels of the target image. This includes: when N is 1, obtaining third image features of the target image extracted by a convolutional layer, and performing feature extraction and feature enhancement on the third image features through an Nth feature extraction network to obtain first image features at the Nth level; when N is not 1, obtaining first image features at the (N-1)th level, and performing feature extraction and feature enhancement on the (N-1)th level first image features through an Nth feature extraction network to obtain first image features at the Nth level, where N is a positive integer.

[0008] In an exemplary embodiment, feature extraction and enhancement of the third image features through an Nth feature extraction network to obtain a first image feature at the Nth level includes: extracting features from the third image features through a first feature extraction unit to determine a fourth image feature; determining a first correlation degree between each image feature in the fourth image features and other image features in the fourth image features through a second feature extraction unit, and determining an updated image feature corresponding to each image feature in the fourth image features based on the first correlation degree; performing an affine transformation on the space corresponding to the fourth image features through the second feature extraction unit to determine a fifth image feature; determining a second correlation degree between each image feature in the fifth image features and other image features in the fifth image features through the second feature extraction unit, and determining an updated image feature corresponding to each image feature in the fifth image features based on the second correlation degree; and determining the first image feature at the Nth level based on the updated fourth image feature and the updated fifth image feature, wherein the Nth feature extraction network includes: the first feature extraction unit and the second feature extraction unit.

[0009] In an exemplary embodiment, feature fusion is performed on the first image features of the multiple layers to obtain second image features of multiple scales. The method further includes: performing a pooling operation on the first image features of the last layer to obtain a sixth image feature; performing feature enhancement on the sixth image feature to obtain an enhanced sixth image feature; and performing feature fusion on the first image features of the multiple layers based on the enhanced sixth image feature to obtain second image features of multiple scales.

[0010] In an exemplary embodiment, feature fusion of the first image features at multiple levels based on the enhanced sixth image feature to obtain second image features at multiple scales includes at least: determining the enhanced sixth image feature as a second image feature at a first scale; upsampling the enhanced sixth image feature to obtain a sampled sixth image feature; fusing the sampled sixth image feature with the first image feature at the last level to obtain a second image feature at a second scale; enhancing the second image feature at the second scale to obtain an enhanced second image feature at a second scale; upsampling the enhanced second image feature at the second scale to obtain a sampled second image feature at a second scale; and fusing the sampled second image feature at the second scale with the first image feature at the penultimate level to obtain a second image feature at a third scale.

[0011] In an exemplary embodiment, determining the detection result of the target image at different scales based on the second image features at the plurality of scales includes: performing feature extraction and feature enhancement on the second image features at each scale to obtain a seventh image feature at each scale; performing feature extraction on the seventh image features at each scale to obtain an eighth image feature at each scale; and determining the detection result of the target image at the corresponding scale based on the eighth image features at the plurality of scales.

[0012] In an exemplary embodiment, before performing feature extraction on a target image using multiple feature extraction networks, the method further includes: determining a loss value of a verification sample in the feature extraction network based on a loss function, wherein the loss function includes: a classification loss function, an equal regularization loss function, an equal cross regularization loss function, a localization loss function, and a confidence loss function; and determining whether the feature extraction network has been trained successfully based on the loss value.

[0013] According to another embodiment of this application, a device for determining detection results is provided, comprising: a feature extraction module, configured to extract features from a target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels; a feature fusion module, configured to fuse the first image features at multiple levels to obtain second image features at multiple scales; and a detection module, configured to determine the detection results of the target image at different scales based on the second image features at multiple scales, and to determine the target detection result of the target image based on the multiple detection results.

[0014] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0015] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0016] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0017] This application utilizes multiple feature extraction networks to extract features from a target image, obtaining first image features at multiple levels. These first image features are then fused to obtain second image features at multiple scales. Based on the second image features at these multiple scales, detection results for the target image at different scales are determined, and the target detection result is determined based on these multiple detection results. In this embodiment, by employing multi-level feature fusion, the network can more comprehensively utilize image information, improving feature resolution and expressiveness, reducing information loss, and effectively detecting targets of different sizes, from small objects to large objects occupying most of the image, thus increasing the breadth of detection. Therefore, it can solve the problem of low accuracy in sensor-based detection methods. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0020] Figure 1 This is a hardware structure block diagram of a computer device for a method of determining detection results according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of a method for determining detection results according to an embodiment of this application;

[0022] Figure 3This is a network framework diagram of the method for determining detection results according to embodiments of this application;

[0023] Figure 4 This is a schematic diagram of the RCM structure according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the DEAM structure according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the DEAM-ELAN structure according to an embodiment of this application;

[0026] Figure 7 This is a LOSS curve diagram according to an embodiment of this application;

[0027] Figure 8 This is a precision curve diagram according to an embodiment of this application;

[0028] Figure 9 This is a PR curve diagram according to an embodiment of this application;

[0029] Figure 10 This is a structural block diagram of a device for determining detection results according to an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0032] The methods and embodiments provided in this application can be executed in a computer device or similar computing device. Taking running on a computer device as an example, Figure 1 This is a hardware structure block diagram of a computer device for a method of determining detection results according to an embodiment of this application. For example... Figure 1 As shown, a computer device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor (MCU) or a field-programmable gate array (FPGA)) and a memory 104 for storing data are also shown. The computer device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the computer device described above. For example, the computer device may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0033] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the method for determining detection results in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to computer devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0034] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer equipment. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0035] This embodiment provides a method for determining the detection result. Figure 2 This is a flowchart of a method for determining the detection result according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0036] Step S202: Feature extraction is performed on the target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels;

[0037] The feature extraction network includes a Convolutional Batch Normalization Sigmoid Linear Unit (CBS) module and a Detection Equivariant Attention Mechanism (DEAM) module. In the feature extraction network, each layer captures information at a specific scale; shallower layers focus on details such as texture and edges, while deeper layers focus on more abstract features, such as the shape or category of an object. Step S202 captures features of different sizes and levels of detail in the target image.

[0038] Step S204: Perform feature fusion on the first image features of the multiple levels to obtain second image features of multiple scales;

[0039] Feature fusion can integrate features from different levels to form a richer and more detailed feature representation.

[0040] Step S206: Determine the detection results of the target image at different scales based on the second image features at multiple scales, and determine the target detection result of the target image based on the multiple detection results.

[0041] After feature fusion, the second image features contain information at multiple scales. The detection network generates detection results at each scale, meaning that this embodiment can identify targets of different sizes. The detection head predicts the location and category of the target based on preset anchor points or candidate regions on the feature map, and can handle objects from small to large. This embodiment includes multiple detection heads, each detecting at different scales, such as 20×20, 40×40, and 80×80.

[0042] The target detection result of the target image is determined based on multiple detection results, including: non-maximum suppression (NMS), which helps the network remove redundant bounding boxes and retain only the most likely detection results. Simultaneously, confidence scores, classification information, and other post-processing steps may be combined to improve the accuracy and completeness of the final result.

[0043] Through the above steps, multiple feature extraction networks are used to extract features from the target image to obtain first image features at multiple levels; these first image features at multiple levels are then fused to obtain second image features at multiple scales; the detection results of the target image at different scales are determined based on the second image features at multiple scales, and the target detection result of the target image is determined based on the multiple detection results. In this embodiment, by utilizing multi-level feature fusion, the network can more comprehensively utilize image information, improve the feature resolution and expressiveness, reduce information loss, and effectively detect targets of different sizes, from small objects to large objects occupying most of the image, thus improving the breadth of detection. Therefore, it can solve the problem of low accuracy in sensor-based detection methods.

[0044] Optionally, feature extraction is performed on the target image through multiple feature extraction networks to obtain first image features at multiple levels of the target image, including: when N is 1, obtaining the third image features of the target image extracted by the convolutional layer, and performing feature extraction and feature enhancement on the third image features through the Nth feature extraction network to obtain the first image features at the Nth level; when N is not 1, obtaining the first image features at the (N-1)th level, and performing feature extraction and feature enhancement on the first image features at the (N-1)th level through the Nth feature extraction network to obtain the first image features at the Nth level, where N is a positive integer.

[0045] In the initial feature extraction, i.e., when N=1, convolutional layers are used to extract the basic features of the target image. After processing by the first feature extraction network (usually a module following a Convolutional Neural Network (CNN), the "first image features of the first level" are formed.

[0046] When dealing with deeper networks, i.e., when N>1, the process becomes iterative. Each level builds upon the output of the previous level, i.e., "the first image feature of the (N-1)th level," to perform deeper feature extraction and enhancement, thereby generating "the first image feature of the Nth level." Regardless of whether N=1 or N>1, the feature extraction network is used for further processing and feature enhancement.

[0047] Through the multi-level feature extraction and enhancement process described above, the features extracted at different levels cover a range from local details to global information, thus effectively handling targets of varying sizes in images and enhancing the flexibility and accuracy of detection and recognition. Furthermore, as the network depth increases, the level of feature abstraction also improves, helping the model learn higher-level patterns and concepts. Simultaneously, feature enhancement (such as residual connections and attention mechanisms) optimizes feature quality and reduces information loss.

[0048] For example, an object detection network processes an image containing communication equipment in a room. First, it performs preliminary feature extraction using standard convolutional layers, followed by feature normalization by the first CBS module. At this point, N=1, the network obtains the third image feature extracted by the convolutional layers and optimizes and enhances this feature using the first feature extraction network (a combination of the CBS and DEAM modules) to obtain the first image feature of the first level. Then, N=2, the network takes the first image feature of the first level as input and passes it through the second feature extraction network (again a combination of CBS and DEAM modules, but with potentially different parameters) for deeper feature extraction and enhancement to obtain the first image feature of the second level. This process is repeated until all predetermined levels N have been processed.

[0049] Optionally, feature extraction and enhancement of the third image features are performed through the Nth feature extraction network to obtain the first image features of the Nth level, including: extracting features from the third image features through the first feature extraction unit to determine the fourth image features; determining the first correlation degree between each image feature in the fourth image features and other image features in the fourth image features through the second feature extraction unit, and determining the updated image features corresponding to each image feature in the fourth image features based on the first correlation degree; performing an affine transformation on the space corresponding to the fourth image features through the second feature extraction unit to determine the fifth image features; determining the second correlation degree between each image feature in the fifth image features and other image features in the fifth image features through the second feature extraction unit, and determining the updated image features corresponding to each image feature in the fifth image features based on the second correlation degree; and determining the first image features of the Nth level based on the updated fourth image features and the updated fifth image features, wherein the Nth feature extraction network includes: the first feature extraction unit and the second feature extraction unit.

[0050] Optionally, the first feature extraction unit further extracts features from the "third image features" to enhance the effectiveness and representativeness of the features, resulting in "fourth image features." Using the second feature extraction unit, a first correlation degree is calculated between each image feature in the "fourth image features" and other features. This first correlation degree can be determined through a self-attention mechanism, correlation calculation, or some form of distance metric. Based on the first correlation degree, each image feature in the "fourth image features" is updated to enhance those features highly correlated with other features while suppressing less correlated features, generating updated image features.

[0051] The second feature extraction unit performs a spatial affine transformation on the "fourth image feature" to generate a transformed feature representation, called the "fifth image feature." The spatial affine transformation can include operations such as rotation, scaling, and translation, used to enhance the model's robustness to changes in image space. The second feature extraction unit then calculates the second correlation degree between each image feature in the "fifth image feature" and other features. Finally, by combining the updated fourth and fifth image features and through comprehensive analysis, the "first image feature of the Nth level" is determined.

[0052] This application's embodiments overcome the limitations of single-scale features by using multi-level feature extraction and enhancement, as well as feature updates and fusion by calculating the correlation between features. This enhances the model's ability to handle targets of different sizes and orientations, while also improving feature selectivity and anti-interference capabilities through correlation calculation.

[0053] For example, in a data center inspection system, the input image first undergoes preliminary feature extraction, and then enters the first-level feature extraction network. In this level, the first feature extraction unit (CBS module) processes the third image features, generating the fourth image features, which contain the basic shape and contour information of the data center equipment. Next, the Region Association Module (RCM) calculates the first correlation degree between each region in the fourth image features and other regions, that is, it measures the feature correlation between each region using cosine similarity.

[0054] Simultaneously, the second feature extraction unit performs a spatial affine transformation on the fourth image features to determine the fifth image features. This process simulates the perspective changes in the computer room environment, helping the model learn equipment features from different angles. In the fifth image features, the Region Correlation Module (RCM) also calculates the second correlation degree between each region and other regions, thereby updating the fifth image features to enhance attention to the transformed key regions.

[0055] Finally, the first image feature of the first level was determined by combining the updated fourth and fifth image features. In subsequent levels, a similar feature extraction and enhancement process was repeated until the first image features of all levels were determined.

[0056] Optionally, feature fusion is performed on the first image features of the multiple levels to obtain second image features of multiple scales. The method further includes: performing pooling operation on the first image features of the last level to obtain a sixth image feature; performing feature enhancement on the sixth image feature to obtain an enhanced sixth image feature; and performing feature fusion on the first image features of the multiple levels based on the enhanced sixth image feature to obtain second image features of multiple scales.

[0057] In this embodiment, firstly, pooling is performed on the first image features extracted from the last layer to generate a "sixth image feature". Pooling operations (such as max pooling, average pooling, etc.) are used to reduce the size of the feature map, reduce computation, and retain important feature information. The "sixth image feature" is then enhanced to generate an "enhanced sixth image feature". Feature enhancement can be achieved by adding more convolutional layers, using more complex activation functions, or introducing attention mechanisms. Finally, the "enhanced sixth image feature" is fused with the "first image features from multiple layers" to generate "second image features at multiple scales". The fusion process can include skip connections, upsampling, feature concatenation, etc.

[0058] Optionally, feature fusion is performed on the first image features of the multiple levels based on the enhanced sixth image features to obtain second image features of multiple scales, including at least: determining the enhanced sixth image feature as the second image feature of the first scale; upsampling the enhanced sixth image feature to obtain the sampled sixth image feature; fusing the sampled sixth image feature with the first image feature of the last level to obtain the second image feature of the second scale; enhancing the second image feature of the second scale to obtain the enhanced second image feature of the second scale; upsampling the enhanced second image feature of the second scale to obtain the sampled second image feature of the second scale; and fusing the sampled second image feature of the second scale with the first image feature of the penultimate level to obtain the second image feature of the third scale.

[0059] Optionally, the "enhanced sixth image feature" is upsampled to increase the size of its feature map, generating a "sampled sixth image feature." Upsampling helps recover some spatial information. The "sampled sixth image feature" is then fused with the "first image feature of the last level" to generate the "second image feature at the second scale." Fusion can be achieved by concatenating feature maps or other operations to integrate feature information from different levels, resulting in a richer and more semantically hierarchical representation. Iteratively, the "second image feature at the second scale" is enhanced, then upsampled again, and fused with the "first image feature of the penultimate level." This process is repeated, with each fusion followed by feature enhancement and upsampling, thereby recovering more spatial details.

[0060] In this embodiment, deep features (enhanced sixth image features) are used as a starting point, and then fused and enhanced with feature maps from different levels. This results in the construction of a feature representation system capable of comprehensively processing information at different scales in an image.

[0061] Optionally, determining the detection results of the target image at different scales based on the second image features at the multiple scales includes: performing feature extraction and feature enhancement on the second image features at each scale to obtain a seventh image feature at each scale; performing feature extraction on the seventh image features at each scale to obtain an eighth image feature at each scale; and determining the detection results of the target image at the corresponding scale based on the eighth image features at the multiple scales.

[0062] In this embodiment, the second image features at each scale are further extracted and enhanced to refine the feature representation at each scale, generating a more accurate "seventh image feature". The "seventh image feature" is then used as input for higher-order feature extraction to obtain the "eighth image feature". This higher-order feature extraction, using the "seventh image feature" as input, includes feature extraction using a CBS module and a convolutional (CONV) module.

[0063] Based on the "eighth image feature at each scale," the model can generate detection results for target images at the corresponding scales, including the target's location, size, and category information. In other words, the model can predict and identify targets at different scales, thereby enhancing the detection capability for small, medium, and large targets in images and addressing the challenges to recognition accuracy posed by the diversity of target sizes and scale variations.

[0064] Optionally, before performing feature extraction on the target image through multiple feature extraction networks, the method further includes: determining the loss value of the verification sample in the feature extraction network according to a loss function, wherein the loss function includes: a classification loss function, an equal regularization loss function, an equal cross regularization loss function, a localization loss function, and a confidence loss function; and determining whether the feature extraction network has been trained successfully based on the loss value.

[0065] The classification loss function is used to measure the difference between the model's prediction and the actual label. In object detection tasks, the classification loss function ensures that the model can correctly distinguish different object categories.

[0066] Isovariant regularization loss function: used to enhance the model's invariance or equivariance to geometric transformations of the image (such as rotation, scaling, deformation), that is, the detection results should remain consistent and not be affected by spatial transformations of the input image.

[0067] Equivariant cross-regularization loss function: Similar to equivariant regularization loss, but going a step further, it optimizes the model by comparing the differences in features between the original input and the transformed input, ensuring that the features learned by the model maintain consistent correlation before and after the geometric transformation, thereby improving the model's generalization ability.

[0068] Localization loss functions, such as CIoU Loss, are specifically designed to optimize bounding box regression, ensuring that the bounding box positions predicted by the model coincide as closely as possible with the actual object positions.

[0069] Confidence loss function: Used to train the model's output target confidence score. In object detection, the model not only needs to locate objects but also provide a confidence score for the detection results. This loss function helps the model learn to provide more reliable results and reduce false positives.

[0070] Before training the feature extraction network, loss values ​​for validation samples are calculated based on a set of loss functions to evaluate the model's current performance. If the values ​​of these loss functions reach a preset threshold or meet a termination condition (e.g., no longer significantly decreasing over several consecutive training epochs), the feature extraction network is considered to have been successfully trained and can be used for subsequent feature extraction tasks.

[0071] To better understand the process of determining the above detection results, the implementation flow of the above detection result determination method will be described below in conjunction with optional embodiments, but it is not intended to limit the technical solution of the embodiments of this application.

[0072] This embodiment provides a network framework diagram for a method to determine detection results. Figure 3 This is a network framework diagram of the detection result determination method according to the embodiments of this application, such as... Figure 3 As shown, it specifically includes:

[0073] Feature extraction networks include:

[0074] 1. Convolutional Layer (CONV) is used to extract local features.

[0075] 2. CBS, which contains modules for convolution, batch normalization, and activation functions (such as SiLU or ReLU) for further processing of features.

[0076] 3. DEAM: A module containing attention mechanisms to enhance feature representations, which may include multi-scale feature fusion and attention weighting.

[0077] It should be noted that DEAM includes an RCM module, such as... Figure 4 As shown, the RCM module performs two 1×1 convolutions on the original feature map size H×W×C to obtain H×W×C1 and H×W×C2 feature maps in sequence. Then, the H×W×C2 feature map is reshaped into a C2×HW matrix, and the processed activation mapping H×W×C is determined based on the C2×HW feature map and the H×W×C feature map.

[0078] The RCM module uses cosine similarity as a metric to measure the intrinsic correlation between features in different regions. In the normalized feature space, it evaluates the correlation strength between region i and other regions by calculating the dot product, effectively identifying the differences between key regions and the background. The introduction of the ReLU activation function suppresses negative similarity weights, ensuring that the final optimized features are the weighted sum of the original region features and the normalized similarity. This strategy guarantees the positive representation of features and enhances the sensitivity to target regions.

[0079] Optionally, in this embodiment, a DEAM module is designed to enhance feature extraction based on the RCM module. This module enhances feature representation through an equal-variable attention mechanism. The feature stream then passes through the second DEAM module and is divided into two paths: the main path continues to optimize features through the CBS module, while the branch path achieves feature fusion by concatenating the features of the main path with those of the fusion module.

[0080] 4. Spatial Pyramid Feature Fusion (SPFF): This technique simulates spatial pyramid pooling on feature maps, enhancing the receptive field of the feature maps.

[0081] 5. Fusion Module (CONCAT): This module is used to stitch feature maps from different layers together along the channel dimension, fusing information at different scales.

[0082] 6. Upsampling module (UPSAMPLE): Upsampling operation is used to increase the spatial resolution of the feature map so that it can be stitched together with feature maps from shallower layers.

[0083] The detection head includes:

[0084] DEAM-ELAN: A module that combines DEAM and ELAN (Efficient Layer Aggregation Network) for further processing and aggregation of features.

[0085] CBS+CONV: Contains modules for convolution and batch normalization, used to convert feature maps into the final output format.

[0086] The final detection result is output, including the target's category, location (bounding box), and confidence score. It typically consists of a series of convolutional layers, each corresponding to a specific scale (e.g., 20x20, 40x40, 80x80), with an output dimension of 85 (4 bounding box parameters + 1 confidence score + 80 class probabilities). A 20x20 scale detector head detects large targets; a 40x40 scale detector head captures medium-sized targets using intermediate-scale features; and an 80x80 scale detector head detects small targets.

[0087] This application embodiment realizes a complete processing flow from low-level feature extraction to high-level semantic parsing. Through the equivariant attention mechanism of the DEAM module and the efficient feature aggregation capability of the ELAN module, the network has excellent multi-scale target detection performance.

[0088] Self-attention is a widely accepted neural network optimization method that can significantly improve the network's representational power. This mechanism modifies feature maps by capturing the dependencies between contextual features. This characteristic aligns perfectly with the idea behind most weakly supervised learning methods that utilize pixel similarity to optimize the original activation map. A standard self-attention mechanism can be defined as follows:

[0089] , Where x and y represent the input and output features, respectively, and i and j represent the spatial indexes of the features. The output signal is normalized by a normalization factor. Perform standardization. (Function) Used to generate input signals Feature representations at each location, these feature representations will be based on The calculated similarity weights are aggregated at position i, where the similarity weights are calculated using the dot product pixel affinity in the embedding space. and : These are the input features and The embedding transformation (usually implemented through linear or convolutional layers) maps them to a new feature space. Dot product Calculate the similarity between two embedded features. The exponential function e is used to convert the similarity into non-negative weights, and is often used in attention mechanisms.

[0090] While equivariance regularization provides additional supervision for network learning, achieving ideal equivariance using only traditional convolutional layers is difficult. This application employs a self-attention mechanism as an effective context modeling module to optimize region proposal and bounding box prediction. Adapting the classic self-attention mechanism described above to the object detection task can be expressed as:

[0091] , Represents the original regional features. This represents the optimized features.

[0092] To further leverage contextual information to optimize region features, a region association module was designed at the end of the detection network. This module is an improvement upon the core idea of ​​the self-attention mechanism and is subject to supervision by equivariant regularization. Cosine similarity is used to measure the correlation between region features:

[0093] By calculating the inner product in the normalized feature space, the correlation strength between the current region i and other regions can be accurately assessed. Integrating this similarity calculation into the above formula, the improved expression is:

[0094] The ReLU activation function is used to suppress negative similarity, and the final optimized feature is a weighted sum of the original region features and the normalized similarity.

[0095] In conventional fully supervised object detection data augmentation, the input image and its corresponding bounding box annotations undergo the same affine transformation (such as scaling and rotation) simultaneously. This process provides the detection network with an implicit equivariant learning signal. However, in weakly supervised object detection tasks, since only image-level category labels are available, this equivariant constraint mechanism is lost. The object detection network F( Simultaneously output the classification confidence score Fcls(I) and the bounding box coordinates Freg(I). The improved dual-task isotropic constraint is written as:

[0096] Where F(·) represents the detection network, T(·) represents any spatial affine transformation, and λcls and λreg are task balancing coefficients, defaulted to 1:2. A symmetric network structure with shared weights is constructed: one branch performs a geometric transformation on the network's output detection result, while the other branch applies the same transformation to the input image before the network's forward computation. By constraining the consistency of the output results of the two branches, the network can spontaneously learn the geometric invariance of the detection results. Unlike semantic segmentation tasks, object detection requires ensuring equivariance between class prediction and bounding box regression, which places higher demands on the design of regularization strategies. Therefore, equivariance constraints are applied at multiple scales of the feature extraction network to ensure stable detection performance for targets of different sizes.

[0097] Affine transformations of bounding box coordinates require matrix operations. Let the transformation matrix be M, then: .

[0098] To eliminate the influence of image size, a normalization term is added:

[0099] Where W×H is the feature map size.

[0100] In particular, to address the multi-scale problem commonly encountered in object detection, this application deploys an RCM module at each layer of the feature extraction network. Through cross-layer feature interactions, the model can better handle targets of different sizes.

[0101] In the field of object detection, image-level classification labels *l* are the only available manually labeled supervision information. A global average pooling layer is used at the end of the detection network to obtain the image classification prediction vector *z*, and a multi-label soft-margin loss function is used for network training. The classification loss for C-1 foreground object categories is defined as:

[0102] .

[0103] Among them, such as Figure 5 As shown, in the symmetric network architecture, the region proposal features output by the original image branch and the transformed image branch are denoted as yo' and yt', respectively. Through global average pooling, these two features are aggregated into prediction vectors zo and zt, respectively. The classification loss applies to both branches simultaneously.

[0104] This classification loss provides a basic supervisory signal for target localization. To ensure spatial consistency of the detection results, an isovariant regularization constraint is applied to the original region features. The isovariant regularization (ER) loss is defined as:

[0105] , where T( The ) indicates the affine transformation applied to the input image in the transformation branch. To further enhance the network's equivariant learning ability, shallow features and original region features are input to the Region Association Module (RCM) for optimization. The initial idea was to establish equivariant constraints between the optimized region features yo and yt, but experiments showed that the RCM output easily gets trapped in local optima (i.e., all regions are predicted to be of the same category). Therefore, an equivariant cross-regularization (ECR) loss is proposed: .

[0106] Ultimately, the overall loss function of the detection network consists of five parts:

[0107] Among them, classification loss Used for coarse target localization, L ER L is used to bridge the gap between region-level and image-level supervision. ECR The RCM module is then deeply integrated with the backbone network to ensure consistent predictions across different geometric transformations. Localization loss L BBox The CIoU Loss is used, which has the advantage of further considering center point distance and aspect ratio consistency. The confidence loss is the BCE Loss, but it distinguishes between positive and negative samples.

[0108] This application's embodiments introduce an equivariant attention mechanism and an ELAN structure into the target detection network architecture, such as... Figure 6 As shown, the ELAN mechanism mainly consists of multiple convolutional + BN + SiLU activation functions, i.e., CBS. The input feature map has c channels. The input feature map is processed in two parts. One part is directly passed through a convolutional layer (Conv + BN + SiLU), with an output of 0.5c channels. The other part is passed through four consecutive convolutional layers (each followed by BN and SiLU activation functions), with each convolutional layer having an output of 0.25c channels. The feature maps processed by these two parts are concatenated along the channel dimension to form a new feature map with 2c channels. The concatenated feature map is further processed through a convolutional layer (Conv + BN + SiLU) to integrate the features. Finally, it passes through the DEAM (Dense EfficientAttention Module), which contains an attention mechanism to enhance the expressive power of the features.

[0109] The detection head employs a task-specific attention branch to achieve collaborative optimization of classification and regression tasks, and improves multi-scale target detection performance through adaptive receptive field adjustment. The improved network architecture demonstrates enhanced detection stability under complex lighting conditions, achieving a significant improvement in detection accuracy while ensuring real-time performance, providing reliable technical support for target detection applications in complex scenarios.

[0110] Optionally, the programming language used in this application can be Python 3.12, the deep learning framework can be PyTorch 2.6, and the libraries involved can include OpenCV, NumPy, etc. The main training parameters are set to 8 classifications, and the number of training epochs is 100. 100 epochs is a relatively reasonable choice, ensuring model convergence while avoiding overfitting. A batch size of 16 ensures high training efficiency without exceeding the GPU's computing power limits.

[0111] Regarding model convergence, such as Figure 7 As shown, due to the RCM module proposed in this embodiment, the model exhibits excellent convergence performance. Specifically, the loss function value decreases rapidly within the first 50 training epochs and reaches a global minimum of 0.023 in the 98th epoch, indicating that the model has efficient learning capabilities.

[0112] In terms of detection accuracy, such as Figure 8 As shown, thanks to the improvement in the equal-variance regularization strategy, the mAP@0.5 metric continuously improved, finally reaching a peak of 0.881 in the 99th round, demonstrating the model's excellent object detection capability. It is particularly noteworthy that, as... Figure 9 As shown, the PR curve exhibits a near-ideal "high convexity" shape, with an area under the curve reaching 0.87. This indicates that the model maintains high precision while still achieving a high recall. When the recall is 0.8, the model's precision remains above 0.85.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0114] This embodiment also provides a device for determining detection results, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0115] Figure 10 This is a structural block diagram of the device for determining the detection results according to an embodiment of this application, such as... Figure 10 As shown, the device includes:

[0116] The feature extraction module 1002 is used to extract features from the target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels;

[0117] Feature fusion module 1004 is used to perform feature fusion on the first image features of the multiple levels to obtain second image features of multiple scales;

[0118] The detection module 1006 is used to determine the detection results of the target image at different scales based on the second image features at multiple scales, and to determine the target detection result of the target image based on the multiple detection results.

[0119] The aforementioned apparatus extracts features from a target image using multiple feature extraction networks to obtain first image features at multiple levels. These first image features are then fused to obtain second image features at multiple scales. Based on the second image features at these multiple scales, the detection results of the target image at different scales are determined, and the target detection result is determined based on these multiple detection results. In this embodiment, by utilizing multi-level feature fusion, the network can more comprehensively utilize image information, improve the resolution and expressiveness of features, reduce information loss, and effectively detect targets of different sizes, from small objects to large objects occupying most of the image, thus increasing the breadth of detection. Therefore, it can solve the problem of low accuracy in sensor-based detection methods.

[0120] In an exemplary embodiment, the feature extraction module 1002 is configured to, when N is 1, obtain the third image feature of the target image extracted by the convolutional layer, and perform feature extraction and feature enhancement on the third image feature through the Nth feature extraction network to obtain the first image feature of the Nth level; when N is not 1, obtain the first image feature of the (N-1)th level, and perform feature extraction and feature enhancement on the first image feature of the (N-1)th level through the Nth feature extraction network to obtain the first image feature of the Nth level, where N is a positive integer.

[0121] In an exemplary embodiment, the feature extraction module 1002 is configured to: extract features from the third image features using a first feature extraction unit to determine a fourth image feature; determine a first correlation degree between each image feature in the fourth image features and other image features in the fourth image features using a second feature extraction unit, and determine an updated image feature corresponding to each image feature in the fourth image features based on the first correlation degree; perform an affine transformation on the space corresponding to the fourth image features using the second feature extraction unit to determine a fifth image feature; determine a second correlation degree between each image feature in the fifth image features and other image features in the fifth image features using the second feature extraction unit, and determine an updated image feature corresponding to each image feature in the fifth image features based on the second correlation degree; and determine a first image feature of the Nth level based on the updated fourth image feature and the updated fifth image feature, wherein the Nth feature extraction network includes: the first feature extraction unit and the second feature extraction unit.

[0122] In an exemplary embodiment, the feature fusion module 1004 is configured to perform pooling operation on the first image features of the last level to obtain a sixth image feature; perform feature enhancement on the sixth image feature to obtain an enhanced sixth image feature; and perform feature fusion on the first image features of the multiple levels based on the enhanced sixth image feature to obtain second image features of multiple scales.

[0123] In an exemplary embodiment, the feature fusion module 1004 is configured to: determine that the enhanced sixth image feature is a second image feature at a first scale; upsample the enhanced sixth image feature to obtain a sampled sixth image feature; fuse the sampled sixth image feature with the first image feature at the last level to obtain a second image feature at a second scale; enhance the second image feature at the second scale to obtain an enhanced second image feature at the second scale; upsample the enhanced second image feature at the second scale to obtain a sampled second image feature at the second scale; and fuse the sampled second image feature at the second scale with the first image feature at the penultimate level to obtain a second image feature at a third scale.

[0124] In an exemplary embodiment, the detection module 1006 is configured to perform feature extraction and feature enhancement on the second image features at each scale to obtain the seventh image features at each scale; perform feature extraction on the seventh image features at each scale to obtain the eighth image features at each scale; and determine the detection result of the target image at the corresponding scale based on the eighth image features at the multiple scales.

[0125] In an exemplary embodiment, the above apparatus further includes: a training module 1006, configured to determine the loss value of the verification sample in the feature extraction network according to a loss function, wherein the loss function includes: a classification loss function, an equal regularization loss function, an equal cross regularization loss function, a localization loss function, and a confidence loss function; and to determine whether the feature extraction network has been trained successfully based on the loss value.

[0126] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0127] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0128] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0129] S1, feature extraction is performed on the target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels;

[0130] S2, perform feature fusion on the first image features of the multiple levels to obtain second image features of multiple scales;

[0131] S3, determine the detection results of the target image at different scales based on the second image features at the multiple scales, and determine the target detection result of the target image based on the multiple detection results.

[0132] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0133] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0134] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0135] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0136] S1, feature extraction is performed on the target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels;

[0137] S2, perform feature fusion on the first image features of the multiple levels to obtain second image features of multiple scales;

[0138] S3, determine the detection results of the target image at different scales based on the second image features at the multiple scales, and determine the target detection result of the target image based on the multiple detection results.

[0139] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0140] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0141] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0142] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0143] S1, feature extraction is performed on the target image through multiple feature extraction networks to obtain first image features of the target image at multiple levels;

[0144] S2, perform feature fusion on the first image features of the multiple levels to obtain second image features of multiple scales;

[0145] S3, determine the detection results of the target image at different scales based on the second image features at the multiple scales, and determine the target detection result of the target image based on the multiple detection results.

[0146] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0147] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0148] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method of determining a result of a test, characterized by, The method comprises the following steps: performing feature extraction on a target image through a plurality of feature extraction networks to obtain a plurality of levels of first image features of the target image; performing feature fusion on the plurality of levels of first image features to obtain a plurality of scales of second image features; determining detection results of the target image at different scales according to the plurality of scales of second image features, and determining a target detection result of the target image according to a plurality of detection results.

2. The method of claim 1, wherein, performing feature extraction on a target image through a plurality of feature extraction networks to obtain a plurality of levels of first image features of the target image, comprising: in the case that N is 1, obtaining third image features of the target image extracted by a convolutional layer, and performing feature extraction and feature enhancement on the third image features through an Nth feature extraction network to obtain first image features of an Nth level; in the case that N is not 1, obtaining first image features of an N-1th level, and performing feature extraction and feature enhancement on the first image features of the N-1th level through an Nth feature extraction network to obtain first image features of an Nth level, wherein N is a positive integer.

3. The method of claim 2, wherein, performing feature extraction and feature enhancement on the third image features through the Nth feature extraction network to obtain first image features of an Nth level, comprising: performing feature extraction on the third image features through a first feature extraction unit to determine fourth image features; determining, through a second feature extraction unit, a first correlation degree between each image feature in the fourth image features and other image features in the fourth image features, and determining an updated image feature corresponding to each image feature in the fourth image features according to the first correlation degree; and, performing spatial affine transformation on the fourth image features through a second feature extraction unit to determine fifth image features; determining, through the second feature extraction unit, a second correlation degree between each image feature in the fifth image features and other image features in the fifth image features, and determining an updated image feature corresponding to each image feature in the fifth image features according to the second correlation degree; determining first image features of an Nth level according to the updated fourth image features and the updated fifth image features, wherein the Nth feature extraction network comprises the first feature extraction unit and the second feature extraction unit.

4. The method of claim 2, wherein, performing feature fusion on the plurality of levels of first image features to obtain a plurality of scales of second image features, the method further comprising: performing a pooling operation on first image features of a last level to obtain sixth image features; performing feature enhancement on the sixth image features to obtain enhanced sixth image features; performing feature fusion on the plurality of levels of first image features according to the enhanced sixth image features to obtain a plurality of scales of second image features.

5. The method of claim 4, wherein, performing feature fusion on the plurality of levels of first image features according to the enhanced sixth image features to obtain a plurality of scales of second image features, at least comprising: determining the enhanced sixth image features as second image features of a first scale; up-sample the enhanced sixth image feature to obtain a sampled sixth image feature; perform feature fusion on the sampled sixth image feature and the first image feature of the last level to obtain a second image feature of a second scale; perform feature enhancement on the second image feature of the second scale to obtain an enhanced second image feature of the second scale; up-sample the enhanced second image feature of the second scale to obtain a sampled second image feature of the second scale; perform feature fusion on the sampled second image feature of the second scale and the first image feature of a penultimate level to obtain a second image feature of a third scale.

6. The method of claim 1, wherein, determining, according to the second image features of the multiple scales, detection results of the target image at different scales, comprising: performing feature extraction and feature enhancement on the second image feature of each scale to obtain a seventh image feature of each scale; performing feature extraction on the seventh image feature of each scale to obtain an eighth image feature of each scale; determining, according to the eighth image features of the multiple scales, detection results of the target image at corresponding scales.

7. The method of claim 1, wherein, Before the feature extraction network extracts features of the target image, the method further comprises: determining, according to a loss function, a loss value of a verification sample in the feature extraction network, wherein the loss function comprises a classification loss function, an equivariant regularization loss function, an equivariant cross-regularization loss function, a positioning loss function, and a confidence loss function; determining, according to the loss value, whether the feature extraction network is trained.

8. A determination device of a detection result, characterized by, comprise: a feature extraction module configured to extract features of a target image by using multiple feature extraction networks to obtain first image features of multiple levels of the target image; a feature fusion module configured to perform feature fusion on the first image features of the multiple levels to obtain second image features of multiple scales; a detection module configured to determine detection results of the target image at different scales according to the second image features of the multiple scales, and determine a target detection result of the target image according to multiple detection results.

9. A computer readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 by using the computer program.

Images