Method and device for constructing a real-time target detection model with low computing power

By building a space pooling sub-model and channel attention calculation with parallel structures, the real-time detection problem of Yolo algorithm in low computing power situations is solved, real-time and accurate object detection on low computing power chips is achieved, and the high-end Graphics card dependency.

CN118710888BActive Publication Date: 2024-11-15HANGZHOU XINZHENG MICROELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411178830.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-15
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

The latest Yolo algorithm uses SiLu activation function and DFL calculation, resulting in large calculation volume, requiring high-end graphics cards to support real-time detection, and high-end graphics cards are expensive, making it difficult to achieve real-time and accurate target detection under low computing power.

Method used

By building a spatial pooling sub-model of parallel structure, the feature expression ability is enhanced, and the channel attention calculation is used to reduce the calculation amount of the Softmax operator, the feature extraction, spatial pooling, upsampling and aggregation sub-models are constructed, combined with the identification detection head, Real-time object detection.

Benefits of technology

Real-time and accurate object detection is achieved on low-computing chips, reducing computing needs and avoiding the dependence of high-end graphics cards.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118710888B_ABST
    Figure CN118710888B_ABST
Patent Text Reader

Abstract

The present application proposes a method and device for constructing a low-computing real-time target detection model, including the following steps: constructing a real-time target detection model including a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model, and a recognition detection head, and training the real-time target construction model so that it can be used to detect the recognition target in the image to be detected. This solution enhances the expressiveness of features by constructing a spatial pooling sub-model with a parallel structure, and uses channel attention calculation to reduce the impact of the Softmax operator in the conventional multi-head attention mechanism on the amount of calculation, thereby achieving the purpose of accurately detecting targets in real time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of deep learning, and in particular to a method and device for constructing a low-computing-power real-time target detection model. Background Art

[0002] At present, the target detection and recognition method based on deep learning has become the mainstream for realizing automatic detection and intelligent recognition. In particular, the target detection framework of the Yolo series has been widely used in transportation, industry, military and other fields. With the support of INVIDIA graphics cards, various target detection and recognition algorithms based on deep learning have excellent application and performance. The update of Yolo algorithm is also moving towards lightweight and high-precision. However, since the network structure of the latest Yolov8, Yolov9, Yolov10 and other network models uses SiLu as the activation function, and the post-processing uses DFL (Distribution Focal Loss) to calculate the detection box, and the attention head in DFL and Yolov10 contains SoftMax operator, the calculation amount of the model is greatly increased, so high-end INVIDIA graphics cards are needed to support real-time detection operations. High-end INVIDIA graphics cards are expensive and are not a regular choice for the general public.

[0003] Therefore, how to improve the latest Yolo algorithm so that it can still perform real-time and accurate target detection under low computing power is a difficult problem that existing technologies urgently need to solve. Summary of the invention

[0004] The embodiments of the present application provide a method and device for constructing a real-time target detection model with low computing power, which enhances the expressiveness of features by constructing a spatial pooling sub-model with a parallel structure, and uses channel attention calculation to reduce the impact of the Softmax operator in the conventional multi-head attention mechanism on the amount of calculation, thereby achieving the purpose of accurately detecting targets in real time.

[0005] In a first aspect, an embodiment of the present application provides a method for constructing a real-time target detection model with low computing power, the method comprising:

[0006] Constructing a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales;

[0007] Constructing a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then performing channel attention processing on the pooling feature map to obtain a spatial pooling result;

[0008] An upsampling sub-model is constructed, where the upsampling sub-model is trained to concatenate the spatial pooling result with sub-feature maps of different scales in an upsampling manner using a plurality of upsampling units to obtain an upsampling result;

[0009] Constructing an aggregation sub-model, where the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result;

[0010] Building a recognition detection head, wherein the recognition detection head is trained to obtain target category, location and confidence information in each aggregation result;

[0011] Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

[0012] In a second aspect, an embodiment of the present application provides a device for constructing a real-time target detection model with low computing power, including:

[0013] A feature extraction sub-model construction module is used to construct a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales;

[0014] A spatial pooling sub-model construction module is used to construct a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then process the pooling feature map by channel attention to obtain a spatial pooling result;

[0015] An upsampling sub-model construction module, used to construct an upsampling sub-model, wherein the upsampling sub-model is trained to use multiple upsampling units to splice the spatial pooling result with sub-feature maps of different scales in an upsampling manner to obtain an upsampling result;

[0016] An aggregation sub-model construction module, used to construct an aggregation sub-model, wherein the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result;

[0017] A recognition detection head construction module, used to construct a recognition detection head, wherein the recognition detection head is trained to obtain target category, position and confidence information in each aggregation result;

[0018] Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for constructing a real-time target detection model with low computing power.

[0020] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a method for constructing a real-time target detection model with low computing power.

[0021] The main contributions and innovations of the present invention are as follows:

[0022] The embodiment of the present application constructs a spatial pooling sub-model, and enhances the feature expression capability and improves the accuracy of target detection by connecting a second pooling convolution layer, a maximum pooling layer, and an average pooling layer in parallel in the spatial pooling sub-model; in the spatial pooling sub-model, channel attention is calculated to avoid the influence of the SoftMax operator in the original multi-head attention mechanism on the amount of calculation, so that the present solution can achieve real-time detection even in chips with low computing power.

[0023] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 is a flowchart of a method for constructing a low-computing-power real-time target detection model according to an embodiment of the present application;

[0026] Figure 2 is a structural schematic diagram of a spatial pooling sub-model according to an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of the structure of an attention module according to an embodiment of the present application;

[0028] Figure 4is a structural schematic diagram of a channel attention unit according to an embodiment of the present application;

[0029] Figure 5 is a schematic diagram of the structure of an aggregation sub-model according to an embodiment of the present application;

[0030] Figure 6 is a schematic diagram of the structure of a double-layer convolution unit according to an embodiment of the present application;

[0031] Figure 7 is a schematic diagram of the overall structure of a head network of a real-time target detection model according to an embodiment of the present application;

[0032] Figure 8 It is a structural block diagram of a device for constructing a low-computing-power real-time target detection model according to an embodiment of the present application;

[0033] Fig. 9 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0035] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0036] Embodiment 1

[0037] The embodiment of the present application provides a method for constructing a low-computing-power real-time target detection model, which enhances the expressiveness of features by constructing a spatial pooling sub-model with a parallel structure, and uses channel attention calculation to reduce the influence of the Softmax operator in the conventional multi-head attention mechanism on the amount of calculation, thereby achieving the purpose of accurately detecting targets in real time. Specifically, reference Figure 1 , the method comprising:

[0038] Constructing a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales;

[0039] Constructing a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then performing channel attention processing on the pooling feature map to obtain a spatial pooling result;

[0040] An upsampling sub-model is constructed, where the upsampling sub-model is trained to concatenate the spatial pooling result with sub-feature maps of different scales in an upsampling manner using a plurality of upsampling units to obtain an upsampling result;

[0041] Constructing an aggregation sub-model, where the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result;

[0042] Building a recognition detection head, wherein the recognition detection head is trained to obtain target category, location and confidence information in each aggregation result;

[0043] Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

[0044] In some specific embodiments, the feature extraction sub-model adopted in this solution is the backbone network framework of Yolov7. The Yolov7 source code is downloaded from the open source website, and a new network structure configuration is created as the initial Yolov7 backbone framework.

[0045] In some specific embodiments, the present solution trains the real-time object detection model by annotating the image data in the coco dataset as training samples, and sets the category and anchor box parameters of the dataset.

[0046] In this scheme, the structural diagram of the spatial pooling sub-model is as follows Figure 2 As shown in the figure, the spatial pooling sub-model uses a parallel second pooling convolution layer, a maximum pooling layer, and an average pooling layer to enhance the feature representation capability, which can improve the recognition accuracy.

[0047] Furthermore, in the spatial pooling sub-model, the first pooling convolution layer is used to convolve the sub-feature map with the smallest scale, and then the convolutions are input into the second pooling convolution layer, the maximum pooling layer and the average pooling layer in parallel respectively. The output features of the first pooling convolution layer, the second pooling convolution layer, the maximum pooling layer and the average pooling layer are concatenated and input into the third pooling convolution layer to obtain the pooling feature map.

[0048] Specifically, in the spatial pooling sub-model, the first pooling convolution layer, the second pooling convolution layer and the third pooling convolution layer all adopt the CBL convolution structure, that is, the Conv2D+Bn+LeakReLU structure, the convolution kernel size of the first pooling convolution layer, the second pooling convolution layer and the third pooling convolution layer is 1, the convolution step is 1, and the number of rows and columns supplemented during convolution is 0.

[0049] In the step of "processing the pooled feature map with channel attention to obtain the spatial pooling result", an attention module is set after the spatial pooling sub-model. The structural diagram of the attention module is as follows: Figure 3 As shown, the attention module includes a channel attention unit and a feedforward network unit. The pooled feature map is input into the attention module and sequentially passes through the channel attention unit and the feedforward network unit to obtain a spatial pooling result.

[0050] Furthermore, the pooled feature map first passes through the first attention convolution layer in the attention module to obtain a first attention convolution result, and channel feature separation is performed on the first attention convolution result to obtain a first separation feature and a second separation feature. The first attention convolution result passes through the channel attention unit and then performs a residual connection with the first separation feature to obtain a first residual result. The first residual result passes through the feedforward network unit and then performs a residual connection with the first residual result to obtain a second residual result. The second residual result is spliced ​​with the second separation feature and then output by the second attention convolution layer to obtain a spatial pooling result.

[0051] Furthermore, the structural diagram of the channel attention unit is as follows Figure 4 As shown, the channel attention unit includes a global average pooling layer and a fully connected layer. The global average pooling layer performs pooling processing on the first attention convolution result to obtain a global average pooling result. The global average pooling result is feature multiplied with the first convolution result after two full connection operations in the fully connected layer and output.

[0052] Furthermore, the feed-forward network unit consists of two convolutional layers connected in series.

[0053] Specifically, in the conventional attention module, MLP is used to calculate the multi-head attention mechanism and the SoftMax activation function is used for output. The calculation amount of the SoftMax activation function is large, so it cannot achieve the real-time detection effect when running on some chips with insufficient performance. Therefore, this solution uses the channel self-attention mechanism. The channel self-attention mechanism uses Sigmoid as the activation function for output, which greatly reduces the required calculation amount, so even if the chip performance is insufficient, the real-time recognition effect can be achieved.

[0054] In this scheme, the upsampling model is composed of multiple upsampling units in series, where the number of upsampling units is one less than the number of sub-feature maps. Each upsampling unit changes the scale of the input feature in a convolution manner and splices it with the sub-feature map of the same scale, and inputs the splicing result into the next upsampling unit. The output result of the last upsampling unit is the upsampling result.

[0055] Furthermore, the upsampling unit consists of a first upsampling convolutional layer, an upsampling layer and a second upsampling convolutional layer.

[0056] In this solution, the number of the aggregated sub-models does not exceed the number of the sub-feature graphs. The structural diagram of the aggregated sub-model is as follows: Figure 5 As shown, in the aggregation sub-model, the upsampling result passes through the first fusion convolution layer to perform channel feature separation to obtain the first fusion feature and the second fusion feature, the second fusion feature passes through the first parameterized convolution to obtain the first parameterized convolution result, the first parameterized convolution result passes through the second parameterized convolution to obtain the second parameterized convolution result, the first fusion feature, the second fusion feature, the first parameterized convolution result, and the second parameterized convolution result are spliced ​​to obtain a fused splicing result, and the fused splicing result is output through the channel attention layer to obtain an aggregation result.

[0057] Furthermore, when there are multiple aggregation sub-models, two adjacent aggregation sub-models are connected by an aggregation upsampling layer, wherein the input of the first aggregation sub-model is the upsampling result, and the output features of each aggregation sub-model are concatenated to obtain the aggregation result, wherein the output features of the aggregation sub-model are resized by the upsampling layer and then concatenated with the input features of the upsampling unit of the same scale in the upsampling model as the input features of the next aggregation sub-model.

[0058] Specifically, this scheme strengthens the feature output by constructing the RepNCSPELAN4_SE module, thereby improving the accuracy of real-time target detection.

[0059] In this scheme, the aggregated upsampling layer is a double-layer convolution unit, and the double-layer convolution unit is composed of a first downsampling convolution layer and a second downsampling convolution layer connected in sequence. The first downsampling convolution layer is a convolution-normalization-activation structure, and the second downsampling convolution layer is a convolution-normalization structure.

[0060] Specifically, the structural diagram of the double-layer convolution unit is as follows: Figure 6 As shown, the convolution kernel of the first downsampling convolution layer in the double-layer convolution unit is 1, the step size is 1, and the number of rows and columns supplemented during convolution is 0. The convolution kernel of the second downsampling convolution layer is 3, the step size is 2, and the number of rows and columns supplemented during convolution is 1.

[0061] In this solution, the recognition detection head detects the recognition targets in the aggregation results through the classification head. Specifically, a loss function is constructed according to the detection results and the labeling results to update the parameters of each sub-model to obtain a trained target detection model.

[0062] In some specific embodiments, the overall structure of the head network of the real-time target detection model is as follows: Figure 7 As shown in Figure 2, since the feature extraction sub-model is based on the backbone framework of yolov7, Figure 7 It is omitted in .

[0063] Specifically, the real-time target detection model in this solution can detect pedestrians, vehicles and other various targets in real time.

[0064] Embodiment 2

[0065] Based on the same idea, refer to Figure 8 , the present application also proposes a low-computing-power real-time target detection model construction device, including:

[0066] A feature extraction sub-model construction module is used to construct a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales;

[0067] A spatial pooling sub-model construction module is used to construct a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then process the pooling feature map by channel attention to obtain a spatial pooling result;

[0068] An upsampling sub-model construction module, used to construct an upsampling sub-model, wherein the upsampling sub-model is trained to use multiple upsampling units to splice the spatial pooling result with sub-feature maps of different scales in an upsampling manner to obtain an upsampling result;

[0069] An aggregation sub-model construction module, used to construct an aggregation sub-model, wherein the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result;

[0070] The recognition detection head building module is used to build a recognition detection head, where the recognition detection head is trained to obtain target category, position and confidence information in each aggregation result.

[0071] Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

[0072] Embodiment 3

[0073] This embodiment also provides an electronic device, referring to Fig. 9 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0074] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0075] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0076] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0077] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the methods for constructing a low-computing-power real-time target detection model in the above embodiments.

[0078] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0079] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0080] The input and output device 408 is used to input or output information. In this embodiment, the input information may be codes of various modules in the Yolov7 head network, and the output information may be parameters of a real-time target detection model.

[0081] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0082] Constructing a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales;

[0083] Constructing a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then performing channel attention processing on the pooling feature map to obtain a spatial pooling result;

[0084] An upsampling sub-model is constructed, where the upsampling sub-model is trained to concatenate the spatial pooling result with sub-feature maps of different scales in an upsampling manner using a plurality of upsampling units to obtain an upsampling result;

[0085] Constructing an aggregation sub-model, where the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result;

[0086] Building a recognition detection head, wherein the recognition detection head is trained to obtain target category, location and confidence information in each aggregation result;

[0087] Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

[0088] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0089] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0090] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Fig. 9 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0091] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a real-time target detection model with low computing power, characterized in that: The following steps are involved: Constructing a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales; Constructing a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then performing channel attention processing on the pooling feature map to obtain a spatial pooling result; An upsampling sub-model is constructed, where the upsampling sub-model is trained to concatenate the spatial pooling result with sub-feature maps of different scales in an upsampling manner using a plurality of upsampling units to obtain an upsampling result; Constructing an aggregation sub-model, where the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result; Building a recognition detection head, wherein the recognition detection head is trained to obtain target category, location and confidence information in each aggregation result; Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

2. The method for constructing a low-computing-power real-time target detection model according to claim 1, characterized in that: In the spatial pooling sub-model, the first pooling convolution layer is used to convolve the sub-feature map with the smallest scale, and then the convolution is respectively input into the second pooling convolution layer, the maximum pooling layer and the average pooling layer connected in parallel. The output features of the first pooling convolution layer, the second pooling convolution layer, the maximum pooling layer and the average pooling layer are spliced ​​and input into the third pooling convolution layer to obtain the pooling feature map.

3. The method for constructing a low-computing-power real-time target detection model according to claim 2, characterized in that: In the step of "performing channel attention on the pooled feature map to obtain a spatial pooling result", an attention module is set after the spatial pooling sub-model, and the attention module includes a channel attention unit and a feedforward network unit. The pooled feature map is input into the attention module and sequentially passes through the channel attention unit and the feedforward network unit to obtain the spatial pooling result.

4. The method for constructing a low-computing-power real-time target detection model according to claim 3, characterized in that: The pooled feature map first passes through the first attention convolution layer in the attention module to obtain the first attention convolution result, and the channel feature separation is performed on the first attention convolution result to obtain the first separation feature and the second separation feature. The first attention convolution result passes through the channel attention unit and then performs a residual connection with the first separation feature to obtain the first residual result. The first residual result passes through the feedforward network unit and then performs a residual connection with the first residual result to obtain the second residual result. The second residual result is spliced ​​with the second separation feature and then output by the second attention convolution layer to obtain the spatial pooling result.

5. The method for constructing a low-computing-power real-time target detection model according to claim 4, characterized in that: The channel attention unit includes a global average pooling layer and a fully connected layer. The global average pooling layer performs pooling processing on the first attention convolution result to obtain a global average pooling result. The global average pooling result is feature multiplied with the first attention convolution result after two full connection operations in the fully connected layer and output.

6. The method for constructing a low-computing-power real-time target detection model according to claim 1, characterized in that: The number of the aggregated sub-models does not exceed the number of sub-feature graphs. In the aggregated sub-model, the upsampling result is subjected to the first fusion convolution layer and then channel feature separation is performed to obtain the first fusion feature and the second fusion feature. The second fusion feature is subjected to the first parameterized convolution to obtain the first parameterized convolution result. The first parameterized convolution result is subjected to the second parameterized convolution to obtain the second parameterized convolution result. The first fusion feature, the second fusion feature, the first parameterized convolution result, and the second parameterized convolution result are spliced ​​to obtain a fused splicing result. The fused splicing result is output through the channel attention layer to obtain an aggregated result.

7. The method for constructing a low-computing-power real-time target detection model according to claim 6, characterized in that: When there are multiple aggregation sub-models, two adjacent aggregation sub-models are connected by an aggregation upsampling layer, where the input of the first aggregation sub-model is the upsampling result, and the output features of each aggregation sub-model are spliced ​​to obtain the aggregation result, where the output features of the aggregation sub-model are resized by the upsampling layer and then spliced ​​with the input features of the upsampling unit of the same scale in the upsampling model as the input features of the next aggregation sub-model.

8. A device for constructing a real-time target detection model with low computing power, characterized in that: include: A feature extraction sub-model construction module is used to construct a feature extraction sub-model, wherein the feature extraction sub-model is used to perform feature extraction training on training samples marked with recognition targets to obtain sub-feature maps of multiple scales; A spatial pooling sub-model construction module is used to construct a spatial pooling sub-model, wherein the spatial pooling sub-model is trained to process the minimum-scale sub-feature map by maximum pooling and spatial pooling to obtain a pooling feature map, and then process the pooling feature map by channel attention to obtain a spatial pooling result; An upsampling sub-model construction module, used to construct an upsampling sub-model, wherein the upsampling sub-model is trained to use multiple upsampling units to splice the spatial pooling result with sub-feature maps of different scales in an upsampling manner to obtain an upsampling result; An aggregation sub-model construction module, used to construct an aggregation sub-model, wherein the aggregation sub-model is trained to extract features from the up-sampling results in a re-parameterized manner to obtain an aggregation result; A recognition detection head construction module, used to construct a recognition detection head, wherein the recognition detection head is trained to obtain target category, position and confidence information in each aggregation result; Among them, the real-time target detection model includes a feature extraction sub-model, a spatial pooling sub-model, an upsampling sub-model, an aggregation sub-model and a recognition detection head. The trained real-time target detection model is used to detect the recognition target in the image to be detected, wherein the image to be detected is image data that does not contain annotation information.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute a method for constructing a low-computing-power real-time target detection model as described in any one of claims 1-7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes a method for constructing a low-computing-power real-time target detection model according to any one of claims 1-7.