Target detection model training method, target detection method and related device

Abstract

The present application provides a training method for a target detection model, a target detection method, and related devices. The training method includes: extracting features from a sample image to obtain a feature-extracted image; performing feature fusion on the feature-extracted image to obtain a feature-fused image; predicting the feature-fused image based on an anchor-free frame algorithm to obtain a corresponding first loss, and predicting the feature-fused image based on an anchor-based frame algorithm to obtain a corresponding second loss; obtaining a total loss based on the first loss and the second loss; and adjusting the parameters of the target detection model based on the total loss so that after the image to be tested is input into the trained target detection model, a corresponding predicted target frame can be obtained based on the anchor-free frame algorithm. In this way, the present application can ensure high detection accuracy while reducing the amount of network parameters and the required computing power.

Description

Technical Field

[0001] The present application belongs to the field of target detection technology, and specifically relates to a target detection model training method, a target detection method and related devices. Background Art

[0002] In recent years, the mobile robotics industry has developed rapidly, cutting-edge algorithms have been continuously updated, and vision-based target detection technology has gradually become a research hotspot in the field of mobile robotics.

[0003] Currently, most high-precision object detection models are too deep and parameter-heavy, requiring high computing power. This makes deployment on embedded platforms difficult and results in low real-time detection frame rates. Object detection models that are too lightweight can suffer from false detections and excessive noise, resulting in low detection accuracy. Therefore, a new object detection method is necessary. Summary of the Invention

[0004] The present application provides a target detection model training method, a target detection method and related devices to ensure high detection accuracy while reducing the number of network parameters and the required computing power, making up for the shortcomings of the current mainstream deep target detection method that has a long running time and occupies a lot of resources, improving the real-time target detection frame rate, and making the target detection method more suitable for target detection tasks of mobile robots.

[0005] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a training method for a target detection model, the training method including: performing feature extraction on a sample image to obtain a feature extraction image; performing feature fusion on the feature extraction image to obtain a feature fusion image; predicting the feature fusion image based on an anchor-free frame algorithm to obtain a corresponding first loss, and predicting the feature fusion image based on an anchor-frame algorithm to obtain a corresponding second loss; obtaining a total loss based on the first loss and the second loss; and adjusting the parameters of the target detection model based on the total loss so that after the image to be tested is input into the trained target detection model, a corresponding predicted target frame can be obtained based on the anchor-free frame algorithm.

[0006] To solve the above technical problems, another technical solution adopted in the present application is: to provide a target detection method, comprising: inputting the image to be tested into a trained target detection model to obtain a predicted target frame predicted by the anchor-free frame algorithm in the target detection model; wherein, the target detection model is trained using the training method described in any of the above embodiments; and obtaining a target detection result based on the predicted target frame.

[0007] To solve the above technical problems, another technical solution adopted in this application is: to provide an electronic device, including a memory and a processor coupled to each other, wherein the memory stores a program, and the processor is used to execute the program to implement the training method or target detection method described in any of the above embodiments.

[0008] In order to solve the above technical problems, another technical solution adopted in this application is: providing a storage device storing a program that can be run by a processor, wherein the program is used to implement the training method or target detection method described in any of the above embodiments.

[0009] Generally speaking, an anchor, also known as an anchor box, is a set of rectangular boxes clustered on a training set using methods such as k-means before training. They represent the main length and width scales of objects in the dataset. Current mainstream object detection technologies can be broadly categorized as anchor-based and anchor-free. Anchor-free object detection suffers from poor detection accuracy and insufficient model reliability. While anchor-based object detection offers high detection accuracy, it suffers from poor universality and a large number of parameters due to the fixed size of the anchor boxes. Therefore, both approaches hinder the implementation of object detection models.

[0010] Based on this, and different from the existing technical situation, the present application complements the advantages of the two and proposes a target detection algorithm that combines anchor-free and anchor-based. Specifically, in the training stage of the target detection model, the second prediction network of the anchor-based algorithm is used to assist the first prediction network of the anchor-free algorithm for training, so as to improve the detection frame regression quality of the first prediction network of the anchor-free algorithm. In the application stage of the target detection model, only the first prediction network of the anchor-free algorithm is used for prediction, which reduces the occupation of computing resources and ensures that the target detection method can still perform well on a platform with limited computing resources. In short, the target detection method provided by the present application can reduce the amount of network parameters and the required computing power while ensuring high detection accuracy, make up for the shortcomings of the current mainstream deep target detection algorithm that has a long running time and occupies a lot of resources, improve the real-time target detection frame rate, and make the target detection method more suitable for target detection tasks of mobile robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0012] Figure 1 This is a schematic diagram of the structure of an implementation scheme of the target detection model of this application;

[0013] Figure 2 This is a flowchart of an implementation method of a training method for an object detection model of the present application;

[0014] Figure 3 for Figure 1 A structural diagram of an implementation scheme corresponding to the target detection model;

[0015] Figure 4 This is a structural diagram of an implementation of a hybrid network block (ShuffleNet Block);

[0016] Figure 5 This is a schematic structural diagram of an implementation of a ghost block (Ghost Block);

[0017] Figure 6 This is a flow chart of an embodiment of the target detection method of the present application;

[0018] Figure 7 This is a structural diagram of an embodiment of a training device for an object detection model of the present application;

[0019] Figure 8 This is a schematic structural diagram of an embodiment of an electronic device of the present application;

[0020] Figure 9 This is a structural diagram of an embodiment of the storage device of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] For ease of understanding, let's first introduce the general structure of the target detection model in this application. Figure 1 , Figure 1This is a structural diagram of an embodiment of the target detection model of the present application. The target detection model includes a feature extraction network 10, a feature fusion network 12, and a first prediction network 14 based on an anchor-free algorithm and a second prediction network 16 based on an anchor-based algorithm, which are arranged side by side; wherein the feature extraction network 10 can be regarded as the backbone of the target detection model, the feature fusion network 12 can be regarded as the neck of the target detection model, and the first prediction network 14 and the second prediction network 16 can be regarded as the head of the target detection model. It can be seen that the output of the feature extraction network 10 is the input of the feature fusion network 12, and the output of the feature fusion network 12 is the input of the first prediction network 14 and the second prediction network 16. In addition, the first prediction network 14 is a prediction network based on an anchor-free algorithm, and its prediction network structure can be CornerNet, ExtremeNet, CenterNet, FCOS, etc. The second prediction network 16 is a prediction network based on an anchor-based algorithm, and its prediction network structure can be Faster R CNN, SSD, YOLO V2 / V3, etc. Figure 1 The second prediction network 16 is represented by a dotted line because the second prediction network 16 is only used in the training process of the target detection model, and the second prediction network 16 is not used in the actual application process of the target detection model.

[0023] For further information, see Figure 2 , Figure 2 This is a flow chart of an embodiment of a method for training a target detection model of the present application. The method for training a target detection model specifically includes:

[0024] S101: Perform feature extraction on a sample image to obtain a feature-extracted image.

[0025] Specifically, if Figure 1 As shown, the sample image can be input into the feature extraction network 10 to obtain a feature-extracted image, that is, the feature extraction network 10 is used to extract features from the sample image to obtain the feature-extracted image.

[0026] In one application scenario, see Figure 3 , Figure 3 for Figure 1A structural diagram of an embodiment of the target detection model in FIG. The feature extraction network 10 includes multiple lightweight feature extraction modules 100, each of which outputs a feature extraction image to the feature fusion network 12; and the output of one lightweight feature extraction module 100 in two adjacent lightweight feature extraction modules 100 is the input of the other lightweight feature extraction module 100, and the sizes of the feature extraction images output by different lightweight feature extraction modules 100 are different. For example, Figure 3 In FIG, three lightweight feature extraction modules 100 are schematically shown. As the size of the feature extraction images output by the three lightweight feature extraction modules 100 decreases gradually in the direction away from the input. For example, Figure 3 The three lightweight feature extraction modules 100 can output three feature extraction images, namely P3, P4 and P5, and the size of P3 is larger than that of P4, and the size of P4 is larger than that of P5.

[0027] On the one hand, by introducing the lightweight feature extraction module 100, the size of the feature extraction network 10 can be reduced, and the computational complexity of the feature extraction network 10 can be reduced; on the other hand, when the number of the above-mentioned lightweight feature extraction modules 100 is three, the three lightweight feature extraction modules 100 can respectively output feature extraction images of large, medium and small sizes to the subsequent feature fusion network 12, thereby reducing the computational complexity while ensuring the computational accuracy.

[0028] Optionally, the lightweight feature extraction module 100 includes a ShuffleNet Block. Specifically, Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of one implementation of the ShuffleNet Block. The ShuffleNet Block significantly reduces the computational overhead of the network while maintaining accuracy by utilizing depthwise separable convolutional layers (DWConv) and channel shuffle layers (channelShuffle).

[0029] Also, please refer again to Figure 3 The feature extraction network 10 may further include other structural layers, for example, a downsampling layer 102 and a maximum pooling layer 104 before the multiple lightweight feature extraction modules 100, and the maximum pooling layer 104 may be located between the downsampling layer 102 and the multiple lightweight feature extraction modules 100.

[0030] Further, please refer again Figure 3 The specific implementation process of the above step S101 may include:

[0031] A1. Perform downsampling and maximum pooling operations on the sample image in sequence to obtain the sample feature map.

[0032] Specifically, the above step A1 can be implemented by the downsampling layer 102 and the maximum pooling layer 104; for example, before the above step A1, the sample image can be standardized so that the size of the sample image is 320*320 and the number of channels is 3; further, the downsampling layer 102 can be used to perform a downsampling operation on the sample image with a convolution kernel of 3 and a step size of 2, and the output channel of the sample image becomes 24; then the maximum pooling layer 104 is used to perform a maximum pooling operation with a step size of 2 on the downsampled sample image to obtain a sample feature map; at this time, the size of the sample feature map is 80*80, and the number of channels is 24.

[0033] B1. Perform feature extraction on the sample feature map to obtain a first feature extraction image P3.

[0034] Specifically, in step B1, a lightweight feature extraction module labeled 100a can perform a ShuffleNet Block operation with a step size of 2 on the sample feature map, and the output first feature extraction image P3 has a size of 40*40 and a channel number of 116.

[0035] C1. Perform feature extraction on the first feature extraction image P3 to obtain a second feature extraction image P4.

[0036] Specifically, in step C1, a lightweight feature extraction module labeled 100b may be used to perform a ShuffleNet Block operation with a step size of 2 on the first feature extraction image P3, and the output second feature extraction image P4 has a size of 20*20 and a channel number of 232.

[0037] D1. Perform feature extraction on the second feature extraction image P4 to obtain a third feature extraction image P5.

[0038] Specifically, in step D1, a lightweight feature extraction module labeled 100c can perform a ShuffleNet Block operation with a step size of 2 on the second feature extraction image P4, and the output third feature extraction image P5 has a size of 10*10 and a channel number of 464.

[0039] It can be seen that the specific implementation process of the above step S101 is relatively simple, and feature extraction images of three sizes, large, medium and small, can be output to the subsequent feature fusion network 12, which can reduce the amount of calculation while ensuring the calculation accuracy.

[0040] S102: Perform feature fusion on the feature-extracted image to obtain a feature-fused image.

[0041] Specifically, if Figure 1 As shown, the feature extraction image can be input into the feature fusion network 12 to obtain a feature fusion image; that is, the feature fusion network 12 is used to perform feature fusion on the feature extraction image to obtain a feature fusion image.

[0042] Among them, the feature fusion network 12 is a key link in the target detection model, which plays the role of fusing the important features extracted by the feature extraction network 10. The feature fusion network 12 is conducive to the feature learning of the specific task of the first prediction network 14 or the second prediction network 16 in the next step. Figure 3 As shown, the feature fusion network 12 includes multiple lightweight feature fusion modules 120, and the input of one lightweight feature fusion module 120 of two adjacent lightweight feature fusion modules 120 is related to the output of the other lightweight feature fusion module 120 and one of the feature extraction images. Figure 3 In the direction from input to output, the lightweight feature fusion module marked as 120a is adjacent to the lightweight feature fusion module marked as 120b, and the input of the lightweight feature fusion module marked as 120b is related to the first feature extraction image P3 and the output of the lightweight feature fusion module marked as 120a.

[0043] Optionally, the lightweight feature fusion module includes a ghost block. Specifically, Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an implementation of the Ghost Block. The Ghost Block can generate more feature maps at a low cost, enhance the feature fusion effect, and reduce the computational complexity while ensuring computational accuracy.

[0044] Also, please refer again to Figure 3 The feature fusion network 12 may also include other structural layers, such as an upsampling layer, a convolution layer, a channel cascade layer, a downsampling layer, etc., which are not described in detail in this application.

[0045] Further, please refer again Figure 3 , the above step S102 specifically includes:

[0046] A2. Obtain a first intermediate fusion image M4 based on the second feature extraction image P4 and the third feature extraction image P5.

[0047] In a specific application scenario, such as Figure 3As shown, after step S101, a first feature extraction image P3, a second feature extraction image P4, and a third feature extraction image P5 can be obtained. The size of the first feature extraction image P3 is 40*40 and the number of channels is 116; the size of the second feature extraction image P4 is 20*20 and the number of channels is 232; the size of the third feature extraction image P5 is 10*10 and the number of channels is 464. For the convenience of description below, the first feature extraction image P3 is directly abbreviated as P3, the second feature extraction image P4 is directly abbreviated as P4, and the third feature extraction image P5 is directly abbreviated as P5.

[0048] Furthermore, the specific implementation process of the above step A2 includes: using a 1*1 convolution kernel to compress the number of channels of P5 from 464 to 96, and upsampling to obtain M5 with a size of 20*20; using a 1*1 convolution kernel to compress the channels of P4 from 232 to 96, obtaining P4' with a size of 20*20; performing a channel cascade operation on P4' and M5 to obtain T1, and performing feature fusion on T1 through a phantom block marked as 120a, and outputting M4 with 256 channels and a size of 20*20.

[0049] B2. Obtain a first feature fusion image H3 based on the first feature extraction image P3 and the first intermediate fusion image M4.

[0050] In a specific application scenario, such as Figure 3 As shown, the specific implementation process of the above step B2 includes: upsampling M4 to obtain U4 with a size of 40*40; using a 1*1 convolution kernel to compress the channel of P3 to 96, obtaining P3' with a size of 40*40; performing a channel cascade operation on P3' and U4 to obtain T2, and then performing feature fusion through the phantom block marked as 120b, and outputting a first feature fusion image H3 with 96 channels and a size of 40*40.

[0051] C2. Obtain a second feature fusion image H4 based on the first feature fusion image H3 and the first intermediate fusion image M4.

[0052] In a specific application scenario, such as Figure 3 As shown, the specific implementation process of the above step C2 includes: downsampling H3 to obtain D3 with a size of 20*20; performing a channel cascade operation on D3 and M4 to obtain T3, and performing feature fusion on T3 through a phantom block marked as 120c to output a second feature fusion image H4 with 96 channels and a size of 20*20.

[0053] D2. Obtain a third feature fusion image H5 based on the second feature fusion image H4 and the third feature extraction image P5.

[0054] In a specific application scenario, such as Figure 3 As shown, the specific implementation process of the above step D2 includes: downsampling H4 to obtain D4 with a size of 10*10; performing a channel cascade operation on D4 and P5 to obtain T4, and performing feature fusion on T4 through the phantom block 120d to output a third fused feature image H5 with 96 channels and a size of 10*10.

[0055] It can be seen that the specific implementation process of the above step S102 is relatively simple, which can reduce the amount of calculation while ensuring the calculation accuracy.

[0056] S103: predicting the feature fusion image based on the anchor-free algorithm to obtain a corresponding first loss, and predicting the feature fusion image based on the anchor-with algorithm to obtain a corresponding second loss.

[0057] Specifically, if Figure 1 As shown, the feature fused image may be input to the first prediction network 14 to obtain a first loss, and the feature fused image may be input to the second prediction network 16 to obtain a second loss.

[0058] Furthermore, the specific implementation process of the above-mentioned step S103 includes: obtaining multiple first samples obtained by predicting the feature fusion image based on the anchor-free algorithm, and multiple second samples obtained by predicting the feature fusion image based on the anchor-based algorithm; obtaining multiple positive samples from the multiple first samples and the multiple second samples based on the first sample allocation method; predicting the multiple positive samples based on the anchor-free algorithm to obtain corresponding first losses, and predicting the multiple positive samples based on the anchor-based algorithm to obtain corresponding second losses. Optionally, the first sample allocation algorithm includes a SimOTA algorithm.

[0059] Generally speaking, the prediction network needs to regress the target frame based on the positive samples, so the first sample allocation algorithm will greatly affect the performance of the target detection algorithm. Therefore, it is necessary to select a suitable sample allocation algorithm when training the target detection model. Conventional sample allocation algorithms usually set a fixed number of positive samples, but such sample allocation algorithms cannot be flexibly applied to different data sets. SimOTA, as a flexible sample allocation method, can determine the appropriate number of positive samples for different targets. When SimOTA is applied to the anchor-free first prediction network, it covers more positive samples. Compared with the anchor-based second prediction network, the recall rate of the anchor-free first prediction network is higher, the positive sample selection range is larger, but the accuracy of the frame regression is lower. In this application, after SimOTA is applied to the first prediction network and the second prediction network, the grid where the anchor point in the anchor-free first prediction network is located and the preset anchor box in the anchor-based second prediction network are used together with SimOTA to select positive samples. This design method can help the anchor-free first prediction network select more suitable positive samples for regression, so that the position of the regressed positive samples is more accurate.

[0060] It should be noted that if Figure 3 As shown, when there are multiple feature fusion images and the sizes of the feature fusion images are different, a first prediction network 14 and a second prediction network 16 of a corresponding size can be set for each size.

[0061] For example, in a specific application scenario, such as Figure 3 As shown, the number of anchor-free first prediction networks is three, and the scales correspond to the output H3, H4, and H5 respectively; at this time, H3, H4, and H5 can be used as input respectively, first undergoing a convolution operation with a convolution kernel of 1*1 and a step size of 1, and then a convolution operation with a convolution kernel of 3*3 and a step size of 1. The output sizes of each scale are 40*40, 20*20, and 10*10 respectively, and the output dimensions at each scale are 3*(number of categories + distance from the center point to the four bounding boxes + confidence level); the "+" here does not represent addition in the mathematical field, but can be understood as "sum", that is, the output at each scale includes the number of categories, the distance from the center point to the four bounding boxes, and the confidence level.

[0062] like Figure 3As shown, there are three anchor-based second prediction networks, and the scales correspond to the output H3, H4, and H5 respectively; H3, H4, and H5 are taken as input respectively, and first undergo a convolution operation with a convolution kernel of 1*1 and a step size of 1, and then are divided into two branches, a classification branch and a regression branch. Each branch performs a convolution operation with a convolution kernel of 3*3 and a step size of 1. The output sizes of each scale are 40*40, 20*20, and 10*10 respectively. The number of classification branch channels is the number of categories, and the output dimensions of the regression branch are the distance from the center point to the four bounding boxes and the confidence level respectively.

[0063] S104: Obtain a total loss based on the first loss and the second loss.

[0064] Specifically, the specific implementation process of the above step S104 can be: obtaining the first loss loss a The first product with the first weight α and the second loss loss b The second product of the first weight α and the second weight β; wherein the first weight α is greater than 0, the second weight β is greater than 0, and the first weight α is greater than the second weight β; the sum of the first product and the second product is taken as the total loss loss; which is expressed as follows:

[0065] loss = αloss a +βloss b .

[0066] Optionally, the first loss loss a It can be the sum of the corresponding classification loss, regression loss and confidence loss, or the sum of the weighted sum; the second loss loss b It can be the sum of the corresponding classification loss, regression loss and confidence loss, or the sum of the weighted sum.

[0067] It can be seen that the above process of obtaining the total loss is relatively simple, and since the anchor-based second prediction network is used to assist the anchor-free first prediction network in training, the loss weight corresponding to the first prediction network is larger when calculating the total loss, so that the output of the first prediction network of the target detection model adjusted based on the total loss is more accurate.

[0068] S105: Adjust the parameters of the target detection model based on the total loss so that after the image to be tested is input into the trained target detection model, the corresponding predicted target frame can be obtained based on the anchor-free frame algorithm.

[0069] In summary, the target detection method provided in this application can reduce the number of network parameters and the required computing power while ensuring high detection accuracy, making up for the shortcomings of the current mainstream deep target detection algorithm that has a long running time and occupies a lot of resources, and improving the real-time target detection frame rate, making the target detection method more suitable for target detection tasks of mobile robots.

[0070] See also Figure 6 , Figure 6 This is a flow chart of an embodiment of the target detection method of the present application, which includes:

[0071] S201: Input the image to be tested into the trained target detection model to obtain a predicted target frame predicted by the target detection model based on the anchor-free frame algorithm.

[0072] Specifically, the target detection model is trained using the training method mentioned in any of the above embodiments, which will not be repeated here.

[0073] S202: Obtain target detection results based on the predicted target frame.

[0074] Specifically, if Figure 3 As shown, corresponding predicted target frames can be obtained respectively through multiple first prediction networks of the target detection model, and the final output target frame can be determined based on the multiple predicted target frames. The content corresponding to the output target frame is the target detection result.

[0075] It can be seen that the present application proposes a target detection algorithm that combines anchor-free and anchor-based. Specifically, in the training stage of the target detection model, the second prediction network of the anchor-based algorithm is used to assist the first prediction network of the anchor-free algorithm in training, so as to improve the detection frame regression quality of the first prediction network of the anchor-free algorithm. In the application stage of the target detection model, only the first prediction network of the anchor-free algorithm is used for prediction, which reduces the occupation of computing resources and ensures that the target detection method can still perform well on a platform with limited computing resources. In short, the target detection method provided by the present application can reduce the amount of network parameters and the required computing power while ensuring high detection accuracy, make up for the shortcomings of the current mainstream deep target detection algorithm that has a long running time and occupies a lot of resources, improve the real-time target detection frame rate, and make the target detection method more suitable for target detection tasks of mobile robots.

[0076] See also Figure 7 , Figure 7This is a schematic diagram of the structure of an embodiment of a training device for an object detection model of the present application. The training device includes a feature extraction module 20, a feature fusion module 22, a first prediction module 24, a second prediction module 26, an acquisition module 28, and an adjustment module 21;

[0077] Among them, the feature extraction module 20 is used to extract features from the sample image to obtain a feature extraction image; the feature fusion module 22 is connected to the feature extraction module 20, and is used to perform feature fusion on the feature extraction image to obtain a feature fusion image; the first prediction module 24 is connected to the feature fusion module 22, and is used to predict the feature fusion image based on the anchor-free frame algorithm to obtain a first loss; the second prediction module 26 is connected to the feature fusion module 22, and is used to predict the feature fusion image based on the anchor-frame algorithm to obtain a second loss; the acquisition module 28 is connected to the first prediction module 24 and the second prediction module 26, and is used to obtain a total loss based on the first loss and the second loss; the adjustment module 21 is connected to the acquisition module 28, and is used to adjust the parameters of the target detection model based on the total loss, so that after the image to be tested is input into the trained target detection model, the corresponding predicted target frame can be obtained based on the anchor-free frame algorithm.

[0078] See also Figure 8 , Figure 8 This is a structural diagram of an embodiment of an electronic device of the present application. The electronic device includes: a memory 32 and a processor 30 coupled to each other, a program stored in the memory 32, and the processor 30 is used to execute the program to implement any of the above-mentioned training methods or target detection methods. Specifically, the electronic device includes but is not limited to: a desktop computer, a laptop computer, a tablet computer, a server, etc., which are not limited here. In addition, the processor 30 can also be called a CPU (Center Processing Unit). The processor 30 may be an integrated circuit chip with signal processing capabilities. The processor 30 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 30 can be implemented by an integrated circuit chip.

[0079] See also Figure 9 , Figure 9This is a schematic diagram of the structure of an embodiment of a storage device of the present application. The storage device 40 stores a program 400 that can be executed by a processor. The program 400 is used to implement any of the above-mentioned training methods or target detection methods. The storage device includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program code.

[0080] In summary, in order to solve the problem of poor target frame regression quality in the anchor-free target detection method, this application uses an anchor-based second prediction network to assist in training the anchor-free first prediction network during the training phase. During the inference phase, only the anchor-free first prediction network is used for target frame regression, which reduces the use of computing resources and ensures that the target detection method can still perform well on platforms with limited computing resources. The target detection method provided by this application is more suitable for the detection needs of mobile terminals.

[0081] Secondly, when solving the sample allocation problem between anchor-free and anchor-based methods, this application uses anchor-based methods to assist anchor-free methods in selecting more appropriate positive samples, making the regressed target box position more accurate. Compared with conventional sample allocation methods, the SimOTA sample allocation method adopted in this application better combines the sample selection advantages of both anchor-free and anchor-based methods.

[0082] Thirdly, to improve the real-time performance and usability of target detection methods for mobile robots, this application uses ShuffleNet Block to reduce the number of parameters in the feature extraction network during the target detection model construction process. During feature fusion, Ghost Block is used to reduce the computational complexity of the feature fusion network. Compared to traditional target detection methods, the target detection method provided by this application has the advantages of good scalability, easy debugging, lightweight, and high flexibility.

[0083] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for training a target detection model, characterized in that: The training method comprises: performing feature extraction on the sample image to obtain a feature-extracted image; Performing feature fusion on the feature extraction image to obtain a feature fusion image; Predicting the feature fusion image based on an anchor-free frame algorithm to obtain a corresponding first loss, and predicting the feature fusion image based on an anchor-frame algorithm to obtain a corresponding second loss; Obtaining a total loss based on the first loss and the second loss; Adjusting the parameters of the target detection model based on the total loss so that after the image to be tested is input into the trained target detection model, a corresponding predicted target frame can be obtained based on the anchor-free frame algorithm; The step of performing feature extraction on the sample image to obtain a feature extraction image comprises: performing downsampling and maximum pooling operations on the sample image in sequence to obtain a sample feature map; performing feature extraction on the sample feature map to obtain a first feature extraction image; performing feature extraction on the first feature extraction image to obtain a second feature extraction image; and performing feature extraction on the second feature extraction image to obtain a third feature extraction image. The step of performing feature fusion on the feature extraction image to obtain a feature fusion image includes: obtaining a first intermediate fusion image based on the second feature extraction image and the third feature extraction image; obtaining a first feature fusion image based on the first feature extraction image and the first intermediate fusion image; obtaining a second feature fusion image based on the first feature fusion image and the first intermediate fusion image; obtaining a third feature fusion image based on the second feature fusion image and the third feature extraction image, wherein the first feature fusion image, the second feature fusion image and the third feature fusion image serve as inputs to the anchor-free frame algorithm and the anchor-frame algorithm.

2. The training method according to claim 1, characterized in that The step of obtaining a total loss based on a first loss corresponding to the first target box and a second loss corresponding to the second target box includes: Obtaining the first loss based on the true value of the first target box and the sample image, and obtaining the second loss based on the true value of the second target box and the sample image; wherein the first loss and the second loss respectively include corresponding classification loss, regression loss, and confidence loss; Obtaining a first product of the first loss and a first weight, and a second product of the second loss and a second weight; wherein the first weight is greater than 0, the second weight is greater than 0, and the first weight is greater than the second weight; The sum of the first product and the second product is taken as the total loss.

3. The training method according to claim 1, characterized in that The step of predicting the feature fusion image based on the anchor-free frame algorithm to obtain a corresponding first loss, and predicting the feature fusion image based on the anchor-frame algorithm to obtain a corresponding second loss includes: Obtaining a plurality of first samples obtained by predicting the feature fusion image based on an anchor-free frame algorithm, and a plurality of second samples obtained by predicting the feature fusion image based on an anchor-based frame algorithm; Obtaining a plurality of positive samples from the plurality of first samples and the plurality of second samples based on a sample allocation algorithm; The plurality of positive samples are predicted based on an anchor-free frame algorithm to obtain a corresponding first loss, and the plurality of positive samples are predicted based on an anchor-within-frame algorithm to obtain a corresponding second loss.

4. The training method according to claim 3, characterized in that The sample allocation algorithm includes a SimOTA algorithm.

5. The training method according to claim 1, wherein: The target detection model includes a feature extraction network and a feature fusion network, and a first prediction network based on an anchor-free frame algorithm and a second prediction network based on an anchor-based frame algorithm arranged side by side; wherein, The feature extraction network is used to extract features from the sample image to obtain a feature-extracted image; The feature fusion network is connected to the feature extraction network and is used to perform feature fusion on the feature extraction image to obtain a feature fusion image; The first prediction network is connected to the feature fusion network, and is used to predict the feature fusion image based on an anchor-free frame algorithm to obtain a corresponding first loss; The second prediction network is connected to the feature fusion network and is used to predict the feature fusion image based on an anchor box algorithm to obtain a corresponding second loss.

6. The training method according to claim 5, characterized in that The feature extraction network includes a plurality of lightweight feature extraction modules, each of which outputs a feature extraction image to the feature fusion network; The output of one lightweight feature extraction module in two adjacent lightweight feature extraction modules is the input of the other lightweight feature extraction module, and the sizes of feature extraction images output by different lightweight feature extraction modules are different.

7. The training method according to claim 6, characterized in that The lightweight feature extraction module includes a hybrid network block ShuffleNet Block.

8. The training method according to claim 5, characterized in that The feature fusion network includes multiple lightweight feature fusion modules, and the input of one lightweight feature fusion module in two adjacent lightweight feature fusion modules is obtained according to the output of the other lightweight feature fusion module and one of the feature extraction images.

9. The training method according to claim 8, characterized in that The lightweight feature fusion module includes a ghost block.

10. A target detection method, characterized in that: include: Inputting the image to be tested into a trained target detection model to obtain a predicted target box predicted by the target detection model based on an anchor-free box algorithm; wherein the target detection model is trained using the training method according to any one of claims 1 to 9; Obtain a target detection result based on the predicted target frame.

11. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores a program, and the processor is used to execute the program to implement the training method described in any one of claims 1 to 9 or the target detection method described in claim 10.

12. A storage device, characterized in that: A program that can be executed by a processor is stored, and the program is used to implement the training method described in any one of claims 1 to 9 or the target detection method described in claim 10.

Images