A small target-oriented adaptive size perception focusing method and electronic equipment

Abstract

The application discloses a small-target-oriented adaptive size perception focusing method and electronic equipment, and the method comprises the following steps: acquiring a to-be-detected image containing a small target; and detecting the to-be-detected image by using a trained complete target detection network to obtain the position and category of the small target in the to-be-detected image; wherein the trained complete target detection network is obtained by training an adaptive small-target focusing weight function and a plurality of training samples; the adaptive small-target focusing weight function contains adaptive weights of the training samples, and the adaptive weight of each training sample is determined based on the data of all the training samples in the training batch in which the training sample is located. The application can better capture and locate the small target, thereby significantly improving the detection performance of the small target.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine learning, and particularly relates to a small target-oriented adaptive size perception focusing method and an electronic device. BACKGROUND

[0002] Target detection is an important task in the field of computer vision, aiming to automatically identify specific objects in images or videos and accurately mark their locations. Traditional target detection methods have achieved remarkable results, but in practical applications, they face problems such as high computational complexity, large model size, slow running speed, etc. Especially in mobile devices, embedded systems or resource-limited scenarios, their performance and efficiency are limited, especially in small target detection tasks, which face many challenges. Small targets usually refer to targets that occupy a very small pixel area in an image, and their characteristics include target blur, low resolution, high similarity to the background, and small target quantity.

[0003] However, traditional target detection algorithms perform poorly on small targets because they are often optimized for large targets while ignoring the characteristics of small targets. SUMMARY

[0004] To solve the above problems in the related art, the present application provides a small target-oriented adaptive size perception focusing method and an electronic device. The technical problem to be solved by the present application is solved by the following technical scheme:

[0005] The present application provides a small target-oriented adaptive size perception focusing method, comprising:

[0006] Obtaining a to-be-detected image containing a small target;

[0007] Detecting the to-be-detected image using a trained complete target detection network to obtain the position and category of the small target in the to-be-detected image; wherein the trained complete target detection network is trained using an adaptive small target focusing weight function and a plurality of training samples; the adaptive small target focusing weight function contains adaptive weights of training samples, and the adaptive weight of each training sample is determined based on data of all training samples in the training batch where the training sample is located.

[0008] In some embodiments, each training sample contains a ground truth box; the ground truth box is used to represent the position of the target contained in the training sample, and the adaptive weight of each training sample is determined according to a first preset value, a second preset value, a third preset value, the area of the ground truth box contained in the training sample, and the area of the largest ground truth box and the area of the smallest ground truth box in all training samples in the training batch where the training sample is located.

[0009] In some embodiments, each training sample contains a ground truth bounding box; the method for determining the adaptive weight of the training sample A comprises:

[0010] determining a first coefficient according to the first preset value, an area of a largest ground truth bounding box among all training samples in a training batch to which the training sample A belongs, and an area of a smallest ground truth bounding box;

[0011] determining a second coefficient according to a second preset value, a third preset value, the area of the largest ground truth bounding box, and the area of the smallest ground truth bounding box;

[0012] determining a difference between an area of a ground truth bounding box contained in the training sample A and the area of the smallest ground truth bounding box;

[0013] determining the adaptive weight of the training sample A according to an arctan function, the first coefficient, the second coefficient, and the difference.

[0014] In some embodiments, the expression of the first coefficient is:

[0015] β = X1 / (area_gt max -area_gt min );

[0016] wherein β represents the first coefficient, X1 represents the first preset value, area_gt max represents the area of the largest ground truth bounding box, and area_gt min represents the area of the smallest ground truth bounding box.

[0017] In some embodiments, the expression of the second coefficient is:

[0018] α = (1-(area_gt min / area_gt max ) X2 ) X3 ;

[0019] wherein α represents the second coefficient, X2 represents the second preset value, and X3 represents the third preset value.

[0020] In some embodiments, the expression of the adaptive weight of the training sample A is:

[0021] ω = 1 + α(1-arctan(β(x-area_gt min ));

[0022] Wherein, omega represents the adaptive weight of the training sample A, arctan() represents the arctan function, x represents the area of the real frame contained in the training sample A, beta represents the first coefficient, alpha represents the second coefficient, area_gt min represents the area of the minimum real frame.

[0023] In some embodiments, the expression of the adaptive small target focus weight function is:

[0024] L AFS = omega x L IoU ;

[0025] Wherein, L AFS represents the adaptive small target focus weight function, and the L IoU represents the IoU loss function.

[0026] In some embodiments, X1=6.228, X2=1 / 2, and X3=2.

[0027] In some embodiments, before the method detects the to-be-detected image by using the training-complete target detection network to obtain the position and category of the small target in the to-be-detected image, the method further comprises:

[0028] Obtaining a plurality of training samples; wherein each training sample contains a real frame and a real category of a target corresponding to the real frame;

[0029] In the training process, the initial target detection network is trained according to all training samples in each training batch and the adaptive small target focus weight function until the training-complete target detection network is obtained.

[0030] The application further provides an electronic device comprising a processor, a communication interface, a memory and a communication bus, characterized in that the processor, the communication interface and the memory complete communication with each other through the communication bus.

[0031] The memory is used for storing a computer program.

[0032] The processor is used for executing the program stored on the memory to realize the steps of the adaptive size-aware focus method for small targets.

[0033] The application has the following beneficial technical effects:

[0034] The adaptive small target focusing mechanism is introduced, in each training batch, the adaptive weight of each sample is determined according to the sample data in the batch instead of a fixed constant, which makes the network can adaptively adjust according to the target attribute in the sample, this adaptability can make the network balance the weight between different size targets, and can better capture and locate small targets, thereby significantly improving the detection performance of the network on small targets.

[0035] The application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A flowchart of the adaptive size perception focusing method for small targets provided by the embodiment of the application is shown in the figure.

[0037] Figure 2 An illustrative diagram of the adaptive weight ω in the adaptive small target focusing mechanism provided by the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0038] The application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0039] In the description of the application, the terms "first", "second", "third" and the like are used only to describe and distinguish one feature from another, and are not necessarily used to describe or imply these features' relative importance or a number of the features. Therefore, a feature with "first", "second", "third" or the like can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0040] In the description of the application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the application. In the description of the application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0041] Although the application is described herein with reference to various embodiments, one skilled in the art will understand that other changes in the described embodiments can be understood and effected by a skilled person in the art, and such changes are within the scope of the application, which is defined by the appended claims, by the disclosure given and by the accompanying drawings. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. A single processor or other unit can fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to an advantage.

[0042] The application aims to solve the small target detection problem which is common in target detection tasks. These small targets are often difficult to accurately detect and locate due to their small size, lack of details, and complex background. To this end, the application provides an adaptive size perception focusing method for small targets.

[0043] Figure 1 is a flowchart of the adaptive size perception focusing method for small targets provided by the embodiments of the application, as shown in Figure 1 The method comprises the following steps:

[0044] S101, obtaining a to-be-detected image containing a small target.

[0045] Here, the to-be-detected image can be any type of to-be-detected image, and the small target can be any type of small target.

[0046] S102, detecting the to-be-detected image using a trained complete target detection network to obtain the position and category of the small target in the to-be-detected image; wherein the trained complete target detection network is obtained by training using an adaptive small target focusing weight function and a plurality of training samples; the adaptive small target focusing weight function contains adaptive weights of the training samples, and the adaptive weight of each training sample is determined based on data of all training samples in the training batch in which the training sample is located.

[0047] Here, each training sample contains a ground truth box, which is used to represent the position of the target contained in the training sample. The adaptive weight of each training sample is determined according to a first preset value X1, a second preset value X2, a third preset value X3, the area of the ground truth box contained in the training sample, and the area of the largest ground truth box and the area of the smallest ground truth box in all training samples in the training batch in which the training sample is located.

[0048] Here, the target detection network can be any network used for target detection, for example, the EtinyNet-YOLOX network.

[0049] Specifically, the method for determining the adaptive weight of the training sample A is implemented through steps S201-S204.

[0050] S201, determining a first coefficient β according to a first preset value X1, an area area_gt of a maximum real box in all training samples of a training batch to which the training sample A belongs, and an area area_gt of a minimum real box. max min S201, determining a first coefficient β according to a first preset value X1, an area area_gt of a maximum real box in all training samples of a training batch to which the training sample A belongs, and an area area_gt of a minimum real box.

[0051] Specifically, the expression of the first coefficient β is: β = X1 / (area_gt max -area_gt min ). Exemplarily, X1 = 6.228.

[0052] S202, determining a second coefficient α according to a second preset value X2, a third preset value X3, the area area_gt of the maximum real box, and the area area_gt of the minimum real box. max min S202, determining a second coefficient α according to a second preset value X2, a third preset value X3, the area area_gt of the maximum real box, and the area area_gt of the minimum real box.

[0053] Specifically, the expression of the second coefficient α is: α = (1-(area_gt min / area_gt max )X2) X3 . In some embodiments, X2 = 1 / 2 and X3 = 2; in some embodiments, X2 = 1 / 3 and X3 = 3, which are not limited by the present application.

[0054] S203, determining a difference (x-area_gt min ) between an area x of a real box contained in the training sample A and the area area_gt of the minimum real box. min

[0055] S204, determining an adaptive weight ω of the training sample A according to an arctan function, the first coefficient β, the second coefficient α, and the difference (x-area_gt min ).

[0056] Specifically, the expression of the adaptive weight ω of the training sample A is: ω = 1+α(1-arctan(β(x-area_gt min )).

[0057] ​​​Here, according to the expression of the adaptive weight ω, the main body of the expression is a 1-arctan function. Since the area of the real box is always greater than 0, the value range of the 1-arctan function is (0, +∞), and it is a monotonically decreasing function in the interval, with a value range of (1, π / 2-1). When x = 1.557, the function value of the 1-arctan function is 0. 1-arctan(x-area_gtmin) represents a translation on the y-axis, so that the maximum value is obtained when x is the minimum real box. It is worth noting that x-area_gtmin = 1.557 is the zero point of the 1-arctan function, which is always greater than 0 on the left side of the zero point, and the derivative gradually increases as x decreases, and the change rate also increases; while on the right side of the zero point, the function value is less than zero, the change rate is slow and gradually converges to (π / 2-1).

[0058] In order to adjust the slope of the adaptive weight and control the increase and decrease space of the adaptive weight, the present application introduces a balance factor β. The size of the real box is equal to the product of the side length, and the median point of the side length of the maximum real box and the minimum real box is set as the boundary of the size target, so it can be approximated as the quarter point between the area of the maximum real box and the minimum real box, that is, x0 = (area_gtmax+3area_gtmin) / 4 as the balance point, so as to determine whether it is a small target sample. As shown in Figure 2 When the size of the real box is less than x0, it is determined to be a small target, and a weight greater than 0 is assigned to these boxes, while the boxes greater than x0 will be assigned a weight slightly less than 0. At this time, the balance point x0 is the zero point of the function (1-arctan(β(x-area_gtmin))), and x0 is substituted into β(x-area_gtmin) = 1.557 to calculate the value function of β. Through the above setting, the weight adjustment can be adaptively applied to targets of different sizes, thereby establishing a balance of the weight between different target sizes, ensuring that the model / network can better adapt to small targets while not ignoring large targets.

[0059] In addition, a is an adaptive expansion factor, the main purpose of a is to adaptively adjust the value range of the weight according to the ratio between the area of the smallest real box and the area of the largest real box in each batch, realize the automatic adjustment of the weight, and better adapt to targets of different sizes. The calculation of a adopts square root transformation and square operation, so as to be more sensitive to the change of the sample area ratio, because in many cases, the area of the smallest real box may be only one percent or even one thousandth of the area of the largest real box, and through this operation, the change of a between 0 to 0.1 can be amplified, and the weight is adjusted more smoothly. When the sample size difference in the training batch is small, a takes a smaller value, resulting in smaller changes of a (1-arctan (beta (x-area_gtmin))). In the extreme case, when the sample size is consistent, a is 0, and the weight omega remains unchanged; when the sample size difference is large, a takes a larger value, and then the adaptive weight exerts greater influence on small targets. Such weight adjustment helps to enhance the learning and detection ability of the model / network on small targets, thereby improving the performance of the model / network on small targets. Since a is calculated according to the size range of the real box in each training batch, it can adapt to the size distribution of targets in different datasets and scenes, and has wider applicability.

[0060] Specifically, the expression of the adaptive small target focusing weight function is: L AFS = omega x L IoU , L AFS represents the adaptive small target focusing weight function, L IoU represents the IoU loss function.

[0061] For example, L IoU may be a CIoU loss function, and thus the expression of L IoU may be:

[0062]

[0063]

[0064] Wherein, b and b gt represent the center point of a prediction box and the center point of a real box corresponding to the prediction box respectively, p represents the Euclidean distance between the two center points, c 2 represents the diagonal distance of the smallest closed region that can contain the prediction box and the real box, w, h and w gt , h gt represent the width, height of the prediction box and the width, height of the real box respectively.

[0065] In the present application, since the size of the real box has been calculated when calculating the IoU loss, LAFS Only a small amount of computation is added when training.

[0066] In some embodiments, S102 can include the following before S102:

[0067] S301, obtaining a plurality of training samples; wherein each training sample contains a real frame and a real class of a target corresponding to the real frame.

[0068] S302, in the training process, according to all training samples in each training batch and the adaptive small target focusing weight function, training the initial target detection network until the trained target detection network is obtained.

[0069] For example, when each training batch has N training samples, in each training, the N training samples are input into the network to be trained for forward propagation. When calculating the loss, the adaptive weight of each training sample is calculated according to the principles of S201-S204 above, the data of the prediction frame corresponding to each training sample is obtained, and then the IoU loss of each training sample is calculated according to the data of the prediction frame and the real frame corresponding to each training sample. Then, the adaptive small target focusing loss (i.e. AFS-IoU loss) of each training sample is obtained by multiplying the IoU loss of each training sample with the adaptive weight of each training sample. Then, the sum of the AFS-IoU losses of the N training samples or the average of the AFS-IoU losses of the N training samples can be used as the final loss calculated this time, and the network parameters of the trained network are back propagated according to the final loss to obtain the trained network for the next training. This iterative cycle continues until the preset number of cycles is reached or the network converges, and the training stops, thereby obtaining a target detection network that can more accurately capture and locate small targets and is trained.

[0070] For example, S301 and S302 can be implemented through the following steps 1 to 5:

[0071] Step 1, building a target detection network, wherein EtinyNet-YOLOX is used as the target detection network and CIoU is used as the IoU loss function.

[0072] Step 2, improving the IoU loss function of the network, introducing an adaptive small target focusing mechanism to adjust the loss function of the network, and obtaining L AFS .

[0073] Step 3, in each training batch, dynamically obtain the maximum and minimum real box information in the current batch, and in the process of forward propagation, the adaptive expansion factor and balance factor of the adaptive small target focusing mechanism are calculated by using the formula alpha = (1-(area_gt min / area_gt max ) 1 / 2 ) 2 and beta = 6.228 / (area_gt max -area_gt min ), and then the adaptive weight of each sample is calculated by omega = 1+alpha(1-arctan(beta(x-area_gt min ))).

[0074] Step 4, multiply the IoU loss of each sample by the adaptive weight calculated in step 3 to obtain the AFS-IoU loss, and then further forward propagation and back propagation are carried out based on the AFS-IoU loss, so that the network is more focused on processing small target samples, thereby more accurately capturing and positioning small targets.

[0075] Step 5, after the training is completed, the trained model is tested for accuracy to evaluate its performance.

[0076] The application also provides an electronic device comprising a processor, a communication interface, a memory and a communication bus, characterized in that the processor, the communication interface and the memory complete communication with each other through the communication bus; the memory is used to store a computer program; and the processor is used to execute the program stored on the memory to realize the steps of the adaptive size perception focusing method for small targets.

[0077] The effects of the application can be further illustrated by the following experimental data.

[0078] In order to verify the performance of the present application under different conditions, the model / network is trained using visdrone and self-made power line foreign object data sets. The visdrone data set mainly contains micro targets taken by a drone, while the self-made power line foreign object data set covers a large number of large target samples and a small number of micro target samples, and the sample size distribution is more uneven. Through these two data sets, the performance of the present application under different conditions can be verified. The present application is verified on EtinyNet-YOLOX, and compared on the visdrone small target data set and the self-made power line foreign object data set. The AdamW optimizer and the cosine annealing learning rate strategy are used for training, the initial learning rate is 0.001, and a total of 300 epochs are trained. The comparison results are listed in Table 1, which shows the average time of training one round under the same conditions using the present application and the original IoU loss, and the precision on the visdrone data set and the self-made power line foreign object data set.

[0079] Training time visdrone Foreign matter on power lines EtinyNet+IoU 160s 19.25％ 93.2％ EtinyNet+AFS-IoU 164s 19.91％ 95.62％

[0080] Table 1

[0081] As can be seen from Table 1, by using the method of the present application to improve the IoU loss function and training on the visdrone and power line foreign object data sets, only a very small amount of calculation is added, but the detection accuracy is improved by 0.69% and 2.41% respectively, which shows that the present application has achieved significant improvement effect in small target detection, especially for the case of sample imbalance, it has more advantages than the original IoU loss. Such experimental data further proves the effectiveness and potential of the present application in target detection tasks.

[0082] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, which should be regarded as falling within the protection scope of the present application.

Claims

1. An adaptive size-aware focusing method for small targets, characterized in that, include: Acquire the image to be detected containing small targets; A fully trained object detection network is used to detect the image to be detected, and the position and category of the small target in the image to be detected are obtained; wherein, the fully trained object detection network is trained using an adaptive small target focusing weight function and multiple training samples; the adaptive small target focusing weight function includes adaptive weights of training samples, and the adaptive weight of each training sample is determined based on the data of all training samples in the training batch in which the training sample belongs. Each training sample contains a ground truth bounding box. Methods for determining the adaptive weights of training sample A include: Based on a first preset value, and the areas of the largest and smallest ground truth bounding boxes among all training samples in the training batch to which training sample A belongs, a first coefficient is determined; the expression for the first coefficient is: β = X 1 / (area_gt max - area_gt min ) ,in, β This represents the first coefficient. X 1 represents the first preset value. area_gt max This represents the area of ​​the largest true bounding box. area_gt min This represents the area of ​​the smallest true bounding box; A second coefficient is determined based on the second preset value, the third preset value, the area of ​​the largest true bounding box, and the area of ​​the smallest true bounding box; the expression for the second coefficient is: α= ( 1 -(area_gt min / area_gt max ) X2 ) X3 ,in, α This represents the second coefficient. X 2 represents the second preset value. X 3 represents the third preset value; Determine the difference between the area of ​​the ground truth bounding box contained in the training sample A and the area of ​​the smallest ground truth bounding box; The adaptive weights of the training sample A are determined based on the arctan function, the first coefficient, the second coefficient, and the difference; the expression for the adaptive weights of the training sample A is: ω=1+α ( 1 -arctan(β (x-area_gt min ))) ,in, ω This represents the adaptive weights of the training sample A. arctan() This represents the arctan function. x This represents the area of ​​the ground truth bounding box contained in the training sample A.

2. The adaptive size-aware focusing method for small targets according to claim 1, characterized in that, Each training sample contains a ground truth bounding box, which is used to characterize the location of the target contained in the training sample; the adaptive weight of each training sample is determined based on a first preset value, a second preset value, a third preset value, the area of ​​the ground truth bounding box contained in the training sample, and the areas of the largest and smallest ground truth bounding boxes among all training samples in the training batch to which the training sample belongs.

3. The adaptive size-aware focusing method for small targets according to claim 1, characterized in that, The expression for the adaptive small target focusing weight function is: L AFS = ω×L IoU ； in, L AFS This represents the adaptive small target focusing weight function, the L IoU express IoU Loss function.

4. The adaptive size-aware focusing method for small targets according to claim 1, characterized in that, X 1 = 6.228， X 2 = 1 / 2， X 3 = 2。 5. The adaptive size-aware focusing method for small targets according to claim 1, characterized in that, Before using a fully trained object detection network to detect the image to be detected and obtaining the location and category of the small target in the image to be detected, the method further includes: Obtain the plurality of training samples; wherein each training sample contains a ground truth bounding box and the ground truth category of the target corresponding to the ground truth bounding box; During the training process, the initial target detection network is trained based on all training samples in each training batch and the adaptive small target focusing weight function until the fully trained target detection network is obtained.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, characterized in that the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the steps of the method described in any one of claims 1-5.

Images