Focused Video Moving Small Target Detection Method Based on Difficulty Perception

Abstract

The present invention relates to a focused video moving small target detection method based on difficulty perception, which includes the steps of: constructing a difficulty perception model for video moving small targets; allocating labels corresponding to n sub-networks and detection head branches of the difficulty perception model for video moving small targets, inputting images into the difficulty perception model for video moving small targets for training to obtain a convolutional neural network model for difficulty perception of video moving small targets; inputting the images obtained in real time into the convolutional neural network model for difficulty perception of video moving small targets to obtain detection results; intercepting the motion regions of the corresponding number of frames of targets according to the target difficulty level to construct a spatio-temporal cube of the targets; and inputting the spatio-temporal cube into a lightweight fine detection model based on the spatio-temporal cube for focused detection to predict the category and position of the targets. When there are targets with various difficulty levels in the dataset, the model of the present invention can perceive the difficulty level of the targets and allocate appropriate computing resources.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for detecting small moving targets in a focused video based on difficulty perception. Background Art

[0002] When humans observe small moving targets in video images, such as flying birds in video images, they are clearer and easier to identify when they are closer to the camera and flying slowly; they are not easy to identify when they are farther from the camera, or the background is complex, or they are flying faster. The recognition methods mainly fall into two types: The first type does not distinguish the difficulty levels of targets and uniformly treats them as targets that are more difficult to identify. This type of method can achieve certain effects but does not conform to the actual situation; the second type uses a model in which the more difficult the target, the more complex the processing procedure to a certain extent. This type of method determines the difficult targets based on the predicted scores of the targets, but it cannot determine whether the model makes a wrong prediction and thus conducts wrong processing. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for detecting small moving targets in a focused video based on difficulty perception when there are scenarios in the dataset where targets are easy to identify, not easy to identify, more difficult to identify, etc., so as to improve the detection rate of difficult targets and reduce the false detection rate.

[0004] In order to solve the problem that when there are targets with various difficulty levels in the dataset, the model cannot distinguish the difficulty levels of the targets and conduct corresponding detections, the embodiments of the present invention provide the following technical solutions:

[0005] A method for detecting small moving targets in a focused video based on difficulty perception includes the following steps:

[0006] Step 1, construct a model for perceiving the difficulty of small moving targets in a video;

[0007] Step 2, assign labels to the n sub-networks of the model for perceiving the difficulty of small moving targets in a video and the corresponding detection head branches, input the image into the model for perceiving the difficulty of small moving targets in a video for training, and obtain a convolutional neural network model for perceiving the difficulty of small moving targets in a video;

[0008] Step 3, input the image obtained in real time into the convolutional neural network model for perceiving the difficulty of small moving targets in a video to obtain a detection result; according to the target difficulty level, intercept the moving regions of the corresponding number of frames of the target to construct a spatio-temporal cube of the target; input the spatio-temporal cube into a lightweight fine-detection model based on the spatio-temporal cube for focused detection, and predict the category and position of the target.

[0009] In the above solution, the video moving small target difficulty perception model has n sub-networks and a detection head branch, which is used to perceive the difficulty level of the target, reasonably allocate computing resources according to the difficulty level of the target, and allocate a longer path for processing targets with a higher difficulty level, which can improve the detection rate of difficult targets and reduce the false detection rate.

[0010] The video moving small target difficulty perception model includes a feature aggregation network, a feature extraction backbone network, n sub-networks, n detection heads, and n - 1 spatial attention modules; among them, the output end of the feature aggregation network is connected to the input end of the feature extraction backbone network, the feature extraction backbone network is connected to the input end of the first sub-network, each sub-network is connected to each detection head in one-to-one correspondence, and the output end of each sub-network is also connected to the input end of a spatial attention module, and the output end of this spatial attention module is connected to the input end of the next sub-network until the output end of the (n - 1)th attention module is connected to the input end of the nth sub-network.

[0011] Before the step 2, it also includes the steps:

[0012] The difficulty levels of the targets are divided into n difficulty levels, namely difficulty level 0, difficulty level 1,..., difficulty level n - 1, where difficulty level 0 is the easiest difficulty level to recognize, and difficulty level n - 1 is the most difficult difficulty level to recognize;

[0013] Among them, the first sub-network and the detection head branch are responsible for predicting targets of all difficulty levels, the second sub-network and the detection head branch are responsible for predicting targets except those with difficulty level 0, the third sub-network and the detection head branch are responsible for predicting targets except those with difficulty levels 0 and 1, and so on, the (n - 1)th sub-network and the detection head branch are responsible for predicting targets with difficulty levels (n - 2) and (n - 1), and the nth sub-network and the detection head branch are responsible for predicting targets with difficulty level (n - 1).

[0014] In the above solution, according to the difficulty level of the target, for targets with a higher difficulty level, more computing power is invested for processing, while for targets with a lower difficulty level, less computing power is used, thus reasonably allocating computing resources. For difficult targets, the recognition is more accurate, and for simple targets, recognition resources are saved.

[0015] In the step 2, the step of allocating labels corresponding to the n sub-networks and the detection head branch of the video moving small target difficulty perception model includes:

[0016] Label the scores of the targets in consecutive multiple frames of images to form labels corresponding to each frame of image, and based on the scores of the targets in the labels, divide the targets into n difficulty levels;

[0017] Randomly select a batch of images and their corresponding labels, and re - allocate the labels according to the difficulty level of the targets in the images. Divide the labels into n groups. The first group contains the labels of all targets, corresponding to the first sub - network and the detection head branch of the video motion small target difficulty perception model; the second group contains the labels of targets except those with a difficulty level of 0, corresponding to the second sub - network and the detection head branch; the third group contains the labels of targets except those with difficulty levels of 0 and 1, corresponding to the third sub - network and the detection head branch; and so on. The (n - 1)th group contains the labels of targets with difficulty levels of (n - 2) and (n - 1), corresponding to the (n - 1)th sub - network and the detection head branch; the nth group contains the labels of targets with a difficulty level of (n - 1), corresponding to the nth sub - network and the detection head branch.

[0018] In the above - mentioned scheme, allocate corresponding labels to the n sub - networks and the detection head branches, so that the model can perceive targets with different levels of difficulty at different structural stages.

[0019] In step 2, the steps of inputting the image into the video motion small target difficulty perception model for training to obtain the video motion small target difficulty perception convolutional neural network model include:

[0020] After allocating the labels corresponding to the n sub - networks and the detection head branches, input the image into the video motion small target difficulty perception model for training. Through forward inference, obtain the result output by the video motion small target difficulty perception model;

[0021] During the training process, use the confidence loss function, the class classification loss function, and the localization regression loss function to calculate the loss between the video motion small target difficulty perception model and the corresponding labels;

[0022] Use the score of the target to weight the loss corresponding to each target, and update the weights of the video motion small target difficulty perception model through loss backpropagation; until after training a certain number of steps or the loss converges to the expected value, fix the weights of the video motion small target difficulty perception model to obtain the video motion small target difficulty perception convolutional neural network model.

[0023] In step 3, the steps of inputting the real - time acquired image into the video motion small target difficulty perception convolutional neural network model to obtain the detection result, and intercepting the motion regions of the corresponding number of frames of the target according to the target difficulty level to construct the spatio - temporal cube of the target include:

[0024] Input a sequence of n consecutive frames of images {x} into the trained video motion small target difficulty perception convolutional neural network model to obtain the output out = HardEasyAwearModel({x});

[0025] Post-process the output of the convolutional neural network model for the perception of difficult and easy video moving small targets to obtain the detection results , where represents the target position on the j-th frame image, and P IDk represents the position of the target with the predicted ID number k, and j ∈ n;

[0026] Use the target tracking algorithm to track the detection results:

[0027]

[0028] where represents the position of the target with ID k on consecutive frame images;

[0029] Determine the number of consecutive frames that need to be focused on for targets of different difficulty levels according to the difficulty level of the target;

[0030] Use the results of target tracking, based on the difficulty level of the target, intercept the moving area of the target on the images of the number of consecutive frames that need to be focused on; intercept the moving area of the target on consecutive frame images and splice and construct the spatio-temporal cube STCube of the target IDk .

[0031] In the above solution, intercept the moving areas of multiple frames of images and perform focused fine detection, which can make the detection results more accurate.

[0032] Compared with the prior art, the beneficial effects of the present invention:

[0033] The present invention relates to a model for the perception of difficult and easy video moving small targets, enabling more difficult-to-detect targets to have a longer processing path; training the model for the perception of difficult and easy video moving small targets using a dataset containing target score (difficulty level) labels, enabling the model to perceive the difficulty level of video moving small targets and achieving the purpose of model design; using the target tracking algorithm to track the moving small targets detected by the model, intercepting spatio-temporal cubes of different durations according to the difficulty level of the moving small targets, and inputting the spatio-temporal cube into a lightweight fine detection model for focused detection, thereby realizing the detection of focused video moving small targets based on difficulty perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0035] Figure 1Schematic diagram of the structure of the video motion small target difficulty perception model of the present invention;

[0036] Figure 2 A detailed schematic diagram of the video motion small target difficulty perception model in Embodiment 2 of the present invention;

[0037] Figure 3 Schematic diagram of the training process of the video motion small target difficulty perception model in Embodiment 2 of the present invention;

[0038] Figure 4 Schematic diagram of the input of the video motion small target difficulty perception model in Embodiment 2 of the present invention;

[0039] Figure 5 Schematic diagram of constructing a spatio-temporal cube in Embodiment 2 of the present invention;

[0040] Figure 6 Schematic diagram of the focused target detection process based on the difficulty level in Embodiment 2 of the present invention. Detailed implementation manners

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents the selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0042] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used for distinguishing descriptions, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "coupled", etc. can be directly connected between components, or indirectly connected through other components.

[0043] Embodiment 1:

[0044] The present invention is realized through the following technical solutions. As Figure 1 shown, the focused video motion small target detection method based on difficulty perception includes the following steps:

[0045] Step 1, construct a video moving small target difficulty perception model.

[0046] Please refer to Figure 1 , the video moving small target difficulty perception model includes a feature aggregation network, a feature extraction backbone network, n sub-networks, n detection heads, and n - 1 spatial attention modules. Among them, the output end of the feature aggregation network is connected to the input end of the feature extraction backbone network, the feature extraction backbone network is connected to the input end of the first sub-network, each sub-network is connected to each detection head in one-to-one correspondence, and the output end of each sub-network is also connected to the input end of a spatial attention module. The output end of this spatial attention module is connected to the input end of the next sub-network until the output end of the (n - 1)th attention module is connected to the input end of the nth sub-network.

[0047] The feature aggregation network is before the feature extraction backbone network and is used to perform feature aggregation on consecutive multiple frames of images. Since there are some moving small targets with unclear features in a single frame of image, if feature extraction is performed first and then feature aggregation, it may lead to the problem of feature loss. Therefore, this solution proposes to use the feature aggregation network to perform feature aggregation on consecutive multiple frames of images before feature extraction, which can avoid the problem of moving small targets being lost due to unclear features.

[0048] The feature extraction backbone network is used to perform high-level abstract feature extraction on the moving small targets after feature aggregation. Each sub-network and its corresponding connected detection head serve as a branch, responsible for predicting the confidence position and category information of the moving small targets.

[0049] This solution divides the difficulty level of the target into n difficulty levels, such as difficulty level 0, difficulty level 1,..., difficulty level n - 1, where difficulty level 0 is the easiest to recognize and difficulty level n - 1 is the most difficult to recognize. Among them, the first sub-network and detection head branch are responsible for predicting targets of all difficulty levels, the second sub-network and detection head branch are responsible for predicting targets except those with difficulty level 0, the third sub-network and detection head branch are responsible for predicting targets except those with difficulty levels 0 and 1, and so on. The (n - 1)th sub-network and detection head branch are responsible for predicting targets with difficulty levels (n - 2) and (n - 1), and the nth sub-network and detection head branch are responsible for predicting targets with difficulty level (n - 1).

[0050] The spatial attention module is connected between two adjacent sub-networks and is used to filter out targets with relatively low difficulty levels. Because relatively simple targets do not require too long a processing path and time, the model is made to pay attention to targets with relatively high difficulty levels so that the subsequent model structure only processes targets with relatively high difficulty levels.

[0051] Step 2: Assign labels to the n sub-networks of the video moving small target difficulty perception model and the corresponding detection head branches, and input the images into the video moving small target difficulty perception model for training to obtain the video moving small target difficulty perception convolutional neural network model.

[0052] After constructing the video moving small target difficulty perception model in Step 1, in order to enable the model to have the ability to perceive the difficulty level of the target, it is necessary to train the model using a dataset of moving small targets with different difficulty levels. In this embodiment, it is considered that the difficulty level of the target is reflected by a score, and the range of the score is 0 - 1. The larger the score, the more difficult the target is, and the higher the difficulty level of the target.

[0053] Annotate the scores of the targets in consecutive multiple frames of images (one or more targets can exist in one frame of image) to form the labels corresponding to each frame of image. According to the scores of the targets in the labels, divide the targets into different difficulty levels, and there are a total of n difficulty levels. For example, the targets with scores in have a difficulty level of 0, which is convenient for subsequent training of the model using the difficulty level of the target.

[0054] Randomly select a small batch (Mini Batch, the batch size is greater than or equal to 1) of images and the corresponding labels, and reassign the labels according to the difficulty level of the targets in the images. The assignment principle for one batch is as follows: Divide the labels into n groups. The first group contains the labels of all targets, corresponding to the first sub-network of the video moving small target difficulty perception model and the corresponding detection head branch; the second group contains the labels of the targets except those with a difficulty level of 0, corresponding to the second sub-network and the corresponding detection head branch; the third group contains the labels of the targets except those with difficulty levels of 0 and 1, corresponding to the third sub-network and the corresponding detection head branch; and so on. The (n - 1)th group contains the labels of the targets with difficulty levels of (n - 2) and (n - 1), corresponding to the (n - 1)th sub-network and the corresponding detection head branch; the nth group contains the labels of the targets with a difficulty level of (n - 1), corresponding to the nth sub-network and the corresponding detection head branch.

[0055] After assigning the labels corresponding to the n sub-networks and the detection head branches, input the images into the video moving small target difficulty perception model for training. Through forward inference, the first detection head predicts targets of all difficulty levels. At this time, the confidence scores of the targets with higher difficulty levels are low, indicating that they cannot be correctly recognized. By setting up a spatial attention module, filter out the targets with high confidence scores, which means the targets that have been predicted, and do not continue to propagate backward. Only propagate the targets with higher difficulty levels that have not been predicted, and continue to be predicted by the subsequent detection heads, so as to obtain the output result of the video moving small target difficulty perception model.

[0056] During the training process, a confidence loss function, a class classification loss function, and a localization regression loss function are used to calculate the loss between the video moving small target difficulty perception model and the corresponding labels. The scores of the targets are used to weight the losses corresponding to each target, which can appropriately increase with the growth of the model processing path to improve the scores of the targets. The loss is backpropagated to update the weights of the video moving small target difficulty perception model.

[0057] Re-select a batch of images and the corresponding labels, reassign the labels according to the difficulty levels of the targets in the images, and repeat the above steps until after a certain number of training steps or the loss converges to the expected value, then fix the weights of the video moving small target difficulty perception model to obtain the video moving small target difficulty perception convolutional neural network model. After being trained in this way, the video moving small target difficulty perception convolutional neural network model has the ability to perceive the difficulty levels of the targets.

[0058] Step 3: Input the images obtained in real time into the video moving small target difficulty perception convolutional neural network model to obtain the detection results; according to the target difficulty levels, intercept the motion regions of the targets for the corresponding number of frames to construct the spatio-temporal cubes of the targets; input the spatio-temporal cubes into the lightweight fine detection model based on the spatio-temporal cubes for focused detection to predict the classes and positions of the targets.

[0059] Input the images obtained in real time into the video moving small target difficulty perception convolutional neural network model to obtain the output of the model, then post-process the output of the model to obtain the detection results, and use the target tracking algorithm to track the detection results.

[0060] According to the target difficulty levels, determine the number of consecutive frames that need to be focused on for targets with different difficulty levels. For example, when the difficulty level is 0, the number of consecutive frames that need to be focused on is set to 1; when the difficulty level is 1, the number of consecutive frames that need to be focused on is set to 3. Using the results of target tracking, according to the target difficulty levels, intercept the motion regions of the targets on the images with the number of consecutive frames that need to be focused on to construct the spatio-temporal cubes of the targets.

[0061] Input the spatio-temporal cubes of the targets into a lightweight fine detection model based on the spatio-temporal cubes to finely predict the classes and positions of the targets and avoid false detections.

[0062] In summary, the video moving small target difficulty perception model designed by the present invention can perceive the difficulty levels of moving small targets, reasonably allocate computing resources for targets with different difficulty levels, on the one hand, improve the detection rate of difficult targets, and on the other hand, save the computing resources of simple targets.

[0063] Embodiment 2:

[0064] Based on the above Embodiment 1, this embodiment gives specific and implementable parameters. The structural schematic diagram of the video moving small target difficulty perception model is as Figure 1 shown. In this embodiment, the difficulty level n is taken as 3. The detailed structural schematic diagram of the video moving small target difficulty perception model is as Figure 2 shown.

[0065] The feature aggregation network uses a convolutional long short-term memory network (ConvLSTM) to aggregate the features of the target on 5 consecutive frames of images. The feature extraction backbone network uses a mainstream feature extraction network, such as DarkNet53. The sub-network selects a U-shaped sub-network. When upsampling, the convolutional feature layer is concatenated with the downsampling feature layer of the same scale in the channel dimension to fully fuse the shallow positioning information and the high-level semantic information. The detection head predicts the confidence, category, and location of the target.

[0066] To implement the training method, the dataset labels need to have the category, location, and a score objectively reflecting the target difficulty level (the lower the score of the target, the higher the difficulty level and the more difficult). How to determine the specific score of the target is not within the scope of this invention. The model training has the following steps:

[0067] Step 11, the difficulty level series n = 3 has been determined. According to the scores of the targets in the labels, the targets in the labels are divided into difficulty levels. The division rule is that the difficulty level of the targets with scores in (0.667, 1] is 0; the difficulty level of the targets with scores in (0.334, 0.667] is 1; the difficulty level of the targets with scores in (0, 0.334] is 2.

[0068] Step 12, please refer to Figure 3 , randomly select a small batch of images and the corresponding labels from the dataset, and perform label assignment according to the difficulty level of the targets. The assignment principle for one batch is to divide the labels into 3 groups. The first group contains the labels of all targets (targets with difficulty levels 0, 1, and 2), corresponding to the first sub-network and the detection head branch; the second group contains the labels of the targets with difficulty levels 1 and 2, corresponding to the second sub-network and the detection head branch; the third group contains the labels of the targets with difficulty level 2, corresponding to the third sub-network and the detection head branch.

[0069] Step 13, input the 5-consecutive-frame image sequence {x} into the video moving small target difficulty perception model. After forward inference, obtain the output out = HardAwareModel({x}) of the video moving small target difficulty perception model. In this embodiment, the position of the target on the middle frame of the consecutive frame images is predicted as the detection result.

[0070] Step 14, calculate the target loss function using the confidence loss function, the category classification loss function, and the localization regression loss function:

[0071]

[0072] where l o represents the loss function of target o; l conf (·, ·) represents the confidence loss function, and l cls (·, ·) represents the class classification loss function, and l loc (·, ·) represents the localization regression loss function; out conf represents the output corresponding to the confidence loss function, and out cls represents the output corresponding to the class classification loss function, and out loc represents the output corresponding to the localization regression loss function; represents the label of target o in the confidence loss function, represents the label of target o in the class classification loss function, represents the label of target o in the localization regression loss function.

[0073] Step 15: Use the target score to weight the loss of the target to obtain the positive sample loss:

[0074]

[0075] where l pos is the positive sample loss; O represents all targets in this training.

[0076] Step 16: Calculate the total loss:

[0077]

[0078] where l is the total loss; l neg is the negative sample loss; count(·) represents the total number of targets.

[0079] Step 17: Perform loss backpropagation to update the weights of the video moving small target difficulty perception model.

[0080] Step 18: Please refer to Figure 4 , repeat Steps 12 to 17. After training for a certain number of steps or when the loss converges to the expected value, fix the weights of the video moving small target difficulty perception model to obtain the video moving small target difficulty perception convolutional neural network model.

[0081] After the model is trained, input the continuously acquired real-time frame images into the video moving small target difficulty perception convolutional neural network model for target detection. The target detection has the following steps:

[0082] Step 21: Input a sequence of 5 consecutive frames of images {x} into the trained convolutional neural network model for perceiving the difficulty of small moving targets in videos, and obtain the output out = HardEasyAwearModel({x}).

[0083] Step 22: Post-process the output of the model to obtain the detection result , where represents the target position on the 3rd frame image, and P IDk represents the position of the target with the predicted ID number k.

[0084] Step 23: Refer to Figure 5 , and use the SORT object tracking algorithm to track the detection result obtained in step (22) , where represents the position of the target with ID k on consecutive frame images.

[0085] Step 24: Determine the number of consecutive frames that need to be focused on for targets of different difficulty levels according to the difficulty level of the target. In this embodiment, the settings are as follows: when the difficulty level is 0, the number of consecutive frames that need to be focused on is 1; when the difficulty level is 1, the number of consecutive frames that need to be focused on is 3; when the difficulty level is 2, the number of consecutive frames that need to be focused on is 5.

[0086] Step 25: Use the result of object tracking, and calculate the motion area of the target on the images of the number of consecutive frames that need to be focused on according to the difficulty level of the target. Specifically, calculate the minimum bounding rectangle of the target's bounding box on consecutive frame images, and this minimum bounding rectangle is the motion area of the target on consecutive frame images.

[0087] Step 26: Refer to Figure 6 , intercept the motion area of the target on consecutive frame images, and then splice and construct the spatio-temporal cube STCube of the target IDk , input it into a lightweight fine-detection model LightWeightDetectionModel based on the spatio-temporal cube, and obtain an accurate detection result after post-processing.

[0088] As mentioned above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A focused video moving small target detection method based on difficulty perception, characterized in that: It includes the following steps: Step 1, construct a video moving small target difficulty perception model; Step 2, allocate labels corresponding to the n sub-networks and detection head branches of the video moving small target difficulty perception model, input the image into the video moving small target difficulty perception model for training, and obtain a video moving small target difficulty perception convolutional neural network model; Before the said Step 2, it further includes the step: Divide the difficulty level of the target into n difficulty levels, namely difficulty level 0, difficulty level 1,..., difficulty level n-1, where difficulty level 0 is the easiest to recognize difficulty level, and difficulty level n-1 is the most difficult to recognize difficulty level; Among them, the first sub-network and detection head branch are responsible for predicting targets of all difficulty levels, the second sub-network and detection head branch are responsible for predicting targets except those with difficulty level 0, the third sub-network and detection head branch are responsible for predicting targets except those with difficulty levels 0 and 1, and so on. The (n-1)th sub-network and detection head branch are responsible for predicting targets with difficulty levels (n-2) and (n-1), and the nth sub-network and detection head branch are responsible for predicting targets with difficulty level (n-1); Step 3, input the real-time acquired image into the video moving small target difficulty perception convolutional neural network model to obtain a detection result; according to the target difficulty level, intercept the motion area of the target corresponding to the corresponding number of frames, and construct a spatio-temporal cube of the target; input the spatio-temporal cube into the lightweight fine detection model based on the spatio-temporal cube for focused detection, and predict the category and position of the target; The said Step 3 specifically includes: Input the continuous n-frame image sequence {x} into the trained video moving small target difficulty perception convolutional neural network model to obtain the output out = HardEasyAwearModel({x}); Post-process the output of the convolutional neural network model for the perception of small moving targets in videos to obtain the detection results , where represents the target position on the j-th frame image, and P IDk represents the position of the target with the predicted ID number k, where j ∈ n; Use the target tracking algorithm to track the detection result, and determine the number of consecutive frames that need to be focused on for targets of different difficulty levels according to the target difficulty level; Using the result of target tracking, according to the difficulty level of the target, on the images of consecutive frames that need to be focused on, intercept the moving area of the target; intercept the moving area of the target on consecutive frame images and splice and construct the spatio-temporal cube STCube of the target IDk 。 2. The method for detecting small moving targets in a focused video based on difficulty perception according to claim 1, characterized in that: The said video moving small target difficulty perception model includes a feature aggregation network, a feature extraction backbone network, n sub-networks, n detection heads, and n-1 spatial attention modules; among them, the output end of the feature aggregation network is connected to the input end of the feature extraction backbone network, the feature extraction backbone network is connected to the input end of the first sub-network, each sub-network is connected to each detection head in one-to-one correspondence, and the output end of each sub-network is also connected to the input end of a spatial attention module, and the output end of this spatial attention module is connected to the input end of the next sub-network until the output end of the (n-1)th attention module is connected to the input end of the nth sub-network.

3. The method for detecting small moving targets in a focused video based on difficulty perception according to claim 2, wherein: In the said Step 2, the step of allocating labels corresponding to the n sub-networks and detection head branches of the video moving small target difficulty perception model includes: Annotate the scores of the targets in the continuous multi-frame images to form the labels corresponding to each frame of image, and divide the targets into n difficulty levels according to the scores of the targets in the labels. Randomly select a batch of images and their corresponding labels, reassign the labels according to the difficulty level of the targets in the images, divide the labels into n groups. The first group contains the labels of all targets, corresponding to the first sub-network and the detection head branch of the video motion small target difficulty perception model; the second group contains the labels of targets except those with a difficulty level of 0, corresponding to the second sub-network and the detection head branch; the third group contains the labels of targets except those with difficulty levels of 0 and 1, corresponding to the third sub-network and the detection head branch; and so on. The (n - 1)-th group contains the labels of targets with difficulty levels of (n - 2) and (n - 1), corresponding to the (n - 1)-th sub-network and the detection head branch; the n-th group contains the labels of targets with a difficulty level of (n - 1), corresponding to the n-th sub-network and the detection head branch.

4. The method for detecting small moving targets in a focused video based on difficulty perception according to claim 3, wherein: In step 2, the steps of inputting the images into the video motion small target difficulty perception model for training to obtain the video motion small target difficulty perception convolutional neural network model include: After distributing the labels corresponding to the n sub-networks and the detection head branches, input the images into the video motion small target difficulty perception model for training, and through forward inference, obtain the results output by the video motion small target difficulty perception model; During the training process, use the confidence loss function, the class classification loss function, and the localization regression loss function to calculate the loss between the video motion small target difficulty perception model and the corresponding labels; Use the scores of the targets to weight the losses corresponding to each target, and backpropagate the losses to update the weights of the video motion small target difficulty perception model; until after training a certain number of steps or the loss converges to the expected value, fix the weights of the video motion small target difficulty perception model to obtain the video motion small target difficulty perception convolutional neural network model.

Images