A high-altitude small target detection method based on video stream data suitable for complex environments

Abstract

The present invention discloses a high-altitude small target detection method based on video stream data suitable for complex environments. First, the input video is divided, and detection is performed every 6 frames, using a neural network and a traditional optical flow method and image calibration technology respectively. The optical flow method is used to track feature points, accurately calculate the transformation matrix between two video frames, and then the frame difference method is used to separate the foreground and background. This method overcomes difficulties such as low video quality and lens shaking, and then combines the candidate targets of the neural network, screens them according to their confidence, and separates the final results. In the case of low video quality and complex environment, the present invention can significantly improve the detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular to a method, system, computer equipment and storage medium for detecting high-altitude small targets based on video stream data suitable for complex environments. Background Art

[0002] The task of object detection is to find all the objects of interest in the image, determine their categories and positions, and object detection is one of the core issues in the field of computer vision. Since various objects have different appearances, shapes and postures, and are interfered by factors such as illumination and occlusion during imaging, object detection has always been the most challenging problem in the field of computer vision. Traditional object detection methods are generally divided into three stages: First, select some candidate regions on a given image. This step is to locate the position of the target. Since the target may appear at any position in the image, and the size and aspect ratio of the target are uncertain, different scales and aspect ratios need to be set. Secondly, extract features from these regions. Commonly used features include SIFT and HOG. Finally, use the trained classifier for classification. The main classifiers are SVM, AdaBoost, etc. Traditional object detection has many shortcomings. First, the time complexity of candidate region generation is high, which affects the speed and performance of subsequent feature extraction and classification. Second, manual feature extraction makes it impossible to guarantee the overall diversity and robustness of the algorithm. For targets of smaller size, their features are likely to be ignored.

[0003] Nowadays, the improvement of computer performance has made deep learning algorithms the mainstream method in the field of target detection. In particular, in 2012, AlexNet based on CNN won the ImageNet image recognition competition, and neural networks have received widespread attention since then. Deep learning uses multi-layer computing models to learn abstract data representations and can discover complex structures in big data. At present, this technology has been successfully applied to various pattern classification problems including the field of computer vision. Common target detection algorithms such as the R-CNN series and the YOLO series have all demonstrated strong detection accuracy. The disadvantage is that the deep neural network of this type of algorithm relies on hardware performance and has a high computational cost.

[0004] Video-based object detection, because the objects in the video have different postures and are often blocked, and their movements are irregular, and considering the depth of field, resolution, weather, lighting and other conditions of the surveillance video and the diversity and complexity of the scene, and the result of the object detection algorithm will directly affect the subsequent tracking, action recognition and behavior description. Therefore, even in today's technological development, the basic task of video object detection is still a very challenging topic, and there is great potential and room for improvement. Especially for airborne computers, their performance is limited and neural networks are difficult to deploy. Secondly, the image blur caused by camera shooting jitter, instability and other factors also seriously limits the accuracy of object detection. Summary of the invention

[0005] The purpose of the present invention is to provide a high-altitude small target detection method based on video stream data suitable for complex environments in order to address the problems existing in the above-mentioned prior art.

[0006] The technical solution to achieve the purpose of the present invention is: a high-altitude small target detection method based on video stream data suitable for complex environments, the method comprising the following steps:

[0007] Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0008] Step 2: Divide the video to be detected into 6 frames each as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0009] Step 3, inputting the i+3th frame image into the target detection model to obtain target information;

[0010] Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0011] Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair;

[0012] Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0013] Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0014] Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

[0015] A high-altitude small target detection system based on video stream data suitable for complex environments, the system includes the following modules executed in sequence:

[0016] A model building module is used to build a neural network and train the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0017] The video segmentation module is used to segment the video to be detected, with every 6 frames as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0018] A target information acquisition module, used for inputting the i+3th frame image into the target detection model to obtain target information;

[0019] The key point extraction module is used to first construct an image pyramid and use SIFT to extract key points in the image for the i+1th frame, i+3th frame and i+5th frame, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0020] The key point matching module is used to estimate the position of the key point in the video frame in the next frame based on the optical flow and obtain the matching key point pair;

[0021] A correction module is used to calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0022] The target classification module is used to decode the target information output by the model in the target information acquisition module, obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0023] The target detection module is used to find a candidate target that matches the IOU of each output result of the correction module. If there are multiple candidate targets, the target with the highest confidence is selected as the final matching result. If there is no matching candidate target, it is discarded.

[0024] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0025] Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0026] Step 2: Divide the video to be detected into 6 frames each as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0027] Step 3, inputting the i+3th frame image into the target detection model to obtain target information;

[0028] Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0029] Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair;

[0030] Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0031] Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0032] Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0034] Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0035] Step 2: Divide the video to be detected into 6 frames each as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0036] Step 3, inputting the i+3th frame image into the target detection model to obtain target information;

[0037] Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0038] Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair;

[0039] Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0040] Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0041] Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

[0042] Compared with the prior art, the present invention has the following significant advantages: 1) The present invention obtains the transformation matrix between different video frames more accurately by calculating the optical flow, and corrects the next frame image through the matrix, so as to obtain a more accurate moving target position, thereby improving the target detection accuracy. 2) In view of the complexity of the video target detection environment, the present invention designs a lightweight neural network, which overcomes the defect of low detection accuracy of traditional methods. 3) In view of the problem of unstable video signals, the present invention uses SIFT and optical flow methods to detect dynamic targets, ensuring the detection accuracy under low video quality and complex environment.

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The figure is a flow chart of the high-altitude small target detection method based on video stream data suitable for complex environments.

[0045] Figure 2 Schematic diagram of video division.

[0046] Figure 3 This is a neural network framework diagram.

[0047] Figure 4 Figure 2 is a video detection result diagram, where (a) to (d) are the detection results of four scenes in the video. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In one embodiment, in combination Figure 1 , provides a high-altitude small target detection method based on video stream data suitable for complex environments, the method comprising the following steps:

[0050] Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0051] Step 2, Combine Figure 2 , the video to be detected is divided into 6 frames, each of which is a target detection cycle, recorded as the i-th frame to the i+5-th frame;

[0052] Step 3, inputting the i+3th frame image into the target detection model to obtain target information;

[0053] Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0054] Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair;

[0055] Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0056] Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0057] Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

[0058] Further, in one embodiment, in combination Figure 3 , the framework of the neural network described in step 1 is specifically:

[0059] Perform a 3*3 convolution on the input image to expand the number of channels to 32 dimensions, then input the feature layer into the residual network, perform residual block calculations once, twice, twice, four times, and once, downsample the input, and expand the number of channels once. The final three feature layers (52, 52, 128), (26, 26, 256), and (13, 13, 512) are used as feature outputs for detecting objects of different sizes.

[0060] The output features are then further processed. The (13, 13, 512) scale feature layer is subjected to L5 convolution regularization, and then compressed through 3*3 convolution and 1*1 convolution, so that the output format of the (13, 13, 512) scale feature layer is c*h*w*(5+cls); the result of the L5 convolution regularization processing on the (13, 13, 512) scale feature layer is upsampled, and concatenated through the (26, 26, 256) fully connected layer and the (26, 26, 256) feature layer, and then subjected to L5 convolution regularization processing, and then compressed through 3*3 convolution and 1*1 convolution, so that the output format of the (26, 26, 256) scale feature layer is c*h*w*(5+cls); The output format is c*h*w*(5+cls); the result of the above splicing and L5 convolution regularization is upsampled, and spliced ​​through a (52, 52, 256) fully connected layer and a (52, 52, 128) feature layer, and then L5 convolution regularization is performed. After that, it is compressed through 3*3 convolution and 1*1 convolution, so that the output format of the (52, 52, 128) scale feature layer is c*h*w*(5+cls); where c is the number of image channels, h is the height of the image, w is the width of the image, cls is the number of detected target categories, that is, it represents the confidence of each category, and 5 represents the target center point coordinates and length and width offsets as well as the confidence of the existence of the target.

[0061] Furthermore, in one embodiment, the calculation formula of the LK optical flow method in step 4 is:

[0062] I(x,y,z,t)=I(x+Δx,y+Δy,z+Δz,t+Δt)

[0063] Among them, x, y, z represent the position coordinates of a certain point, t represents the current time, I(x, y, z, t) represents the gray value of the point at time t, that is, the brightness, Δx, Δy, Δz represent the displacement of the point after Δt time. The above formula assumes that the brightness of the point is constant over a period of time, and Taylor expansion can be obtained as follows:

[0064]

[0065] Dividing both sides of the above equation by Δt and moving the terms, it can be simplified to the following form:

[0066] I x V x +I y V y =-I t

[0067] In the formula, I x ,I y ,I t is the partial derivative of I with respect to x, y, and t, V x and V y That is and Represents the optical flow of the point in a short time;

[0068] Using 9 pixels in a 3*3 window to establish 9 equations, it can be simplified into the following form:

[0069]

[0070] The optical flow (Vx, Vy) of this point can be calculated using the least squares method:

[0071]

[0072] Further, in one embodiment, the step 5 of obtaining the matching key point pairs specifically includes:

[0073] Calculate the Euclidean distance between the key points in the current frame and the key points in the next frame, and find matching key points based on the Euclidean distance. If the Euclidean distance is less than the set threshold, the key point pair is matched successfully.

[0074] Further, in one embodiment, the calculation formula of the transformation matrix in step 6 is:

[0075] K t=HK t+Δt

[0076] In the formula, K t and K t+Δt are the coordinates of the key points matched in the previous and next frames, and H is the transformation matrix.

[0077] Further, in one embodiment, step 8 is expressed by the formula:

[0078]

[0079] IOU(obj 1 ,O)>P,...,IOU(obj i ,O)>P}

[0080] Where, obj final indicates the final matching result, obj indicates the candidate target, conf obj Indicates the confidence of the candidate target obj, obj i represents the i-th candidate target, represents the confidence of the i-th candidate target, O is the output result of step 6, IOU(obj i ,O) indicates the calculation of obj i and the IOU value of O, P represents the set threshold, and the preferred value of P is 0.7.

[0081] In one embodiment, a high-altitude small target detection system based on video stream data suitable for complex environments is proposed, and the system includes the following modules executed in sequence:

[0082] A model building module is used to build a neural network and train the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0083] The video segmentation module is used to segment the video to be detected, with every 6 frames as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0084] A target information acquisition module, used for inputting the i+3th frame image into the target detection model to obtain target information;

[0085] The key point extraction module is used to first construct an image pyramid and use SIFT to extract key points in the image for the i+1th frame, i+3th frame and i+5th frame, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0086] The key point matching module is used to estimate the position of the key point in the video frame in the next frame based on the optical flow and obtain the matching key point pair;

[0087] A correction module is used to calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0088] The target classification module is used to decode the target information output by the model in the target information acquisition module, obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0089] The target detection module is used to find a candidate target that matches the IOU of each output result of the correction module. If there are multiple candidate targets, the target with the highest confidence is selected as the final matching result. If there is no matching candidate target, it is discarded.

[0090] For the specific definition of the high-altitude small target detection system based on video stream data suitable for complex environments, please refer to the definition of the high-altitude small target detection method based on video stream data suitable for complex environments mentioned above, which will not be repeated here. Each module in the above-mentioned high-altitude small target detection system based on video stream data suitable for complex environments can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0091] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0092] Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0093] Step 2: Divide the video to be detected into 6 frames each as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0094] Step 3, inputting the i+3th frame image into the target detection model to obtain target information;

[0095] Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0096] Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair;

[0097] Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0098] Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0099] Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

[0100] For the specific limitations of each step, please refer to the limitations of the high-altitude small target detection method based on video stream data suitable for complex environments mentioned above, which will not be repeated here.

[0101] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0102] Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model;

[0103] Step 2: Divide the video to be detected into 6 frames each as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively;

[0104] Step 3, inputting the i+3th frame image into the target detection model to obtain target information;

[0105] Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method;

[0106] Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair;

[0107] Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image;

[0108] Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target;

[0109] Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

[0110] For the specific limitations of each step, please refer to the limitations of the high-altitude small target detection method based on video stream data suitable for complex environments mentioned above, which will not be repeated here.

[0111] In one embodiment, the present invention is further verified. In this embodiment, a test platform is built using a quad-rotor drone (DJI M-100) and a laptop computer to take pictures in a forest scene to verify the detection accuracy of the invention in a complex environment. The drone DJI M-100 is equipped with an onboard computer Mainfold, a camera sensor, a GPS and other modules.

[0112] like Figure 4 As shown in (a), the person in the lower right corner who is squatting and wearing camouflage clothes is successfully detected. Figure 4 The person in the upper left corner of (b) is successfully detected. Figure 4 The person in the lower right corner of (c) is successfully detected. Figure 4 The two overlapping people at the top in (d) are detected separately, which is enough to illustrate the robustness of the detection algorithm and its adaptability to complex environments.

[0113] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A high-altitude small target detection method based on video stream data suitable for complex environments, characterized in that: The method comprises the following steps: Step 1, constructing a neural network and training the neural network using a high-altitude small target image dataset to obtain a target detection model; Step 2: Divide the video to be detected into 6 frames each as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively; Step 3, inputting the i+3th frame image into the target detection model to obtain target information; Step 4: For the i+1th frame, i+3th frame and i+5th frame images, first construct an image pyramid and use SIFT to extract key points in the image, and then estimate the optical flow of the current feature point by the LK optical flow method; Step 5: Estimate the position of the key point in the video frame in the next frame based on the optical flow, and obtain the matching key point pair; Step 6, calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image; Step 7, decode the target information output by the model in step 3 to obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target; Step 8: For each output result of step 6, find a candidate target that matches its IOU. If there are multiple candidate targets, select the target with the highest confidence as the final matching result. If there is no candidate target that matches it, discard it.

2. The high-altitude small target detection method based on video stream data suitable for complex environments according to claim 1 is characterized in that: The framework of the neural network described in step 1 is specifically: Perform a 3*3 convolution on the input image to expand the number of channels to 32 dimensions, then input the feature layer into the residual network, perform residual block calculations once, twice, twice, four times, and once, downsample the input, and expand the number of channels once. The final three feature layers (52,52,128), (26,26,256), and (13,13,512) are used as feature outputs for detecting objects of different sizes. The output features are then further processed. The (13,13,512) scale feature layer is subjected to L5 convolution regularization, and then compressed through 3*3 convolution and 1*1 convolution, so that the output format of the (13,13,512) scale feature layer is c*h*w*(5+cls); the result of the L5 convolution regularization processing on the (13,13,512) scale feature layer is upsampled, and concatenated through the (26,26,256) fully connected layer and the (26,26,256) feature layer, and then subjected to L5 convolution regularization processing, and then compressed through 3*3 convolution and 1*1 convolution, so that the output format of the (26,26,256) scale feature layer is c*h*w*(5+cls); The output format is c*h*w*(5+cls); the result of the above splicing and L5 convolution regularization is upsampled, and spliced ​​through the (52,52,256) fully connected layer and the (52,52,128) feature layer, and then L5 convolution regularization is performed, and then compressed through 3*3 convolution and 1*1 convolution, so that the output format of the (52,52,128) scale feature layer is c*h*w*(5+cls); where c is the number of image channels, h is the height of the image, w is the width of the image, cls is the number of detected target categories, that is, it represents the confidence of each category, and 5 represents the target center point coordinates and length and width offsets as well as the confidence of the existence of the target.

3. The high-altitude small target detection method based on video stream data suitable for complex environments according to claim 1 is characterized in that: The calculation formula of the LK optical flow method in step 4 is: I(x,y,z,t)=I(x+Δx,y+Δy,z+Δz,t+Δt) Among them, x, y, z represent the position coordinates of a certain point, t represents the current time, I(x, y, z, t) represents the gray value of the point at time t, that is, the brightness, Δx, Δy, Δz represent the displacement of the point after Δt time. The above formula assumes that the brightness of the point is constant over a period of time, and Taylor expansion can be obtained as follows: Dividing both sides of the above equation by Δt and moving the terms, it can be simplified to the following form: I x V x +I y V y =-I t In the formula, I x ,I y ,I t is the partial derivative of I with respect to x, y, and t, V x and V y That is and Represents the optical flow of the point in a short time; Using 9 pixels in a 3*3 window to establish 9 equations, it can be simplified into the following form: The optical flow (Vx, Vy) of this point can be calculated using the least squares method:

4. The high-altitude small target detection method based on video stream data suitable for complex environments according to claim 1 is characterized in that: Step 5 is to obtain the matching key point pairs, and the specific process includes: Calculate the Euclidean distance between the key points in the current frame and the key points in the next frame, and find matching key points based on the Euclidean distance. If the Euclidean distance is less than the set threshold, the key point pair is matched successfully.

5. The high-altitude small target detection method based on video stream data suitable for complex environments according to claim 4 is characterized in that: The calculation formula of the transformation matrix in step 6 is: K t =HK t+Δt In the formula, K t and K t+Δt are the coordinates of the key points matched in the previous and next frames, and H is the transformation matrix.

6. The high-altitude small target detection method based on video stream data suitable for complex environments according to claim 1 is characterized in that: Step 8 is expressed as: IOU(obj1,O)>P,...,IOU(obj i ,O)>P} Where, obj final Indicates the final matching result, obj indicates the candidate target, conf obj Indicates the confidence of the candidate target obj, obj i represents the i-th candidate target, represents the confidence of the i-th candidate target, O is the output result of step 6, IOU(obj i ,O) indicates the calculation of obj i and the IOU value of O, P represents the set threshold.

7. The high-altitude small target detection method based on video stream data suitable for complex environments according to claim 6 is characterized in that: The P value is 0.

7.

8. A high-altitude small target detection system based on video stream data and applicable to complex environments based on the method described in any one of claims 1 to 7, characterized in that: The system includes the following modules executed in sequence: A model building module is used to build a neural network and train the neural network using a high-altitude small target image dataset to obtain a target detection model; The video segmentation module is used to segment the video to be detected, with every 6 frames as a target detection cycle, which are recorded as the i-th frame to the i+5-th frame respectively; A target information acquisition module, used for inputting the i+3th frame image into the target detection model to obtain target information; The key point extraction module is used to first construct an image pyramid and use SIFT to extract key points in the image for the i+1th frame, i+3th frame and i+5th frame, and then estimate the optical flow of the current feature point by the LK optical flow method; The key point matching module is used to estimate the position of the key point in the video frame in the next frame based on the optical flow and obtain the matching key point pair; A correction module is used to calculate the transformation matrix according to the key points that are successfully matched, apply the transformation matrix to the next frame image for correction, and then use the frame difference method and the binary method to obtain the position of the moving target in the frame image; The target classification module is used to decode the target information output by the model in the target information acquisition module, obtain the position, category and category confidence of the target, and set two thresholds, threshold 1 and threshold 2, respectively, and threshold 1 is greater than threshold 2. The target with a confidence greater than threshold 1 is the received target, the target with a confidence less than threshold 2 is discarded, and the target with a confidence between threshold 1 and threshold 2 is taken as the candidate target; The target detection module is used to find a candidate target that matches the IOU of each output result of the correction module. If there are multiple candidate targets, the target with the highest confidence is selected as the final matching result. If there is no matching candidate target, it is discarded.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images