A method, device, storage medium and terminal for detecting moving targets

Abstract

The present invention provides a moving target detection method, device, storage medium and terminal, belonging to the technical field of target recognition. The method includes: performing image enhancement on a moving target image, where the moving target image is an image containing a moving target to be detected; extracting features from the enhanced moving target image to obtain target global features and target local features; constructing a current descriptor of the moving target to be detected based on the target global features and target local features; determining a matching descriptor truth value corresponding to the current descriptor from multiple descriptor truth values in a moving target database by means of feature matching; and mapping the target local features of the current descriptor to the target global feature truth value of the matching descriptor truth value to detect the moving target to be detected. The moving target detection method and device provided by the present invention improve the accuracy of moving target detection in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and particularly to a moving target detection method, device, storage medium and terminal. Background Art

[0002] Target detection refers to accurately segmenting the region of interest of humans in a given image. Such tasks have extensive and meaningful practical applications in fields such as information retrieval and public safety.

[0003] In current moving target detection, problems such as lack of sufficient and effective data and overly long model training time bring new challenges to the moving target detection task. Currently, traditional moving target detection methods mainly rely on artificial feature extraction methods, and at the same time extract the global features of the entire image to identify the category and location of the target.

[0004] However, artificial feature extraction methods are difficult to extract deep implicit features and have poor transfer and expansion capabilities. In addition, in the presence of complex perturbations, it is difficult to capture complete image features, and extracting the global features of the entire target will affect the accuracy of moving target detection. Summary of the Invention

[0005] The present invention provides a moving target detection method, device, storage medium and terminal to solve the defect of low accuracy in moving target detection in the prior art and achieve accurate detection of moving targets.

[0006] In a first aspect, the present invention provides a moving target detection method, including: performing image enhancement on a moving target image; extracting features from the enhanced moving target image to obtain target global features and target local features; constructing a current descriptor of the moving target to be detected based on the target global features and target local features; determining a matching descriptor truth value corresponding to the current descriptor from multiple descriptor truth values in a moving target database through feature matching; wherein, the descriptor truth values in the moving target database correspond to known types of moving targets, and the descriptor truth values include target global feature truth values and target local feature truth values; mapping the target local features of the current descriptor to the target global feature truth values of the matching descriptor truth value to detect the moving target to be detected.

[0007] According to the moving target detection method provided by the present invention, mapping the target local features of the current descriptor to the target global feature truth value that matches the descriptor truth value to detect the moving target to be detected, including: determining the corresponding candidate region anchor box according to the target local features of the current descriptor; and determining the corresponding global feature truth value border according to the target global feature truth value that matches the descriptor truth value; calculating the intersection over union of the candidate region anchor box and the global feature truth value border; in the case where the intersection over union is greater than a preset threshold, determining that the moving target to be detected and the moving target corresponding to the matching descriptor truth value are of the same category.

[0008] According to the moving target detection method provided by the present invention, before image enhancement of the moving target image, it further includes: collecting the original moving target image; performing preprocessing operations on the original moving target image to generate a moving target image; the preprocessing operations include one or a combination of the following operations: splitting operation, annotation operation, augmentation operation, and normalization operation.

[0009] According to the moving target detection method provided by the present invention, extracting features from the enhanced moving target image to obtain target global features and target local features, including: inputting the enhanced moving target image into a pre-trained target detection model for convolution operation to output target global features; and inputting the enhanced moving target image into a pre-trained candidate region recognition model to identify the candidate region anchor box of the moving target; based on the candidate region anchor box, determining the unoccluded local candidate region of the moving target to be detected; inputting the image corresponding to the local candidate region into a pre-trained target detection model for convolution operation to output target local features.

[0010] According to the moving target detection method provided by the present invention, the steps of generating a trained target detection model include: creating a target detection model to be trained using a convolutional neural network; collecting multiple target image training samples and inputting the target image training samples into the target detection model for training to output the loss value of the model; in the case where the loss value reaches a preset minimum value, generating a trained target detection model.

[0011] According to the moving target detection method provided by the present invention, image enhancement of the moving target image includes: inputting the moving target image into a pre-trained quality enhancement model for enhancement operation to improve the image / feature quality of the moving target image.

[0012] According to the moving target detection method provided by the present invention, the preset threshold is 0.75.

[0013] In a second aspect, the present invention further provides a moving target detection device, including:

[0014] An image enhancement module for enhancing the image of a moving target;

[0015] A feature extraction module for extracting features from the enhanced moving target image to obtain the target global feature and the target local feature;

[0016] A feature integration module for constructing the current descriptor of the moving target to be detected based on the target global feature and the target local feature;

[0017] A feature matching module for determining the matching descriptor truth value corresponding to the current descriptor from multiple descriptor truth values in the moving target database through feature matching; wherein, the descriptor truth values in the moving target database correspond to known types of moving targets, and the descriptor truth values include the target global feature truth value and the target local feature truth value;

[0018] A moving target detection module for mapping the target local feature of the current descriptor to the target global feature truth value of the matching descriptor truth value to detect the moving target to be detected.

[0019] In a third aspect, the present invention provides a terminal, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned moving target detection methods are implemented.

[0020] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned moving target detection methods are implemented.

[0021] The moving target detection method, device, storage medium, and terminal provided by the present invention integrate the target global feature and the target local feature extracted from the image of the moving target to be detected and then perform clone matching from a preset database, thereby improving the accuracy of moving target detection in a complex environment. At the same time, this set of methods can also be used for static target detection and video target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 is one of the flow diagrams of the moving target detection method provided by the present invention;

[0024] Figure 2It is the second flow schematic diagram of the moving target detection method provided by the present invention;

[0025] Figure 3 It is the structural schematic diagram of the moving target detection device provided by the present invention;

[0026] Figure 4 It is the detection result schematic diagram of the moving target provided by the present invention;

[0027] Figure 5 It is the mAP comparison result schematic diagram of different target detection algorithms provided by the present invention;

[0028] Figure 6 It is the running time comparison result schematic diagram of different target detection algorithms provided by the present invention;

[0029] Figure 7 It is the structural schematic diagram of the terminal provided by the present invention. Detailed implementation manners

[0030] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0031] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0032] The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0033] The following will describe the moving target detection method and device provided by the embodiments of the present invention in conjunction with Figures 1-7 The method can be implemented depending on a computer program and can run on a mobile target detection system based on the von Neumann architecture. The computer program can be integrated into an application or run as an independent tool application. It should be noted that the present invention is also applicable to static target image detection and video target detection.

[0034] Figure 1 is one of the flow diagrams of the moving target detection method provided by the present invention. As Figure 1 shown, it includes but is not limited to the following steps:

[0035] Step 101: Perform image enhancement on the moving target image; the moving target image is an image containing the moving target to be detected.

[0036] Among them, the moving target image to be detected can be a moving target image obtained by cameras or other image acquisition devices installed in various places such as roads, communities, airports / railway stations, and large venues. The moving target can be moving or stationary.

[0037] The moving target image in the present invention can be an ISAR (Inverse Synthetic Aperture Radar) image.

[0038] In the embodiments of this application, the moving target image is input into a pre-trained quality enhancement model for enhancement operations to improve the image / feature quality of the moving target image.

[0039] Image enhancement is an important branch in the fields of computer vision and image processing. It purposefully emphasizes the overall or local characteristics of an image, makes the original unclear image clear or emphasizes certain interesting features, enlarges the differences between the features of different objects in the image, suppresses uninteresting features, so as to improve the image quality, enrich the information content, strengthen the image interpretation and recognition effects, and meet the needs of certain special analyses. Commonly used image enhancement methods include but are not limited to:

[0040] (1) Spatial domain filters:

[0041] Smoothing filters: such as average filters, Gaussian filters, used to reduce image noise.

[0042] Sharpening filters: such as Laplacian filters, high-pass filters, used to enhance image edges.

[0043] (2) Frequency domain filters:

[0044] Low-pass filters: used to remove high-frequency noise, such as ideal low-pass filters, Butterworth low-pass filters.

[0045] High-pass filters: used to enhance the high-frequency part of the image, such as ideal high-pass filters, Butterworth high-pass filters.

[0046] (3) Histogram equalization:

[0047] By adjusting the histogram distribution of the image, the contrast of the image is enhanced.

[0048] (4) Histogram specification:

[0049] Similar to histogram equalization, but the goal is to map the histogram to a specific shape.

[0050] (5) Non-local means denoising:

[0051] Utilize the information of similar pixel blocks in the image to remove noise.

[0052] (6) Bilateral filtering:

[0053] Consider both the spatial proximity and intensity similarity of pixels, used to smooth the image while preserving edges.

[0054] (7) Deep learning methods:

[0055] Utilize convolutional neural networks (CNNs) for image quality enhancement, such as using autoencoder structures for image super-resolution. For example, build a quality enhancement model.

[0056] (8) Gamma correction:

[0057] Adjust the Gamma value of the image to improve the details of the dark and bright parts of the image.

[0058] These algorithms can be used alone or in combination to achieve better image quality enhancement effects. The choice of which algorithm usually depends on the specific problems of the image and the enhancement goals.

[0059] Based on the content of the above embodiments, as an alternative embodiment, before step 101, it further includes the step of image preprocessing: collecting the original moving target image; performing preprocessing operations on the original moving target image to generate a moving target image; the preprocessing operations include one or a combination of the following operations: splitting operation, annotation operation, augmentation operation, and normalization operation.

[0060] In a possible implementation manner, when performing moving target detection, first, the target image obtained by cameras or other image acquisition devices installed in various places such as roads, communities, airports / railway stations, and large venues is collected. After being collected by the acquisition device, it is sent to the image analysis terminal by wired or wireless means. When the image analysis terminal obtains the moving target image to be detected, it performs a splitting operation on the original data picture, splitting the wide image picture into several small pictures. On this basis, the pictures are subjected to annotation, augmentation, and normalization processing to generate the finally preprocessed moving target image to be detected.

[0061] Optionally, the relationship between the splitting, annotation, augmentation, and normalization is an AND / OR relationship; and the preprocessing operations are not limited to the preprocessing methods of splitting, annotation, augmentation, and normalization, nor is the order between them limited.

[0062] Optionally, the present invention uses the annotation tool Make Sense to perform data annotation on the image. Install Anaconda to create a virtual environment, install cuda and torch to complete the environment configuration (the versions of the two should be compatible), and verify the availability of the GPU; prepare samples according to the training set, validation set, and test set; open the annotation tool Make Sense, select all samples in the training set - object detection, add the labels to be detected for annotation, and then use relevant tools to annotate the specific position and size of the target in the picture, which is represented as a one-dimensional array containing the horizontal and vertical coordinate values of the target in the picture and the length and width values of the aircraft detection box, and save the relevant data and export the annotation data. [[ID=**11**]]

[0063] Optionally, the present invention uses the OpenCV image processing library to perform image splitting. Load the picture and determine the width and height of the sub-pictures. Calculate the width and height of the original picture to determine the number of rows and columns for splitting. Use a loop to traverse the position of each sub-picture; at each position, extract the corresponding sub-picture from the original picture; save the extracted sub-picture to the disk.

[0064] Optionally, the present invention can perform horizontal / vertical flipping on the image to create a mirror image to increase data diversity; perform rotation, scaling, or cropping on the image to enhance the model's robustness to different scales and angle changes; add random noise to the image data to make the model more capable of handling noise and changes in the real world, and use data interpolation to fill in missing values; for an imbalanced dataset, increase the samples of the minority class and reduce the samples of the majority class to improve data balance. The Thin Plate Spline (TPS) is used to insert sparse key points to generate the reference ground truth, and the TPS interpolation method is directly used to estimate the distortion function based on the reference ground truth key point annotation.

[0065] Optionally, the present invention can normalize the ISAR image using the z-score normalization method. Calculate the pixel mean μ and variance σ of the ISAR image, and for each pixel x in the ISAR, perform normalization according to Equation (1), and the normalization result is x':

[0066]

[0067] Step 102: Extract features from the enhanced moving target image to obtain the target global feature and the target local feature.

[0068] Among them, the target global feature refers to the feature that can represent the moving target after comprehensively considering the information of the entire image; the target local feature is the feature of the unoccluded local candidate region of the enhanced moving target image.

[0069] The above features can be extracted using a pre-trained object detection model.

[0070] Step 103: Based on the target global feature and the target local feature, construct the current descriptor of the moving target to be detected.

[0071] Optionally, integrate the dimensions of the target global feature and the target local feature in Step 102 to form the integrated target feature (i.e., the current descriptor), and the current descriptor refers to the descriptor of the moving target currently being detected.

[0072] Assume that the global feature dimension is X and the local feature dimension is Y, then the dimension of the current descriptor is X + Y dimensions.

[0073] Step 104: Determine the matching descriptor ground truth corresponding to the current descriptor from multiple descriptor ground truths in the moving target database through feature matching.

[0074] The moving target database is pre - constructed and contains a large number of descriptor true values. The descriptor true values in the moving target database correspond to moving targets of known types, and the descriptor true values include target global feature true values and target local feature true values.

[0075] When constructing the moving target database, first, a moving target image of a known moving target type (which can be called a real moving target image) is collected in real - time by an image acquisition device, then image / feature enhancement is performed, and then the target global feature true value and the target local feature true value of the enhanced real moving target image are extracted. Feature stitching of the target global feature true value and the target local feature true value can form the corresponding descriptor true value.

[0076] The present invention can perform feature matching according to the feature distance between the current descriptor and the descriptor true value (the feature distance can measure the similarity between descriptors) to determine the matching descriptor true value corresponding to the current descriptor.

[0077] The feature between the matching descriptor true value and the current descriptor in the present invention is closer.

[0078] Step 105: Map the target local feature of the current descriptor to the target global feature true value of the matching descriptor true value to detect the moving target to be detected.

[0079] Optionally, the present invention can use the type of the moving target corresponding to the matching descriptor true value as the final detection result.

[0080] Based on the content of the above - mentioned embodiments, as an optional embodiment, mapping the target local feature of the current descriptor to the target global feature true value of the matching descriptor true value to detect the moving target to be detected includes: determining the corresponding candidate region anchor box according to the target local feature of the current descriptor; and determining the corresponding global feature true value border according to the target global feature true value of the matching descriptor true value; calculating the intersection - over - union ratio of the candidate region anchor box and the global feature true value border; and when the intersection - over - union ratio is greater than a preset threshold, determining that the moving target to be detected and the moving target corresponding to the matching descriptor true value are of the same category.

[0081] Specifically, let the prediction box be the candidate region anchor box in the present invention and the ground - truth box be the global feature true value border, then the intersection - over - union ratio (IoU) calculation method includes at least:

[0082] (a) The original intersection - over - union ratio, which measures the overlapping degree between the prediction box and the ground - truth box in the target detection process, and the formula is as follows:

[0083]

[0084] Where A represents the prediction box and B represents the ground - truth box.

[0085] (b) Generalized Intersection over Union (GIoU). Compared with the original Intersection over Union (IoU), GIoU is insensitive to object scale changes. The formula is as follows:

[0086]

[0087] U = A ∪ B

[0088] where A C represents the area of the minimum closed region of the predicted box and the ground truth box.

[0089] (c) Distance-IoU (DIoU). By considering the distance, overlap rate, and scale between the target and the anchor point, the regression of the target box becomes more stable. The formula is as follows:

[0090]

[0091] where b represents the center point of the predicted box, b gt represents the center point of the ground truth box, ρ represents the Euclidean distance between the two center points, and C represents the diagonal distance of the minimum closed region that can contain both the predicted box and the ground truth box.

[0092] (d) Complete-IoU (CIoU). It takes into account the overlapping area, the distance between the center points, and the aspect ratio.

[0093]

[0094] a represents the weight parameter, and v represents the aspect ratio consistency metric parameter.

[0095] where the preset threshold can be 0.75.

[0096] Optionally, when the Intersection over Union is less than the preset threshold, it is determined that the detected moving target and the moving target corresponding to the true value of the matching descriptor are not of the same category. Further, a new different true value of the matching descriptor can be re-matched in the moving target database to re-perform moving target detection until the newly calculated Intersection over Union is less than the preset threshold.

[0097] Based on the content of the above embodiments, as an alternative embodiment, Figure 2 is the second flow diagram of the moving target detection method provided by the present invention. The following describes the implementation process of the present invention for extracting features from the enhanced moving target image to obtain the target global features and target local features in combination with Figure 2 illustrates the implementation process of the present invention for extracting features from the enhanced moving target image to obtain the target global features and target local features.

[0098] Optionally, the present invention inputs the enhanced moving target image into a pre-trained target detection model for convolution operation to output the target global feature. The present invention inputs the enhanced moving target image into a pre-trained candidate region recognition model to identify the candidate region anchor boxes of the moving target; based on the candidate region anchor boxes, determines the unoccluded local candidate regions of the moving target to be detected; inputs the image corresponding to the local candidate regions into a pre-trained target detection model for convolution operation to output the target local feature.

[0099] As an optional embodiment, the steps of the present invention for generating a trained target detection model include: creating a target detection model to be trained using a convolutional neural network; collecting a plurality of target image training samples, and inputting the target image training samples into the target detection model for training to output the loss value of the model; generating a trained target detection model when the loss value reaches a preset minimum value.

[0100] As another possible implementation manner, the detection of the local candidate regions in the present invention includes but is not limited to the following methods:

[0101] (1) Method based on matrix transformation: First, input the enhanced moving target image into a preset spatial transformation network to generate affine transformation matrix parameters, and then perform an affine transformation on the moving target image based on the affine transformation matrix parameters to locate the local candidate regions.

[0102] (2) Method based on visual saliency: This method can determine the position of the local candidate regions through the image energy distribution;

[0103] (3) Localization of specific regions: For example, the localization of the wing and the front of the vehicle head, the local candidate regions of the target can be obtained through prior knowledge or a trained model.

[0104] (4) Method based on feature points: Use existing feature point detection algorithms to obtain feature points, and find the region where the feature points are relatively concentrated as the local candidate regions of the target.

[0105] (5) Method based on key point detection: Use a preset target key point detection model to detect the positions of multiple key points of the target, and locate the local candidate regions of the target according to the key points.

[0106] In actual application, the corresponding local candidate region detection method can be selected according to actual needs.

[0107] Further, after detecting the local candidate regions in the moving target image to be detected according to the above detection method, local features are extracted from the detected local candidate regions. This extraction method is the same as the global feature extraction method and will not be elaborated here.

[0108] Among them, regarding the generation and optimization of local candidate regions, a fully convolutional network can be used to construct a candidate region mechanism to generate local candidate regions, and at the same time, rough classification and regression of candidate regions are performed to obtain object confidence and bounding box position information.

[0109] To generate local candidate regions, a small sliding network is constructed on the feature map output by the last shared convolutional layer of the convolutional neural network. This network takes the spatial window of the input feature map as input, and each sliding window is mapped to a low-dimensional feature.

[0110] Then this feature is fed into the regression layer and the classification layer for regression and classification. Both the regression layer and the classification layer are fully connected layers of size 1x1. "Seeds" boxes are designed in the candidate region mechanism to effectively predict candidate regions of multiple scales and aspect ratios. "Pyramid seeds" are constructed based on the seeds boxes of multiple scales and aspect ratios, and classification and regression are performed on the corresponding bounding boxes / enclosing boxes of the seeds boxes.

[0111] Only operate on single-scale images and feature maps, and use single-size filters (sliding windows on the feature map). To adapt to different scales, k bounding box regressors are learned. Each regressor corresponds to a scale and aspect ratio, and these k bounding box regressors do not share weights.

[0112] Due to the design of the seeds, although the feature is of a fixed scale, it is still possible to predict bounding boxes of different scales. The generation of local candidate regions is achieved by minimizing the objective function:

[0113]

[0114] where i is the seed index, p i is the probability that the i-th seed has an object, is the Ground-truth label of the i-th seed. If the seed is positive, then otherwise t i represents the 4 coordinate points of the predicted bounding box, represents the true bounding box of the positive seed. L cls represents the classification (object and non-object) log loss, and L reg represents the regression log loss. The outputs of the classification layer and the regression layer respectively include {p i} and {t i}.

[0115]

[0116]

[0117]

[0118] Among them, R is the loss function. N cls and N reg respectively represent the size of the randomly selected mini-batch data (N cls = 256) and the number of seed positions (N cls = 2304). N cls and N reg and the balance factor λ are respectively used to normalize the classification loss L cls and the regression loss L reg .

[0119] Meanwhile, a candidate region classification and sorting strategy and a region optimization mechanism are adopted to further improve the candidate region quality and algorithm efficiency.

[0120] After obtaining the weakly labeled images and unlabeled images in the training set, a labeling value prediction function is learned according to the convolutional features of the images obtained by the convolutional neural network. The training of the weakly labeled images can be optimized by the weakly weighted binary sorting loss, and the training of the unlabeled images can be optimized by the ternary similarity sorting loss. The shared-exclusive mechanism can be adopted to enhance the object detection performance.

[0121] To more clearly illustrate the technical solution of the moving target detection of the present invention, based on the content of the above embodiments, a specific embodiment is further combined below to illustrate the present invention.

[0122] The method of the embodiment of the present application may include the following steps:

[0123] Step 1: Collect the original moving target image and perform a splitting operation on the original moving target image;

[0124] Step 2: Split the wide original moving target image into several small pictures, and perform annotation, augmentation, and normalization processing on the pictures. The image after splitting, annotation, augmentation, and normalization is determined as the preprocessed moving target image;

[0125] Step 3: Input the moving target image into a pre-trained quality enhancement model for enhancement operation to improve the image / feature quality.

[0126] Step 4: Input the enhanced target image to be detected into a pre-trained object detection model for convolution operation to generate the target global feature.

[0127] Step 5: Input the enhanced moving target image into the target anchor / box monitoring model (i.e., the candidate region recognition model) to generate multiple anchor / box positions of the target (i.e., the candidate region anchor boxes).

[0128] Step 6: Locate the local candidate regions of the moving target to be detected based on the multiple anchor / box positions of the target;

[0129] Step 7: Input the local candidate region of the moving target to be detected into a pre-trained object detection model for convolution operation to generate the target local features.

[0130] Step 8: After integrating the target global features and the target local features, generate the integrated target features (i.e., the current descriptor).

[0131] Step 9: Match the second target features (i.e., the matching descriptor ground truth) corresponding to the integrated target features from the moving target database.

[0132] Step 10: Determine the position of the moving target according to the candidate region anchor box, and calculate the intersection over union (IoU) between the candidate region anchor box and the global feature ground truth border. When the IoU is greater than a preset threshold,

[0133] determine that the moving target to be detected and the moving target corresponding to the matching descriptor ground truth are of the same category; and when the IoU is less than the preset threshold, the moving target to be detected and the moving target corresponding to the matching descriptor ground truth are not of the same category.

[0134] Figure 3 is a schematic structural diagram of the moving target detection device provided by the present invention, as Figure 3 shown, the device includes: an image enhancement module 301, a feature extraction module 302, a feature integration module 303, a feature matching module 304, and a moving target detection module 305.

[0135] The image enhancement module 301 is configured to perform image enhancement on the moving target image;

[0136] The feature extraction module 302 is configured to extract features from the enhanced moving target image to obtain the target global features and the target local features;

[0137] The feature integration module 303 is configured to construct the current descriptor of the moving target to be detected based on the target global features and the target local features;

[0138] The feature matching module 304 is configured to determine the matching descriptor ground truth corresponding to the current descriptor from multiple descriptor ground truths in the moving target database by means of feature matching; wherein, the descriptor ground truths in the moving target database correspond to known types of moving targets, and the descriptor ground truths include the target global feature ground truth and the target local feature ground truth;

[0139] The moving target detection module 305 is configured to map the target local features of the current descriptor to the target global feature ground truth of the matching descriptor ground truth for detecting the moving target to be detected.

[0140] It should be noted that the moving target detection device provided in the embodiments of the present invention can execute the moving target detection method described in any of the above embodiments during specific operation, and this embodiment will not be elaborated herein.

[0141] In order to verify the technical advantages of the moving target detection method and device provided by the present invention, further analysis and description will be carried out below in combination with the embodiments of experimental simulation.

[0142] 1. Embodiment conditions

[0143] This embodiment runs in the Ubuntu 18.04 operating system, uses the Matlab / Python language, and conducts experiments with a PC Intel i7-8750H CPU (2.20GHz) and an NVIDIA GeForce RTX 2060 GPU.

[0144] 2. Embodiment content

[0145] In the embodiment, two datasets are used for experiments, namely Dataset-1 and Dataset-2. Among them, Dataset-1 contains 5 types of targets, and each image contains 10 key point annotations. Dataset-2 contains 8 types of targets, and each image contains 12 key point annotations. The moving targets include moving targets such as airplanes, cars, and ships.

[0146] Figure 4 It is a schematic diagram of the detection result of the moving target provided by the present invention. The dataset used by the present invention is relatively complex, with different perspectives, scales, postures, and there are different degrees of noise, deformation, and low resolution, and the geometric features are not obvious and continuous. It can be seen from the figure that the detection result of the method of the present invention has a good response.

[0147] Figure 5 It is a schematic diagram of the mAP comparison result of different target detection algorithms provided by the present invention. Figure 6 It is a schematic diagram of the comparison result of the running time of different target detection algorithms provided by the present invention.

[0148] Among them, the comparison methods adopted are OverFeat, RCNN, Fast RCNN, Faster RCNN, SPP, YOLO, R-FCN, and Mask RCNN. The larger the mAP value, the better the accuracy of the method, and the shorter the running time, the faster the method runs. It can be seen from the figure that the accuracy and running time of the method of the present invention are better than those of other methods. It shows that this method has good detection performance.

[0149] Figure 7 It is a schematic diagram of the structure of the terminal provided by the present invention, as Figure 7As shown in the figure, the terminal may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logical instructions in the memory 730 to execute a moving target detection method, which includes: performing image enhancement on a moving target image; the moving target image is an image containing the moving target to be detected; extracting features from the enhanced moving target image to obtain a target global feature and a target local feature; constructing a current descriptor of the moving target to be detected based on the target global feature and the target local feature; determining a matching descriptor truth value corresponding to the current descriptor from multiple descriptor truth values in a moving target database through feature matching; where the descriptor truth values in the moving target database correspond to known types of moving targets, and the descriptor truth values include a target global feature truth value and a target local feature truth value; mapping the target local feature of the current descriptor to the target global feature truth value of the matching descriptor truth value to detect the moving target to be detected.

[0150] In addition, when the logical instructions in the above-mentioned memory 730 are implemented in the form of software functional units and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0151] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the moving target detection method provided in each of the above embodiments. The method includes: performing image enhancement on a moving target image, where the moving target image is an image containing a moving target to be detected; extracting features from the enhanced moving target image to obtain a target global feature and a target local feature; constructing a current descriptor of the moving target to be detected based on the target global feature and the target local feature; determining a matching descriptor truth value corresponding to the current descriptor from multiple descriptor truth values in a moving target database through feature matching, where the descriptor truth values in the moving target database correspond to moving targets of known types, and the descriptor truth values include a target global feature truth value and a target local feature truth value; mapping the target local feature of the current descriptor to the target global feature truth value of the matching descriptor truth value to detect the moving target to be detected.

[0152] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the moving target detection method provided in each of the above embodiments. The method includes: performing image enhancement on a moving target image, where the moving target image is an image containing a moving target to be detected; extracting features from the enhanced moving target image to obtain a target global feature and a target local feature; constructing a current descriptor of the moving target to be detected based on the target global feature and the target local feature; determining a matching descriptor truth value corresponding to the current descriptor from multiple descriptor truth values in a moving target database through feature matching, where the descriptor truth values in the moving target database correspond to moving targets of known types, and the descriptor truth values include a target global feature truth value and a target local feature truth value; mapping the target local feature of the current descriptor to the target global feature truth value of the matching descriptor truth value to detect the moving target to be detected.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0154] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A moving target detection method, characterized in that, Comprising: Performing image enhancement on the moving target image; The moving target image is an image containing the moving target to be detected; Performing feature extraction on the enhanced moving target image to obtain the target global feature and the target local feature; Based on the target global feature and the target local feature, constructing the current descriptor of the moving target to be detected; By means of feature matching, determining the matching descriptor ground truth corresponding to the current descriptor from multiple descriptor ground truths in the moving target database; wherein, the descriptor ground truth in the moving target database corresponds to a known type of moving target, and the descriptor ground truth includes the target global feature ground truth and the target local feature ground truth; Mapping the target local feature of the current descriptor to the target global feature ground truth of the matching descriptor ground truth for detecting the moving target to be detected; Performing feature extraction on the enhanced moving target image to obtain the target global feature and the target local feature, including: Inputting the enhanced moving target image into a pre-trained target detection model for convolution operation to output the target global feature; and, Inputting the enhanced moving target image into a pre-trained candidate region recognition model to recognize the candidate region anchor boxes of the moving target; Based on the candidate region anchor boxes, determining the unoccluded local candidate regions of the moving target to be detected; Inputting the image corresponding to the local candidate region into a pre-trained target detection model for convolution operation to output the target local feature; Mapping the target local feature of the current descriptor to the target global feature ground truth of the matching descriptor ground truth for detecting the moving target to be detected, including: Determining the corresponding candidate region anchor box according to the target local feature of the current descriptor; and determining the corresponding global feature ground truth border according to the target global feature ground truth of the matching descriptor ground truth; Calculating the intersection over union of the candidate region anchor box and the global feature ground truth border; When the intersection over union is greater than the preset threshold, determining that the moving target to be detected and the moving target corresponding to the matching descriptor ground truth are of the same category.

2. The motion target detection method according to claim 1, wherein Before performing image enhancement on the moving target image, it further includes: Collecting the original moving target image; Performing preprocessing operations on the original moving target image to generate the moving target image; The preprocessing operations include one or a combination of the following operations: splitting operation, annotation operation, augmentation operation, and normalization operation.

3. The motion target detection method according to claim 1, wherein The steps for generating the trained target detection model include: Using a convolutional neural network to create the target detection model to be trained; Collecting multiple target image training samples and inputting the target image training samples into the target detection model for training to output the loss value of the model; When the loss value reaches the preset minimum value, generating the trained target detection model.

4. The motion target detection method according to claim 1, wherein Performing image enhancement on the moving target image, including: Inputting the moving target image into a pre-trained quality enhancement model for enhancement operation to improve the image / feature quality of the moving target image.

5. The motion target detection method according to claim 1, wherein, The preset threshold is 0.

75.

6. A moving target detection device, characterized in that, Comprising: An image enhancement module for performing image enhancement on the moving target image; A feature extraction module, configured to extract features from the enhanced moving target image to obtain the target global feature and the target local feature; The feature extraction module is specifically configured to input the enhanced moving target image into a pre-trained target detection model for convolution operation to output the target global feature; And, Input the enhanced moving target image into a pre-trained candidate region recognition model to identify the candidate region anchor boxes of the moving target; Based on the candidate region anchor boxes, determine the unoccluded local candidate regions of the moving target to be detected; Input the image corresponding to the local candidate region into a pre-trained target detection model for convolution operation to output the target local feature; A feature integration module, configured to construct the current descriptor of the moving target to be detected based on the target global feature and the target local feature; A feature matching module, configured to determine the matching descriptor ground truth corresponding to the current descriptor from multiple descriptor ground truths in the moving target database through feature matching; wherein, the descriptor ground truths in the moving target database correspond to known types of moving targets, and the descriptor ground truths include the target global feature ground truth and the target local feature ground truth; A moving target detection module, configured to map the target local feature of the current descriptor to the target global feature ground truth of the matching descriptor ground truth to detect the moving target to be detected; The moving target detection module is specifically configured to: Determine the corresponding candidate region anchor box according to the target local feature of the current descriptor; and determine the corresponding global feature ground truth border according to the target global feature ground truth of the matching descriptor ground truth; Calculate the intersection over union of the candidate region anchor box and the global feature ground truth border; When the intersection over union is greater than a preset threshold, determine that the moving target to be detected and the moving target corresponding to the matching descriptor ground truth are of the same category.

7. A terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the moving target detection method according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the moving target detection method according to any one of claims 1 to 5 are implemented.

Images