Pprimitive guidance-based binary hash unmanned aerial vehicle image target detection method
Through the binary hashing method based on element guidance, combined with SLIC and random walk algorithm, the GMM model of the EM algorithm is trained, and the balance of energy consumption and recognition accuracy in drone image object detection is solved, achieving fast, accurate and Low energy consumption target detection effect.
Patent Information
- Application Number
- CN202410979707.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-15
AI Technical Summary
In the detection of drone image targets, it is difficult for the prior art to find a balance between energy consumption and recognition accuracy, especially when designing identification models on low-energy platforms, a method that can quickly, accurately and consume the least energy is needed. Detect the target object.
Using the binary hashing method based on element guidance, the primitives are constructed through the SLIC superpixel algorithm and the random walk algorithm, the primitives hashing are used to use the binary MF framework, and the GMM model based on EM algorithm is trained to output the depth hashing model. Mixed with Gaussian models to process noise and extract the topology of the target object.
It realizes the target detection with high accuracy in the drone image while reducing energy consumption. It provides an anti-noise hash framework that can flexibly encode the topology of the target object and faces complex and changeable drones. The image is robust, ensuring a balance between recognition effect and energy consumption.
Smart Images

Figure CN118968353A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image target detection, and in particular relates to a binary hashing drone image target detection method based on primitive guidance. Background Art
[0002] In recent years, with the development of electronic technology and UAV technology, UAVs have been widely used in geological disaster detection, environmental monitoring, disaster relief, urban planning and other fields. UAVs have the advantages of high flexibility, relatively low cost, and the ability to perform tasks in complex environments, so they have become an important platform in the field of target detection. The development of target detection technology has benefited from breakthroughs in computer vision and artificial intelligence. The rise of deep learning technology has greatly improved the accuracy of image recognition and target detection. However, designing recognition models on low-energy platforms such as UAVs is more challenging than on traditional personal computers. Therefore, it is urgent to propose a solution to find a better balance between energy consumption and target detection effect.
[0003] Since many objects have fixed topological structures, such as aircraft, vehicles and even building surfaces, and drone aerial photos are usually acquired from constantly changing angles and heights, it is necessary to design robust features to capture target objects. Secondly, due to high-speed flight and uncontrollable external environment, drone aerial pictures usually have poor visual quality and possible wrong labels, which increases the complexity of recognition. In addition, due to limited battery capacity, it is not feasible in practice to use complex and time-consuming recognition models for drone object detection. On the contrary, if there is a super lightweight model that can accurately and quickly detect all targets from drone pictures while consuming minimal energy. Summary of the invention
[0004] In order to make up for the shortcomings of the prior art, the present invention provides a binary hashing drone image target detection method based on primitive guidance to solve the problem of balancing energy consumption and recognition accuracy in drone image target detection.
[0005] The binary hashing drone image target detection method based on primitive guidance includes a model training phase and a testing phase. The specific process is as follows:
[0006] S1. Collect labeled drone images and divide them into training set and test set;
[0007] S2. component primitives, hashing the primitives through the binary MF framework to obtain a deep hash model;
[0008] S3. According to the obtained deep hash model, input the training set, train the GMM model based on the EM algorithm, and output the trained deep hash model and GMM;
[0009] S4. Input the test set into the trained deep hashing model and GMM for testing and output the bounding boxes of C categories.
[0010] Furthermore, in step S2, the SLIC superpixel algorithm is used to decompose the drone image into multiple superpixels to obtain the topological structure of the object or part of the object; and the random walk algorithm is used to construct the primitive; that is, given a primitive of size d, it is represented by a matrix G=[G1, G2], wherein G1 is a matrix of size d×137, and these 137 dimensions are composed of a 9-dimensional color matrix associated with a 128-dimensional HOG; G2 represents a binary indicator matrix of size d×d, if the i-th and j-th object blocks are adjacent in space, then G2(i, j)=1, otherwise G2(i, j)=0; the matrix G is connected row by row to construct the feature vector.
[0011] Furthermore, in step S2, object-aware primitives with annotations are extracted for each category, each primitive corresponds to an object or a part of an object, and then a binary hash code is calculated for each primitive, and a model is trained to characterize the primitive distribution of multiple ground object categories. In the context dominated by drone images, a Gaussian mixture model is used to model the primitive distribution in drone images, that is:
[0012]
[0013] Where p(f|γ) represents the probability distribution of the graph of the drone image represented by the binary hash code, f represents the hash code of the graph subset, represents the weight of the i-th Gaussian component, p i is the probability that the graph belongs to this Gaussian component, μ i and∑ i are the corresponding binarized mean and variance respectively, γ is a model parameter, which is the sum of the binarized mean and variance; the number of Gaussian components is set to c, that is, the number of categories, and the Jaccard distance is used to measure the similarity between paired hash codes. The Jaccard distance is a distance metric used to measure the difference between two sets.
[0014] Furthermore, in step S3, the EM algorithm performs expectation steps and maximization steps alternately in an iterative manner until it converges to a predetermined threshold or reaches a maximum number of iterations; GMM is a generative model based on a probability density function, which is used to describe the probability distribution formed by a mixture of multiple Gaussian distributions.
[0015] Furthermore, the specific testing process in step S4 is as follows:
[0016] S41. Constructing primitives based on UAV images using the SLIC algorithm;
[0017] S42. Calculate the binary hash code of the primitive;
[0018] S43. Rank the posterior probabilities of all primitives and output the bounding box.
[0019] Compared with the prior art, the present invention has the following advantages: In the model training stage, the present application proposes a noise-resistant hash framework to reduce the impact of noise in drone images on target detection results. Through the calculation of the hash framework, the characteristics of the object can be retained to the greatest extent, and the topological structure of each target object can be flexibly encoded. It is robust in the face of complex and changeable drone images, ensuring a balance between accuracy and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a flow chart of the model training phase of the method of the present invention;
[0021] Figure 2 This is a flow chart of the testing phase of the method of the present invention. DETAILED DESCRIPTION
[0022] The following is a further description of the binary hashing drone image target detection method based on primitive guidance of the present invention in conjunction with the accompanying drawings.
[0023] A binary hashing drone image target detection method based on primitive guidance includes a model training phase and a testing phase.
[0024] (I) Model training stage
[0025] like Figure 1 As shown in the figure, the specific process of the model training phase is as follows:
[0026] (1) Collect labeled drone images and divide them into training set and test set.
[0027] (2) Component primitives are hashed through the binary MF framework to obtain a deep hash model.
[0028] The topology of objects is more stable than images with changing drone viewpoints / heights, which is particularly useful for drone-based object detection. Using primitives, the SLIC superpixel algorithm (SLIC superpixel algorithm is a popular image segmentation method that aims to segment images into compact regions with similar colors and spatial positions, called superpixels) is used to decompose drone images into multiple superpixels to obtain the topology of objects or parts of objects. The random walk algorithm is used to construct primitives. The core idea of the random walk algorithm is to simulate the movement process in space through random steps. The definition of primitives is as follows: visually / semantically significant parts of the image, graphics with a small number of vertices, to describe the topology of the object; the core idea of the random walk algorithm is to simulate the movement process in space through random steps.
[0029] Superpixels and their spatial interactions together determine the appearance of the primitive. Given a primitive of size d, it is represented by the matrix G = [G1, G2], where G1 is a matrix of size d×137, and these 137 dimensions consist of a 9-dimensional color matrix associated with a 128-dimensional HOG. G2 represents a binary indicator matrix of size d×d. Detailed description of G2: If the i-th and j-th object blocks are spatially adjacent, then G2(i, j) = 1, otherwise G2(i, j) = 0. The matrix G is connected row by row to construct the feature vector.
[0030] By calculating the hash code for each primitive, a model is trained to characterize the primitive distribution of multiple ground object categories. In the context dominated by drone images, a Gaussian mixture model is used to model the primitive distribution in drone images, namely:
[0031]
[0032] Where p(f|γ) represents the probability distribution of the graph of the drone image represented by the binary hash code, f represents the hash code of the graph subset, represents the weight of the i-th Gaussian component, p i is the probability that the graph belongs to this Gaussian component, μ i and∑ iare the corresponding binarized mean and variance respectively, γ is a model parameter, which is the sum of the binarized mean and variance; the number of Gaussian components is set to c, that is, the number of categories, and the Jaccard distance is used to measure the similarity between paired hash codes. The Jaccard distance is a distance metric used to measure the difference between two sets. (3) According to the obtained deep hash model, the training set is input, the GMM model based on the EM algorithm is trained, and the trained deep hash model and GMM are output. The basic idea of the EM algorithm is to alternately perform the expectation step and the maximization step in an iterative manner until it converges to a predetermined threshold or reaches the maximum number of iterations. GMM is a commonly used probability model used to describe the probability distribution formed by a mixture of multiple Gaussian distributions; it is a generative model based on the probability density function and can be used for tasks such as data clustering, density estimation, and anomaly detection.
[0033] Due to the influence of unstable factors such as fire and bad weather, the images collected by drones may be affected by noise. Therefore, this paper designs a binary MF framework that can only handle label noise. MF is a matrix decomposition technology. Matrix decomposition is a technique that decomposes a large matrix into two or more smaller matrices, which is often used for implicit feature extraction and dimensionality reduction. The most important integer property of the binary label matrix is retained. Mathematically, the objective function is:
[0034]
[0035] in, They represent image-level labels and drone images in the latent space, t represents the dimension of the latent space, n represents the number of samples, c represents the category number, The loss of matrix decomposition is quantified, Θ(·) represents the regularization term, T is the observable image label, and the binary hash code of each image subset in each drone image is calculated by the following function: h = sgn(f(x)Z), x i is the original regional feature, and f(x i )Z is used to obtain its hash code, where Z is the transformation matrix that converts the feature vector into a binary hash code.
[0036] According to the above objective function, the least squares loss is used For the regularization term, set In this way, the objective function becomes:
[0037]
[0038] where H∈{-1, 1} n×lis a matrix where each row is a binary coded vector h. The F-norm of the first and second terms of the objective function can theoretically avoid Gaussian noise in drone images. The F-norm is the Frobenius norm of the matrix. The Frobenius norm is a norm commonly used for matrices, defined as the square root of the sum of the squares of all elements in the matrix. Gaussian noise is a statistical distribution of noise that presents a Gaussian distribution. By fixing the remaining matrices, L, Q, H, and Z can be optimized separately, where L∈{-1, 1} n×c is the association matrix between drone images and labels, L ij Represents the correlation between the i-th picture and the j-th image-level label; is the image-level label, each row represents a category label, H∈{-1, 1} n×l is the binary hash code matrix after the image is hashed, and each row is a binary code vector; Z is the feature dimension conversion matrix that converts the feature vector into a binary hash code. Z is an α×l matrix, where α is the dimension of the visual feature before conversion and l is the dimension after conversion. After the conversion of Z, n visual features can be converted into an n×l matrix. The optimized structure of L, Q, H, and Z can be seen in the table below, and there is a more detailed optimization process after the table.
[0039]
[0040] Next, we solve the objective function by fixing the matrices Q, H, and Z. The optimization problem about L can be reorganized as follows:
[0041]
[0042] Due to the discrete and non-negative constraints, it is an NP-hard problem. NP-hard refers to an important concept in computational complexity theory, which refers to a class of problems that can be verified to be correct in polynomial time, but there is currently no known polynomial time algorithm that can solve them.
[0043] Here, an auxiliary variable L is introduced to obtain the closed-form solution of L:
[0044]
[0045] After that, we get the following equivalent objective function:
[0046]
[0047] Where L* = 2HQ T -1, so the above function can be further solved in closed form:
[0048]
[0049] When other matrices are fixed, the optimization problem about Q can be reformulated as:
[0050]
[0051] By taking the derivative of Q and setting the inverse to 0, we can get the following new rule:
[0052] Q=L T H(H T H+μI L ) -1
[0053] Among them I L is an identity matrix of size L×L.
[0054] When other matrices are fixed, the optimization problem about Z can be reformulated as:
[0055]
[0056] By taking the derivative of z and setting the inverse to 0, we can get the following new rule:
[0057]
[0058] Where c is a diagonal matrix with the following elements:
[0059] When other matrices are fixed, the optimization problem about H can be reformulated as:
[0060]
[0061] Obviously tr(HH T ) is a constant, so we can get the following objective function:
[0062]
[0063] By setting the derivative with respect to H to 0, we get the following Sylvester equation with discrete constraints. The Sylvester equation can be used to represent the relationship between continuous and discrete systems and can be solved in many ways.
[0064] HQ T Q=LQ+βf(X)Z
[0065] The above Sylvester equation can be solved by existing methods. The Sylvester function implemented in MATLAB is used to solve the above conditions, and then the Sylvester matrix H is obtained. * , that is, H * =sgn(H *). Sylvester matrices are often used to solve polynomial results, stability analysis, and matrix equations.
[0066] Based on the above analysis, it is not feasible to directly calculate these five matrices. By initializing the five matrices, we developed an iterative algorithm to update them. The specific operation method is: the operation of each iteration is X t =X t-1 -w×dX t-1 , where t is the number of iterations, X t-1 represents the optimization matrix in the t-1th iteration, Xt represents the optimization matrix in the tth iteration, w is a constant and is the learning rate in the iteration process, dX t-1 The convergence criterion used to obtain the derivative value of the current operation matrix is in represents the objective function value in the tth iteration. The above iterative operation is performed on matrices L, Q, H, and Z until the objective function meets the convergence criterion.
[0067] (II) Testing Phase
[0068] like Figure 2 As shown in Figure 1, the specific operations in the testing phase are: inputting the test set into the trained deep hashing model and GMM for testing, and outputting the bounding boxes of C categories.
[0069] The specific training process is as follows: construct primitives based on UAV images using the SLIC algorithm; calculate the binary hash code of the primitives; rank the posterior probabilities of all primitives and output the bounding box.
[0070] In summary, for drone images, the present invention extracts visually or semantically significant features, extracts object topological structures to construct primitives, converts primitives into binary hash codes through a binary framework that can handle label noise, and uses binary hash codes to train Gaussian mixture models; in the model testing stage, firstly, primitives are extracted from drone images, and then the binary hash codes of the primitives are calculated, and the trained Gaussian mixture model can finally output the drone image target detection classification box.
[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A binary hashing drone image target detection method based on primitive guidance, characterized in that: Including model training phase and testing phase, the specific process is as follows: S1. Collect labeled drone images and divide them into training set and test set; S2. component primitives, hashing the primitives through the binary MF framework to obtain a deep hash model; S3. According to the obtained deep hash model, input the training set, train the GMM model based on the EM algorithm, and output the trained deep hash model and GMM; S4. Input the test set into the trained deep hashing model and GMM for testing and output the bounding boxes of C categories.
2. According to the method of claim 1, the binary hashing drone image target detection method based on primitive guidance is characterized in that: In the step S2, the SLIC superpixel algorithm is used to decompose the drone image into multiple superpixels to obtain the topological structure of the object or part of the object; and the random walk algorithm is used to construct the primitive; that is, given a primitive of size d, it is represented by a matrix G=[G1, G2], where G1 is a matrix of size d×137, and these 137 dimensions are composed of a 9-dimensional color matrix associated with a 128-dimensional HOG; G2 represents a binary indicator matrix of size d×d, if the i-th and j-th object blocks are adjacent in space, then G2(i, j)=1, otherwise G2(i, j)=0; the matrix G is connected row by row to construct the feature vector.
3. The binary hashing drone image target detection method based on primitive guidance according to claim 2 is characterized in that: In step S2, object-aware primitives with annotations are extracted for each category. Each primitive corresponds to an object or a part of an object. A binary hash code is then calculated for each primitive. A model is trained to characterize the primitive distribution of multiple ground object categories. In the context of drone images, a Gaussian mixture model is used to model the primitive distribution in drone images, namely: Where p(f|Υ) represents the probability distribution of the graphics of the drone image represented by the binary hash code, f represents the hash code of the graphics subset, represents the weight of the i-th Gaussian component, p i is the probability that the graph belongs to this Gaussian component, μ i and∑ i are the corresponding binarized mean and variance respectively, Υ is a model parameter, which is the sum of the binarized mean and variance; the number of Gaussian components is set to c, that is, the number of categories, and the Jaccard distance is used to measure the similarity between paired hash codes. The Jaccard distance is a distance metric used to measure the difference between two sets.
4. The binary hashing drone image target detection method based on primitive guidance according to claim 1 is characterized in that: In step S3, the EM algorithm performs the expectation step and the maximization step alternately in an iterative manner until it converges to a predetermined threshold or reaches the maximum number of iterations; GMM is a generative model based on a probability density function, which is used to describe the probability distribution formed by a mixture of multiple Gaussian distributions.
5. The binary hashing drone image target detection method based on primitive guidance according to claim 1 is characterized in that: The specific testing process in step S4 is as follows: S41. Constructing primitives based on UAV images using the SLIC algorithm; S42. Calculate the binary hash code of the primitive; S43. Rank the posterior probabilities of all primitives and output the bounding box.
Citation Information
Patent Citations
Hash coding method based on saliency characteristic sparse embedding and extreme learning machine
CN106780639A
remote sensing image rapid target detection method based on a deep Hash auxiliary network
CN109919108A
Cardiovascular surgery data processing method and service platform based on Internet
CN116612899A
Method and apparatus for extracting hash code from image, and image retrieval method and apparatus
WO2020010691A1
