Unmanned aerial vehicle target detection method based on morphological adaptive network
By adopting a target detection method based on a morphological adaptive network in the inspection of drone oil and gas pipelines, the problems of slow target detection speed and reduced accuracy in the prior art are solved, and accurate detection and efficient inspection of oil and gas pipeline safety risk targets are achieved.
Patent Information
- Application Number
- CN202411758172.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-03-25
AI Technical Summary
In the existing drone oil and gas pipeline inspection, the inference speed of deep learning-based object detection methods is slow, and the problems of gradient diffusion and accuracy are prone to occur. The multiple changes in the safety risk targets of oil and gas pipelines increase the difficulty of inspection.
UAV target detection method based on morphological adaptive network is adopted, including improving the depth confidence network, deep feature transformation module, Q-learning search mechanism and deep forest classifier, and the training samples are expanded through selective search algorithm, occlusion and rotation samples are added, and the model is optimized to improve robustness.
Accurate detection of oil and gas pipeline safety risk targets has been achieved, detection efficiency and accuracy have been improved, and the target can be effectively adapted to the occlusion and rotation changes, reducing the risk of gradient dispersion and accuracy reduction.
Smart Images
Figure CN119693825A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of oil and gas pipeline safety monitoring, and in particular to a UAV target detection method based on a morphological adaptive network. Background Art
[0002] The oil and gas pipeline management industry is an important link in the energy industry chain, responsible for the long-distance, large-scale and efficient transportation of oil and gas resources. With the continuous growth of global energy demand and the continuous optimization of energy structure, the oil and gas pipeline management industry has ushered in unprecedented development opportunities.
[0003] At present, drones have been widely used due to their advantages such as small size, light weight and simple operation. Combining deep learning methods with drone systems can help to quickly and accurately detect the required targets on drone aerial images with high clarity and wide field of view. However, although target detection based on deep learning overcomes the shortcomings of traditional methods in terms of accuracy, its reasoning speed is slow, and gradient diffusion and accuracy reduction will occur. At the same time, the multi-morphological changes of oil and gas pipeline safety risk targets further increase the difficulty of inspection. Therefore, how to effectively mine the deep features of oil and gas pipeline safety risk targets and optimize the detection model to achieve accurate oil and gas inspection target recognition is the basis for studying safety risk target detection in drone oil and gas pipeline inspection. To this end, this scheme proposes a drone target detection method based on morphological adaptive network. Summary of the invention
[0004] In order to solve the technical problems raised in the background technology, the present invention provides a UAV target detection method based on a morphological adaptive network.
[0005] The present invention is implemented by the following technical scheme: a method for detecting unmanned aerial vehicle targets based on a morphological adaptive network, comprising:
[0006] Obtain image data taken by drones;
[0007] Constructing morphologically adaptive network models;
[0008] Inputting the acquired real-time image data into the morphological adaptive network model and outputting the target detection result;
[0009] The morphological adaptive network model includes: an improved deep belief network, a deep feature transformation module, a Q learning search mechanism and a deep forest classifier; and the model training steps are as follows:
[0010] First, the selective search algorithm is used to expand the training samples to obtain target occlusion samples and target rotation samples;
[0011] The improved deep belief network is used to extract the deep features of image data. The deep features are rotation invariant and robust to occlusion changes.
[0012] Then the deep feature transformation algorithm is used to obtain the target pre-positioning position;
[0013] Optimize the target box position based on the search mechanism in the Q-learning search module;
[0014] Use deep forest to obtain target classification information and finally get accurate target detection results.
[0015] As a further improvement of the above solution, the improved deep belief network is composed of multiple convolutional restricted Boltzmann machines stacked in an integrated learning manner;
[0016] Among them: the output of the previous restricted Boltzmann machine is used as the input of the next restricted Boltzmann machine; during the training process, the first restricted Boltzmann machine is first trained according to the loss function, and then the output of the first restricted Boltzmann machine is used as the input of the second restricted Boltzmann machine, and all restricted Boltzmann machines are trained in sequence. The output of the last restricted Boltzmann machine is used as the deep feature of the improved deep belief network output, corresponding to a block area in the original image.
[0017] As a further improvement of the above scheme, a rotation invariant constraint term is designed in the improved deep belief network to make the convolutional restricted Boltzmann machine model rotation invariant:
[0018]
[0019] Where: O i (p i ) is the rotation invariant feature output by the training sample pi in the i-th hidden layer node; is the training sample p i After multiple rotation changes, the average rotation invariant feature output in the i-th hidden layer node is calculated as:
[0020]
[0021] By adding the rotation invariant regularization constraint, the target data with different rotation angles can be similarly represented after being reconstructed by the multi-layer restricted Boltzmann machine.
[0022] The loss function of the improved convolutional deep belief network model consists of the original convolutional restricted Boltzmann machine loss function term, the rotation invariant constraint term and the weight attenuation term.
[0023] As a further improvement of the above scheme, after the convolutional deep belief network is trained, for the first L frames of the video in the training sample, the deep features of each frame image in the video can be obtained. First, the average value of the deep features is calculated, and then the correlation matrix of the deep features is calculated. After obtaining the correlation matrix, the eigenvector of the correlation matrix can be calculated, and then the projection of the deep features on the eigenvector is calculated. The larger the projection calculation value, the stronger the correlation between the feature and the target. By connecting the largest number of positive areas, the pre-positioning area of the target can be obtained.
[0024] As a further improvement of the above scheme, in the Q learning search mechanism, the action sequence of its model corresponds to 7 kinds of search box scale position changes, namely upper edge movement, lower edge movement, left edge movement, right edge movement, overall zoom-in, overall zoom-out and stop;
[0025] The current state is defined as a triple consisting of the depth feature of the original image, the depth feature of the pre-positioned area, and the historical action;
[0026] Specifically, the current frame is input into the improved deep belief network to obtain the current frame depth features and multi-dimensional feature vectors; similarly, the deep belief network model is used to extract the depth features of the pre-positioned frame.
[0027] As a further improvement of the above solution, the deep forest classifier is used to provide category information of the detection target;
[0028] The deep forest structure is composed of multiple layers of cascaded random forests. Each layer of cascaded forests includes two completely random forests and two random forests. Each forest contains 500 random trees. The completely random trees use all samples as input. Each node of the decision tree randomly selects a feature for splitting until the instances contained in each leaf node belong to the same class, or the number of instances is no more than 10. This method increases the diversity of samples. The training process of the random forest is to input a random subset of samples to each tree, and the splitting node selects a class with the optimal Gini coefficient for splitting. This method ensures the effect of the algorithm.
[0029] The deep features of the pre-positioned area are input into the first layer of the cascade forest of the deep forest. For the Z classification problem, this layer of the cascade forest outputs a Z-dimensional class feature vector and aggregates it with the deep features to obtain a new feature vector, which is input into the next layer of the cascade forest. The process is iterated in sequence, and the category with the maximum value of the class feature vector output by the last layer of the cascade forest is taken as the category information of the detected target.
[0030] Compared with the prior art, the present invention has the following beneficial effects:
[0031] The deep belief network proposed in the present invention is composed of multiple convolutional restricted Boltzmann machines stacked together. In essence, it is a multi-layer neural network structure in an unsupervised mode. The log-likelihood maximization algorithm is used to continuously adjust the weights between the layers of the model, so that the model can fit the joint probability distribution between the visible layer and the hidden layer data, thereby learning the deep features in the input data, which have strong representation capabilities. It is applied in this solution. In view of the problems that targets in drone image videos are susceptible to occlusion, deformation, and complex background interference, an improved deep belief network model is proposed. By expanding the original sample data set and adding occluded samples and rotated samples for training, the model has better robustness to occlusion and rotation problems.
[0032] The present invention uses a selective search algorithm to amplify the original training samples during the training phase. By adding training samples of occlusion situations, the model can adapt to the occlusion changes of the target.
[0033] The morphologically adaptive target detection method proposed in the present invention can effectively mine the deep features of oil and gas pipeline safety risk targets to achieve accurate oil and gas inspection target identification, and provide practical technical support for UAV oil and gas pipeline inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 The following is a flowchart of the target detection method proposed by the present invention;
[0035] Figure 2 A working principle diagram of the adaptive network model proposed in the present invention;
[0036] Figure 3 This is a principle block diagram of the Q learning search mechanism proposed by the present invention;
[0037] Figure 4 Schematic diagram of each action of the Q learning search mechanism proposed by the present invention;
[0038] Figure 5 Structural diagram of the proposed degree forest classifier for the invention. DETAILED DESCRIPTION
[0039] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.
[0040] Example:
[0041] Please combine Figure 1-Figure 5 ,This scheme proposes a UAV target detection method based on ,morphological adaptive network, including
[0042] Obtain image data taken by drones;
[0043] Constructing morphologically adaptive network models;
[0044] Inputting the acquired real-time image data into the morphological adaptive network model and outputting the target detection result;
[0045] The morphological adaptive network model includes: an improved deep belief network, a deep feature transformation module, a Q learning search mechanism and a deep forest classifier; and the model training steps are as follows:
[0046] First, the selective search algorithm is used to expand the training samples to obtain target occlusion samples and target rotation samples;
[0047] The improved deep belief network is used to extract the deep features of image data. The deep features are rotation invariant and robust to occlusion changes.
[0048] Then the deep feature transformation algorithm is used to obtain the target pre-positioning position;
[0049] Optimize the target box position based on the search mechanism in the Q-learning search module;
[0050] Use deep forest to obtain target classification information and finally get accurate target detection results.
[0051] In this scheme, the improved deep belief network is composed of multiple convolutional restricted Boltzmann machines stacked in an integrated learning manner. It is an unsupervised multi-layer neural network structure that aims to use the log-likelihood maximization algorithm to continuously adjust the weights between the layers of the model so that the model can fit the joint probability distribution between the visible layer and the hidden layer data, thereby learning the deep features in the input data, which have strong representation capabilities;
[0052] That is, the output of the previous restricted Boltzmann machine is used as the input of the next restricted Boltzmann machine; during the training process, the first restricted Boltzmann machine is first trained according to the loss function, and then the output of the first restricted Boltzmann machine is used as the input of the second restricted Boltzmann machine. All restricted Boltzmann machines are trained in sequence, and the output of the last restricted Boltzmann machine is used as the deep feature output by the deep belief network, corresponding to a block area in the original image.
[0053] In this scheme, a rotation invariant constraint is designed in the improved deep belief network to make the convolutional restricted Boltzmann machine model rotation invariant:
[0054]
[0055] Where: O i (p i) is the rotation invariant feature output by the training sample pi in the i-th hidden layer node; is the training sample p i After multiple rotation changes, the average rotation invariant feature output in the i-th hidden layer node is calculated as:
[0056]
[0057] By adding the rotation invariant regularization constraint, the target data with different rotation angles can be similarly represented after being reconstructed by the multi-layer restricted Boltzmann machine.
[0058] The loss function of the improved convolutional deep belief network model consists of the original convolutional restricted Boltzmann machine loss function term, the rotation invariant constraint term and the weight attenuation term.
[0059] In this scheme, after the training of the convolutional deep belief network is completed, for the first L frames of the video, the deep features of each frame image in the video can be obtained. First, the average value of the deep features is calculated, and then the correlation matrix of the deep features is calculated. After obtaining the correlation matrix, the eigenvector of the correlation matrix can be calculated, and then the projection of the deep features on the eigenvector is further calculated. The larger the projection calculation value, the stronger the correlation between the feature and the target. By connecting the largest number of positive areas, the pre-positioning area of the target can be obtained.
[0060] In order to further refine the position of the target detection positioning box, an active target positioning method based on the Q-learning algorithm is adopted. During the training process, the dynamic attention adjustment method is used. The pre-positioned target box is continuously adjusted to be close to the true label, and the corresponding search mechanism of target detection is trained. During the detection process, the pre-positioning box is input into the trained search mechanism to obtain the precise positioning of the target to be detected.
[0061] In this scheme, in the Q learning search module, the action sequence of its model is as follows Figure 4 As shown, there are 7 kinds of search box scale position changes, namely upper edge movement, lower edge movement, left edge movement, right edge movement, overall zoom-in, overall zoom-out and stop;
[0062] The current state is defined as a triplet consisting of the depth features of the original image, the depth features of the pre-positioned area, and the historical action. Specifically, the current frame is input into the improved deep belief network to obtain the current frame depth features and multi-dimensional feature vectors; similarly, the deep belief network model is used to extract the depth features of the pre-positioned frame;
[0063] The predicted box position is represented by the image coordinates of the top-left and bottom-right corners of the box. Figure 3 The transformation action is completed by transforming the predicted box position coordinate representation. The target search mechanism based on the Q learning algorithm is as follows Figure 3As shown in the figure, d represents the dimension. Given the state of the image to be detected, the action with the largest expected benefit is selected from the given action set according to the reward value, so that the pre-positioning frame continuously approaches the real position of the target, and the optimal solution is obtained as the target detection result output.
[0064] In this scheme, in order to enable the proposed method to identify the category information of the detection target, a classifier needs to be added to the top layer, and the deep forest model is used as the classifier. Compared with traditional classifiers (such as Softmax models), deep forests have the characteristics of strong robustness, high accuracy, and few hyperparameters. They can obtain intra-class similarities and inter-class differences from complex features, thereby providing accurate classification detection information of oil and gas pipeline safety risk targets;
[0065] The deep forest structure is as follows Figure 5 As shown in the figure, it consists of multiple layers of cascaded random forests. Each layer of cascaded forests includes two completely random forests (green) and two random forests (blue). Each forest contains 500 random trees. The completely random trees use all samples as input. Each node of the decision tree randomly selects a feature for splitting until the instances contained in each leaf node belong to the same class, or the number of instances is no more than 10. This method increases the diversity of samples. The training process of the random forest is to input a random subset of samples to each tree, and the splitting node selects a class with the optimal Gini coefficient for splitting. This method ensures the effect of the algorithm. The deep features of the pre-positioned area are input into the first layer of the cascade forest of the deep forest. For the Z classification problem, this layer of cascade forest outputs the Z-dimensional class feature vector and aggregates it with the deep features to obtain a new feature vector, which is input into the next layer of cascade forest. Iterates in sequence, and takes the category where the maximum value of the class feature vector output by the last layer of cascade forest is the category information of the detected target.
[0066] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.
Claims
1. A UAV target detection method based on morphological adaptive network, characterized in that: Including obtaining image data taken by drones; Constructing morphologically adaptive network models; Inputting the acquired real-time image data into the morphological adaptive network model and outputting the target detection result; The morphological adaptive network model includes: an improved deep belief network, a deep feature transformation module, a Q learning search mechanism and a deep forest classifier; and the model training steps are as follows: First, the selective search algorithm is used to expand the training samples to obtain target occlusion samples and target rotation samples; The improved deep belief network is used to extract the deep features of image data. The deep features are rotation invariant and robust to occlusion changes. Then the deep feature transformation algorithm is used to obtain the target pre-positioning position; Optimize the target box position based on the search mechanism in the Q-learning search module; Use deep forest to obtain target classification information and finally get accurate target detection results.
2. The method for detecting unmanned aerial vehicle targets based on a morphological adaptive network as claimed in claim 1, characterized in that: The improved deep belief network is composed of multiple convolutional restricted Boltzmann machines stacked in an integrated learning manner; Among them: the output of the previous restricted Boltzmann machine is used as the input of the next restricted Boltzmann machine; during the training process, the first restricted Boltzmann machine is first trained according to the loss function, and then the output of the first restricted Boltzmann machine is used as the input of the second restricted Boltzmann machine, and all restricted Boltzmann machines are trained in sequence. The output of the last restricted Boltzmann machine is used as the deep feature of the improved deep belief network output, corresponding to a block area in the original image.
3. The method for detecting unmanned aerial vehicle targets based on a morphological adaptive network as claimed in claim 1, characterized in that: In the improved deep belief network, a rotation invariant constraint is designed to make the convolutional restricted Boltzmann machine model rotation invariant: Where: O i (p i ) is the rotation invariant feature output by the training sample pi in the i-th hidden layer node; is the training sample p i After multiple rotation changes, the average rotation invariant feature output in the i-th hidden layer node is calculated as: By adding rotation invariant regularization constraints, target data with different rotation angles can be similarly represented after being reconstructed by multi-layer restricted Boltzmann machines. The loss function of the improved convolutional deep belief network model consists of the original convolutional restricted Boltzmann machine loss function term, the rotation invariant constraint term and the weight attenuation term.
4. The method according to claim 1, wherein: After training the convolutional deep belief network, for the first L frames of the video in the training sample, the deep features of each frame in the video can be obtained. First, the average value of the deep features is calculated, and then the correlation matrix of the deep features is calculated. After obtaining the correlation matrix, the eigenvector of the correlation matrix can be calculated, and then the projection of the deep features on the eigenvector is calculated. The larger the projection calculation value, the stronger the correlation between the feature and the target. The pre-positioning area of the target can be obtained by connecting the largest number of positive areas.
5. The method according to claim 1, wherein: In the Q-learning search mechanism, the action sequence of the model corresponds to seven kinds of search box scale position changes, namely, upper edge movement, lower edge movement, left edge movement, right edge movement, overall zoom-in, overall zoom-out and stop; The current state is defined as a triple consisting of the depth feature of the original image, the depth feature of the pre-positioned area, and the historical action; Specifically, the current frame is input into the improved deep belief network to obtain the current frame depth features and multi-dimensional feature vectors; similarly, the deep belief network model is used to extract the depth features of the pre-positioned frame.
6. The method according to claim 1, wherein: The deep forest classifier is used to provide category information of the detection target; The deep forest structure is composed of multiple layers of cascaded random forests. Each layer of cascaded forests includes two completely random forests and two random forests. Each forest contains 500 random trees. The completely random trees use all samples as input. Each node of the decision tree randomly selects a feature for splitting until the instances contained in each leaf node belong to the same class, or the number of instances is no more than 10. This method increases the diversity of samples. The training process of the random forest is to input a random subset of samples to each tree, and the splitting node selects a class with the optimal Gini coefficient for splitting. This method ensures the effect of the algorithm. The deep features of the pre-positioned area are input into the first layer of the cascade forest of the deep forest. For the Z classification problem, this layer of the cascade forest outputs a Z-dimensional class feature vector and aggregates it with the deep features to obtain a new feature vector, which is input into the next layer of the cascade forest. The process is iterated in sequence, and the category with the maximum value of the class feature vector output by the last layer of the cascade forest is taken as the category information of the detected target.