Visual navigation quality factor classification method and computer equipment based on deep learning
Through the visual navigation quality factor classification method based on deep learning, the convolutional neural network and image matching algorithm are used to solve the problem of low accuracy in quality evaluation of visual navigation systems, and automatic classification and quantitative evaluation of visual navigation quality are realized, improving the Performance and stability of navigation systems.
Patent Information
- Application Number
- CN202410318659.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-20
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-03-20
AI Technical Summary
In the prior art, the accuracy of the quality evaluation of visual navigation systems is low, which affects the performance and stability of the navigation system. Especially in the event of environmental changes and sensor failures, it is difficult to accurately quantify and optimize navigation performance.
Adopting the visual navigation quality factor classification method based on deep learning, the quality factor classification data set is constructed by building a convolutional neural network, combining image matching algorithm and navigation data, and training the network to output visual navigation quality levels and confidence to achieve visual navigation Automatic classification and quantitative evaluation of quality.
It improves the positioning accuracy and stability of the visual navigation system, accurately identify and classify visual navigation quality factors through automated means, optimizes system performance, and enhances the accuracy and reliability of the navigation system.
Smart Images

Figure CN118196498B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology and relates to a visual navigation quality factor classification method based on deep learning and a computer device. The method combines computer vision, deep learning and navigation system technology, and aims to identify and classify the positioning quality of the visual navigation system in an automated manner, thereby improving the performance and stability of the navigation system. Background Art
[0002] With the rapid development of artificial intelligence and drone technology, visual navigation systems have become an important part of drones, autonomous vehicles and other fields. However, in practical applications, due to environmental changes, sensor failures and other reasons, the performance of visual navigation systems may fluctuate or decline, affecting the accuracy and stability of navigation. Therefore, based on the problems of low accuracy and poor effect of quality evaluation of traditional visual navigation systems, it is of great significance to accurately quantify and evaluate the quality factor of visual navigation systems for the optimization and improvement of system performance. Summary of the invention
[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.
[0004] To this end, the present invention provides a visual navigation quality factor classification method and computer equipment based on deep learning.
[0005] The technical solution of the present invention is as follows:
[0006] According to one aspect, a method for classifying visual navigation quality factors based on deep learning is provided, the method comprising:
[0007] Collect image data and navigation data, and pre-process the collected data using an image matching algorithm to construct a visual navigation quality factor classification data set as the original training data;
[0008] Construct a quality factor classification network based on deep learning, including:
[0009] The quality factor classification network adopts a convolutional neural network architecture to complete the binary classification task, which includes a backbone network module and a fully connected layer module, wherein the backbone network module is used to map input data to the feature space of the network and learn the feature map of the network, and the fully connected layer decodes the feature map of the backbone network, and outputs the confidence and category of the classification by nonlinearly combining various patterns and features learned by the backbone network;
[0010] Inputting the original training data set into the quality factor classification network to perform visual navigation quality classification training, so that the trained quality factor classification network outputs the visual navigation quality grade and confidence;
[0011] Image data and navigation data are collected in real time, and the real-time collected data are processed by an image matching algorithm and then input into the trained quality factor classification network for automatic classification of visual navigation quality.
[0012] Furthermore, the collecting of image data and navigation data, and preprocessing of the collected data using an image matching algorithm to construct a visual navigation quality factor classification data set specifically includes:
[0013] The infrared aerial images are collected by the camera, while the navigation data reference is provided by the inertial navigation and satellite receivers;
[0014] The commercial map range is determined according to the corresponding flight area obtained from the navigation reference data, and the commercial map corresponding to the aerial image is obtained through commercial satellite map software. The commercial map and the infrared aerial image are used together as inputs of the image matching algorithm, and the data processed by the image matching algorithm is used to produce a quality factor classification data set.
[0015] Furthermore, the image matching algorithm processing specifically includes:
[0016] The aerial images and commercial maps are inferred through the image matching algorithm, and the output results are reprocessed through the RANSAC outlier removal algorithm to finally obtain the homography matrix and the number of inliers. Among them, the width and height of the aerial images and commercial maps, the number of feature points, the number of matches, the number of inliers and the homography matrix parameters in the output results, a total of 16 parameters, are used as the input of the quality factor classification network.
[0017] Furthermore, when constructing the classification data set, the categories are divided according to the final positioning error of the visual navigation. The positioning error is less than 200m and is classified as Class 0, otherwise it is classified as Class 1. The same preset number of groups of Class 0 images and Class 1 images are selected from the collected data to construct a training set.
[0018] Furthermore, the backbone network module backbone network is composed of a 5-layer stacked convolution structure, each convolution structure includes a convolution layer and an activation function, wherein the convolution layer performs a convolution operation through a 3*3 convolution kernel, and the number of output channels corresponding to the 5-layer convolution structure is 64-128-256-128-64 respectively.
[0019] Furthermore, the activation function uses a ReLU activation function for nonlinear activation.
[0020] Furthermore, the fully connected layer module includes three layers of full connection, and the output paths of the three layers of full connection are 32, 16, and 2 respectively.
[0021] Furthermore, during training, the classification results output by the fully connected layer module are normalized by Softmax to output the confidence of the two categories, wherein the one with the larger confidence is taken as the quality grade, and the corresponding confidence is the probability of the quality grade.
[0022] Furthermore, when constructing the quality factor classification network based on deep learning, the cross entropy loss function is also used, which is defined as follows:
[0023]
[0024] Among them, x1 is the value when the quality level of the quality factor classification network output is 0, and vice versa, it is the value when the quality level is 1.
[0025] According to another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.
[0026] The above technical solution combines the output results and quality of image matching to produce a quality factor classification data set. At the same time, the quality factor classification network is designed to adopt a convolutional neural network architecture to complete the binary classification task. The quality factor classification network outputs the corresponding quality level and confidence. That is, the positioning accuracy of visual navigation can be directly determined only through the output at the image matching level, thereby accurately realizing the automatic classification and quantitative evaluation of the visual navigation quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0028] Figure 1 is a quality factor classification network according to an embodiment of the present invention;
[0029] Figure 2 It is the training loss curve of the embodiment of the present invention. DETAILED DESCRIPTION
[0030] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0031] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0032] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values of the parts and steps set forth in these embodiments do not limit the scope of the present invention. Meanwhile, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant art may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.
[0033] In one embodiment of the present invention, a visual navigation quality factor classification method based on deep learning is provided, the method comprising:
[0034] Step 1: Collect image data and navigation data, and pre-process the collected data using an image matching algorithm to construct a visual navigation quality factor classification data set as original training data;
[0035] Step 2: Build a quality factor classification network based on deep learning, including:
[0036] The quality factor classification network adopts a convolutional neural network architecture to complete the binary classification task, which includes a backbone network module and a fully connected layer module, wherein the backbone network module is used to map input data to the feature space of the network and learn the feature map of the network, and the fully connected layer decodes the feature map of the backbone network, and outputs the confidence and category of the classification by nonlinearly combining various patterns and features learned by the backbone network;
[0037] Step 3: inputting the original training data set into the quality factor classification network to perform visual navigation quality classification training, so that the trained quality factor classification network outputs the visual navigation quality grade and confidence;
[0038] Step 4: collect image data and navigation data in real time, process the real-time collected data using an image matching algorithm, and then input the data into the trained quality factor classification network for automatic classification of visual navigation quality.
[0039] Applying the above configuration method, by combining the output results and quality of image matching, a quality factor classification dataset is produced. At the same time, the quality factor classification network is designed to adopt a convolutional neural network architecture to complete the binary classification task. The quality factor classification network outputs the corresponding quality level and confidence. That is, the positioning accuracy of visual navigation can be directly determined only through the output of the image matching level, and the automatic classification and quantitative evaluation of the visual navigation quality can be accurately realized.
[0040] In the above embodiment, constructing a visual navigation quality factor classification data set is the key point of the present invention. In step 1, a large amount of image data and navigation data needs to be collected, and the collected data is preprocessed to construct a quality factor classification data set, and establish a connection between image matching results and visual positioning accuracy; specifically, step 1 specifically includes:
[0041] 1.1 Data Collection
[0042] Mount cameras, inertial navigation, satellite receivers and other equipment on the drone to collect time-synchronized image data and navigation reference data. The camera is installed on the belly of the drone to collect infrared aerial images; the inertial navigation and satellite receiver are installed at the center of mass of the drone to provide navigation data references. Determine the commercial map range based on the corresponding flight area obtained from the navigation reference data, and obtain the commercial map corresponding to the aerial image through free commercial satellite map software. Use it together with the aerial image as the input of the image matching algorithm, and reprocess the output result to produce a quality factor classification data set;
[0043] 1.2 Dataset Creation
[0044] The aerial images and commercial maps are inferred by the existing image matching algorithm, and the output results are reprocessed by the RANSAC and other external point elimination algorithms to finally obtain the homography matrix and the number of internal points. The width and height, number of feature points, number of matches, number of internal points and homography matrix parameters of the aerial images and commercial maps in the output results are 16 parameters (where the width and height, number of feature points are for two images, a total of 6 parameters; the number of matches and number of internal points are shared by two images, a total of 2; the valid parameters in the homography matrix are 8, a total of 16 parameters) as the input of the network; Among them, according to the final positioning error of the visual navigation, the positioning error is less than 200m and is classified as Class 0, otherwise it is classified as Class 1; 4000 groups of Class 0 images and Class 1 images (such images are processed by the above algorithm) can be selected from the collected data as training sets, preferably, 800 groups each as validation sets, and a total of 9600 groups of data constitute the quality factor classification data set Quality-Dataset.
[0045] It can be seen that the embodiment of the present invention first obtains aerial images and navigation reference data through airborne cameras, inertial navigation and satellite receivers, then determines the commercial map range based on the corresponding flight area obtained from the navigation reference data, and then obtains the commercial map corresponding to the aerial image through free commercial map software. The aerial image and the commercial map are matched through the existing image matching algorithm, and the RANSAC and other algorithms are used to eliminate false matches, perform visual positioning, determine the quality of the image matching result based on the positioning result and the navigation reference, and finally combine the output result of the image matching with the quality to produce a quality factor classification data set Quality-Dataset. Among them, the quality factor classification network training part is trained and verified based on the Quality-Dataset data set, in order to obtain the relationship between the output result of the image matching algorithm and the quality factor, so as to determine the positioning accuracy and credibility of the visual navigation result.
[0046] In the above embodiment, if Figure 1 As shown, constructing a quality factor classification network based on deep learning is the key point of the present invention. Specifically:
[0047] The quality factor classification network uses a convolutional neural network architecture to complete the binary classification task, including a backbone network and a fully connected layer. The backbone network maps the input 16-dimensional input vector to a hidden space that is easy for the network to understand, and learns the network's feature map. The fully connected layer decodes the backbone network's feature map and outputs the classification confidence and category through a nonlinear combination of features. The overall network framework is as follows: Figure 1 shown.
[0048] The backbone network is composed of a stack of 5 convolutional structures, each of which includes a convolutional layer and an activation function. The convolutional layer of the backbone network performs convolution operations through a 3*3 convolution kernel, and the number of output channels is 64-128-256-128-64 respectively; the activation function uses the ReLU activation function for nonlinear activation. After passing through the backbone network, the 16-dimensional input variable can be mapped to the feature space of the network for pattern learning.
[0049] The fully connected layer contains three layers of full connection, which nonlinearly combine the various patterns and features learned by the backbone network. The output channels of the three layers of full connection are 32, 16, and 2 respectively. Finally, the classification results are normalized by Softmax, and the confidence of the two categories is output. The larger confidence is taken as the quality level (0 or 1 in the figure), and the corresponding confidence is the probability of the quality level.
[0050] Moreover, for the binary classification problem, the network of the embodiment of the present invention adopts the cross entropy loss function. The output of the model usually represents the probability of a certain category, and the cross entropy loss function can be used to measure the difference between this probability distribution and the distribution of the true label, that is, how accurate the model's estimation of the correct category is. The loss function is defined as follows:
[0051]
[0052] Among them, x1 is the value when the quality level of the quality factor classification network output is 0, and vice versa, it is the value when the quality level is 1.
[0053] As a specific embodiment of the present invention, when training the quality factor classification network, a quality factor classification network is built by python under the pytorch framework, and the quality factor classification data set reading code, network training code and loss function are written, and epochs, optimizer and learning rate are set for supervised training. According to the verification results of the verification set, the network with the best classification performance is determined as the final quality factor classification network.
[0054] Specifically, we first build a data set reading program, use the pytorch framework to read and divide the training set and validation set, and use iterative supervised training to let the network learn the relationship between the image matching output results and the visual navigation positioning accuracy. Since the quality factor is a binary classification task with low difficulty, the network structure is relatively simple, and the model parameters are 114KB in total, which is relatively lightweight. The optimizer uses the Adam optimizer, and the initial value of the learning rate is 0.001 and is reduced by 1 / 10 every 5 epochs. The loss value curve of the network training process is as follows: Figure 2 shown.
[0055] Finally, after obtaining the final quality factor classification network, the inference and acceleration engine of the quality factor classification network is constructed, and the output results of the real-time image matching algorithm are used as the input of the network. The quality factor classification network will output the corresponding quality level and confidence. That is, the positioning accuracy of visual navigation can be directly determined only by the output at the image matching level.
[0056] According to another embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.
[0057] In summary, the embodiments of the present invention aim to address the problems of low quality evaluation accuracy and poor effect of traditional visual navigation systems, and conduct research on visual navigation quality factor classification technology based on deep learning. The positioning accuracy of visual navigation can be directly and accurately determined only through the output of the image matching level.
[0058] Features described and / or illustrated above for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or used in place of features in other embodiments.
[0059] It should be emphasized that the term "include / comprises" when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps, components or combinations thereof.
[0060] The above method of the present invention can be implemented by hardware, or by hardware combined with software. The present invention relates to such a computer-readable program, which, when executed by a logic component, enables the logic component to implement the above-mentioned device or component, or enables the logic component to implement the above-mentioned various methods or steps. The present invention also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.
[0061] The many features and advantages of these embodiments are apparent from this detailed description, and thus the appended claims are intended to cover all such features and advantages of these embodiments that fall within their true spirit and scope. Furthermore, since numerous modifications and changes will readily occur to those skilled in the art, it is not intended that the embodiments of the invention be limited to the exact construction and operation illustrated and described, but rather all suitable modifications and equivalents falling within the scope thereof are intended to be covered.
[0062] Parts of the present invention that are not described in detail are well known to those skilled in the art.
Claims
1. A visual navigation quality factor classification method based on deep learning, characterized in that: The method comprises: Image data and navigation data are collected, and the collected data are preprocessed using an image matching algorithm to construct a visual navigation quality factor classification data set as the original training data, specifically including: The infrared aerial images are collected by the camera, while the navigation data reference is provided by the inertial navigation and satellite receivers; Determine the scope of the commercial map according to the corresponding flight area obtained from the navigation reference data, and obtain the commercial map corresponding to the aerial image through commercial satellite map software, use the commercial map and the infrared aerial image as inputs of the image matching algorithm, and produce a quality factor classification data set after the data is processed by the image matching algorithm; Construct a quality factor classification network based on deep learning, including: The quality factor classification network adopts a convolutional neural network architecture to complete the binary classification task, which includes a backbone network module and a fully connected layer module, wherein the backbone network module is used to map the input data to the feature space of the network and learn the feature map of the network, and the fully connected layer module decodes the feature map of the backbone network, and outputs the confidence and category of the classification by nonlinearly combining various modes and features learned by the backbone network; the fully connected layer module includes three layers of full connection, and the outputs of the three layers of full connection are 32, 16, and 2 respectively. During training, the classification results output by the fully connected layer module are also normalized by Softmax, and the confidence of the binary classification is output, wherein the larger confidence is taken as the quality level, and the corresponding confidence is the probability of the quality level; Inputting the original training data set into the quality factor classification network to perform visual navigation quality classification training, so that the trained quality factor classification network outputs the visual navigation quality grade and confidence; Collecting image data and navigation data in real time, processing the real-time collected data with an image matching algorithm and inputting the data into the trained quality factor classification network for automatic classification of visual navigation quality; The image matching algorithm processing specifically includes: The aerial images and commercial maps are inferred through the image matching algorithm, and the output results are reprocessed through the RANSAC outlier removal algorithm to finally obtain the homography matrix and the number of inliers. Among them, the width and height of the aerial images and commercial maps, the number of feature points, the number of matches, the number of inliers and the homography matrix parameters in the output results, a total of 16 parameters, are used as the input of the quality factor classification network.
2. The method for classifying visual navigation quality factors based on deep learning according to claim 1, characterized in that: When constructing the classification data set, the categories are divided according to the final positioning error of the visual navigation. The positioning error is less than 200m and is classified as Class 0, otherwise it is classified as Class 1. The same preset number of groups of Class 0 images and Class 1 images are selected from the collected data to construct a training set.
3. A visual navigation quality factor classification method based on deep learning according to claim 1 or 2, characterized in that: The backbone network module backbone network is composed of a 5-layer stacked convolution structure, each convolution structure includes a convolution layer and an activation function, wherein the convolution layer performs a convolution operation through a 3*3 convolution kernel, and the output channel numbers corresponding to the 5-layer convolution structure are 64-128-256-128-64 respectively.
4. The method for classifying visual navigation quality factors based on deep learning according to claim 3, characterized in that: The activation function uses the ReLU activation function for nonlinear activation.
5. The method for classifying visual navigation quality factors based on deep learning according to claim 1, characterized in that: When constructing the quality factor classification network based on deep learning, the cross entropy loss function is also used, which is defined as follows: in, The quality factor classification network outputs the value when the quality level is 0, and vice versa, it outputs the value when the quality level is 1.
6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Electronic navigation map quality detection method and device
CN112414430A
Image matching and positioning method based on deep learning and computer equipment
CN117253029A
Methods for generating a deep neural net and for localising an object in an input image, deep neural net, computer program product, and computer-readable storage medium
US20220076117A1
Deep learning system for navigating feedback
WO2023224672A1
