Unmanned aerial vehicle ranging method, system and device based on uwb and binocular vision

By combining UWB and binocular vision in a drone ranging method, and using convolutional neural networks and BP neural networks to compensate and correct the ranging results of the binocular camera, the problems of low ranging accuracy and consistency of drones are solved, and a higher accuracy ranging effect is achieved.

CN116125377BActive Publication Date: 2025-11-04SHANTOU UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310087438.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-01-18
Publication Date
2025-11-04
Estimated Expiration
2043-01-18

AI Technical Summary

Technical Problem

Existing UAV ranging methods suffer from problems such as low ranging accuracy, high cost, weak anti-interference ability, and inconsistent measurement results. In particular, the inherent drawbacks of ultrasonic, infrared laser, and binocular vision ranging methods make it difficult to guarantee the accuracy of UAV ranging.

Method used

By combining UWB and binocular vision, a drone equipped with a binocular camera and UWB device is used to process target images using a convolutional neural network to obtain matching costs. A backpropagation neural network is built and a loss function is constructed using the ranging results from the UWB device. The ranging results from the binocular camera are then iteratively corrected to improve ranging accuracy.

Benefits of technology

By introducing a BP neural network to compensate and correct the ranging results of the binocular camera, and using the ranging results of the UWB device as a reference value, the ranging accuracy of binocular vision is improved, solving the problems of low ranging accuracy and consistency in the existing technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116125377B_ABST
    Figure CN116125377B_ABST
Patent Text Reader

Abstract

The application discloses a UAV ranging method, system and equipment based on UWB and binocular vision, wherein a binocular camera and a UWB device are carried on the body of the UAV, and the method comprises the following steps: inputting a target image pair collected by the binocular camera into a trained convolutional neural network for processing to obtain a matching cost; determining a first target distance according to the matching cost and parameters of the binocular camera; building a BP neural network and constructing a loss function of the BP neural network by using a second target distance measured by the UWB device; and inputting the first target distance into the BP neural network for iterative correction processing and outputting a final target distance. The ranging result of the binocular camera is compensated and corrected by introducing the BP neural network, and in this process, the ranging result of the UWB device is used as a reference value to construct the loss function of the BP neural network, so that the ranging precision of the binocular vision can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, in particular to an unmanned aerial vehicle ranging method, system and equipment based on UWB and binocular vision. BACKGROUND

[0002] The existing unmanned aerial vehicle ranging methods include ultrasonic ranging method, infrared laser ranging method and binocular vision ranging method, but each of the three methods has certain disadvantages: first, the action distance of the ultrasonic sensor is short, and the ranging result accuracy is not high; second, the infrared laser ranging method has high cost, weak anti-interference ability and high requirement for device computing power; third, the measurement accuracy of the binocular vision ranging method is affected by camera performance, current lighting conditions and baseline length (i.e. the distance between the two cameras). More importantly, the distance data measured by the unmanned aerial vehicle using the three methods in the same scene may not be the same, and it is difficult to ensure the accuracy of the unmanned aerial vehicle ranging. SUMMARY

[0003] The present application provides an unmanned aerial vehicle ranging method, system and equipment based on UWB and binocular vision to solve one or more technical problems in the prior art and at least provide a beneficial choice or create conditions.

[0004] In a first aspect, an unmanned aerial vehicle ranging method based on UWB and binocular vision is provided, wherein a binocular camera and a UWB device are carried on the body of the unmanned aerial vehicle, and the method comprises:

[0005] inputting a target image pair collected by the binocular camera into a trained convolutional neural network for processing to obtain a matching cost;

[0006] determining a first target distance according to the matching cost and parameters of the binocular camera;

[0007] building a BP neural network and constructing a loss function of the BP neural network using a second target distance measured by the UWB device;

[0008] importing the first target distance into the BP neural network for iterative correction processing and outputting a final target distance.

[0009] Further, the training method of the convolutional neural network is:

[0010] using the unmanned aerial vehicle carrying the binocular camera to collect images of multiple targets in a real scene to obtain an image data set;

[0011] after assigning a label representing the matching cost to each image pair in the image data set, inputting the image data set into the convolutional neural network for model training and optimizing the model weight.

[0012] Further, the convolutional neural network comprises a first input layer, a first output layer and four first hidden layers, wherein each first hidden layer is internally provided with a convolution kernel with a size of 64*1*3*3, and the output end of each first hidden layer is introduced into a ReLU function for processing.

[0013] The first input layer is used to acquire two gray image blocks with a size of 9*9, which are corresponding to the target image pair intercepted; the four first hidden layers are used to perform multi-layer convolution processing on the two gray image blocks respectively to obtain two feature vectors with a dimension of 64*1; and the first output layer is used to perform vector inner product accumulation on the two feature vectors to obtain a matching cost.

[0014] Further, the determination of the first target distance according to the matching cost and the parameters of the binocular camera comprises:

[0015] The cross-domain cost aggregation algorithm is used to perform iterative operation on the matching cost, and the winner-takes-all strategy is used to analyze the output iterative operation result to obtain disparity information.

[0016] The focal length of the binocular camera and the center distance between the two cameras are acquired, and the first target distance is calculated in combination with the disparity information.

[0017] Further, the BP neural network comprises a second input layer, a second hidden layer and a second output layer, wherein the activation function adopted by the second hidden layer is a S-type transfer function, and the activation function adopted by the second output layer is a linear function.

[0018] Further, the loss function is used to represent the absolute value of the difference between the target distance output by the BP neural network after each iteration correction processing and the second target distance.

[0019] In a second aspect, an unmanned aerial vehicle ranging system based on UWB and binocular vision is provided, wherein the body of the unmanned aerial vehicle is mounted with a binocular camera and a UWB device, and the system comprises:

[0020] A processing module is configured to input a target image pair acquired by the binocular camera into a trained convolutional neural network for processing to obtain a matching cost.

[0021] A determination module is configured to determine a first target distance according to the matching cost and the parameters of the binocular camera.

[0022] A building module is configured to build a BP neural network and construct a loss function of the BP neural network by using a second target distance measured by the UWB device.

[0023] A correction module is configured to introduce the first target distance into the BP neural network for iterative correction processing, and output a final target distance.

[0024] Further, the determination module comprises:

[0025] An analysis unit is configured to perform iterative operation on the matching cost by using a cross-domain cost aggregation algorithm, and analyze the iterative operation result by using a winner-takes-all strategy to obtain disparity information.

[0026] A calculation unit is configured to obtain the focal length of the binocular camera and the center distance between the two cameras, and calculate a first target distance in combination with the disparity information.

[0027] In a third aspect, a computer device is provided, comprising:

[0028] at least one memory;

[0029] at least one processor;

[0030] The memory stores a computer program, and the processor executes the computer program to implement the UWB and binocular vision based UAV ranging method according to the first aspect.

[0031] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the UWB and binocular vision based UAV ranging method according to the first aspect is implemented.

[0032] The present application has at least the following beneficial effects: the ranging result of the binocular camera is compensated and corrected by introducing the BP neural network, and the ranging result of the UWB device is used as a reference value to construct the loss function of the BP neural network, so that the ranging accuracy of the binocular vision can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with the embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0034] Figure 1 is a composition schematic diagram of an integrated measuring device in the embodiment of the present application;

[0035] Figure 2 is a flowchart of a UWB and binocular vision based UAV ranging method in the embodiment of the present application;

[0036] Figure 3 is a composition schematic diagram of a UWB and binocular vision based UAV ranging system in the embodiment of the present application;

[0037] Figure 4 Figure 1 is a hardware structure schematic diagram of a computer device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0039] It should be noted that although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0040] First, some of the terms involved in the present application are explained as follows:

[0041] Convolutional neural network is a kind of feedforward neural network containing convolution calculation and having deep structure, and is one of the representative algorithms of deep learning. Convolutional neural network has the ability of feature learning and can perform translation-invariant classification on input information according to its hierarchical structure, so it is also called "translation-invariant artificial neural network". Convolutional neural network contains a feature extractor composed of convolution layer and subsampling layer. The convolution layer usually contains several feature maps, each of which is composed of some rectangular arranged neurons, and the neurons of the same feature map share weights (i.e. convolution kernel). The convolution kernel is usually initialized in the form of a random small number matrix and updated during the training process of the network, and the sharing of weights can reduce the connection between layers of the network and reduce the risk of overfitting. The subsampling layer is also called the pooling layer, which usually has two forms: mean pooling and maximum pooling, both of which can be regarded as a special convolution process.

[0042] BP (Back Propagation) neural network is a kind of multi-layer feedforward neural network trained according to the error back propagation algorithm. It is actually a data processing system that can automatically realize the nonlinear mapping relationship between the input layer and the output layer. The essence of the BP algorithm proposed is to process the input signal from the input layer through the hidden layer layer by layer until the output layer. If the output layer does not get the expected output, it will enter the reverse propagation. According to the prediction error, the network weights and thresholds are adjusted, so that the error square sum between the network output value and the expected value is minimized or less than the set precision.

[0043] Cross-based Cost Aggregation algorithm aggregates cost by constructing a cross-domain of a pixel, which is based on an assumption that pixels with similar color and distance have similar disparity value, and finds a cross-domain with similar color and distance to a pixel, and aggregates cost through the cross-domain to obtain a final reliable matching cost.

[0044] WTA (Winner Take All) strategy lists matching cost of each depth value in cost space, and the smallest disparity value is the disparity value of a reference pixel.

[0045] Reference Figure 1 , Figure 1 is a component diagram of the integrated measuring device provided by the embodiment of the application, the integrated measuring device is installed on the body of the unmanned aerial vehicle, and the integrated measuring device is composed of a microcomputer device, a binocular camera and a UWB (Ultra Wide Band) device. The binocular camera actually includes a left camera and a right camera. The UWB device is installed on the top surface of the microcomputer device. The left camera is installed on the left side surface of the microcomputer device. The right camera is installed on the right side surface of the microcomputer device. In the embodiment of the application, the microcomputer device is an Intel NUC microcomputer, and NUC stands for Next Unit of Computing.

[0046] In the specific implementation process, the binocular camera is used to collect a target image pair and feed it back to the microcomputer device. The target image pair includes a left image containing a target object collected by the left camera and a right image containing the target object collected by the right camera. The UWB device is used to measure a second target distance between the unmanned aerial vehicle and the target object and feed it back to the microcomputer device. The microcomputer device is used to process the target image pair by using a convolutional neural network built and trained internally to obtain a matching cost, and then calculate a first target distance according to the previously recorded parameters of the binocular camera and the matching cost. The microcomputer device is also used to build a BP neural network, use the second target distance to construct a loss function of the BP neural network, and use the BP neural network to iteratively correct the first target distance to obtain a final target distance.

[0047] Based on the integrated measuring device provided by the embodiment of the application, Figure 1 Figure 2 ​is a flowchart of a UAV ranging method based on UWB and binocular vision provided by an embodiment of the present application, and the method comprises the following steps.

[0048] In step S110, the target image pair collected by the binocular camera is input to the trained convolutional neural network for processing to obtain a matching cost.

[0049] In step S120, a first target distance is determined according to the matching cost and the parameters of the binocular camera.

[0050] In step S130, a BP neural network is built, and a loss function of the BP neural network is constructed by using a second target distance measured by the UWB device.

[0051] In step S140, the first target distance is input to the BP neural network for iterative correction processing, and a final target distance is output.

[0052] In the embodiment of the present application, the training method of the convolutional neural network mentioned in step S110 is as follows: first, a plurality of targets are arranged at certain interval distances in a real scene, the UAV is controlled to fly above each target and the binocular camera carried by the UAV is used to perform image collection at different shooting angles for each target, thereby obtaining an image data set composed of different image pairs, and each image pair contains only one target; then, the existing algorithm is used to solve the matching cost of each image pair in the image data set, and the matching cost obtained by solving is bound with the corresponding image pair; finally, the image data set bound with the label is used to train the model of the convolutional neural network, and the model weight is optimized during the training process.

[0053] In the embodiment of the present application, the convolutional neural network mentioned in step S110 is actually a full convolutional neural network structure, which is composed of a first input layer, a first output layer and four first hidden layers, wherein each first hidden layer is internally provided with a convolution kernel and has a limited size of 64*1*3*3, and the output result of each first hidden layer needs to be processed by a ReLU (Rectified Linear Unit) function first.

[0054] In the implementation process, based on the target image pair collected by the binocular camera, which has been converted into a target image pair displayed in a gray mode in advance, two gray image blocks containing the target object are first intercepted according to a specific size, which is set to 9*9, and then input into the first input layer. Then, the two gray image blocks output by the first input layer are subjected to multi-layer convolution processing by the four first hidden layers, and two feature vectors are output, and the dimensions of the two feature vectors are both 64*1. It should be noted that in this convolution process, the padding width of each first hidden layer is set to 0. Finally, the two feature vectors output by the fourth first hidden layer connected to the first output layer are subjected to inner product summation processing by the first output layer, and the matching cost is output. It should be noted that in this processing process, the padding width of the first output layer is set to 1.

[0055] In the embodiment of the application, the specific implementation process of step S120 includes the following steps:

[0056] Step S121, using an existing cross-domain cost aggregation algorithm to perform four iteration operations on the matching cost obtained through step S110 to obtain the final iteration operation result C4(p,d), and the corresponding iteration calculation formula is:

[0057]

[0058]

[0059] wherein C CNN (p,d) is the matching cost obtained through step S110, i is the iteration number, i is a positive integer and 1≤i≤4, C i (p,d) is the matching cost of p point at disparity d obtained after the i-th iteration, C i-1 (q,d) is the matching cost of q point at disparity d obtained at the i-1-th iteration, p is a pixel point on any gray image block, U d (p) is a cross window (also known as a support region) generated based on p point, q is a pixel point contained in the cross window U d (p).

[0060] Step S122, according to the winner-takes-all strategy, the iteration operation result obtained through step S121 is analyzed to obtain the disparity information d=argminC4(p,d).

[0061] Step S123, according to the focal length f of the binocular camera and the center distance T between the left camera and the right camera, and in combination with the parallax information d obtained through the above step S122, the first target distance Z measured by the binocular camera can be calculated as Z=(f*T) / d.

[0062] In the embodiment of the present application, the specific measurement process of the second target distance mentioned in the above step S130 is as follows: a plurality of UWB ground base stations are set around the target object, and the relative positions between each two UWB ground base stations and the phase positions between each UWB ground base station and the target object can be obtained through field measurement or other measurement methods; the UWB device carried on the body of the UAV sends a ranging request data packet to each UWB ground base station and records the sending time as T1; when each UWB ground base station receives the ranging request data packet, it returns a reply information data packet to the UWB device; the UWB device records the time of receiving the reply information data packet returned by each UWB ground base station as T2, and the distance D between the UAV and each UWB ground base station can be calculated as D=[C*(T2-T1)] / 2, C being the speed of light, and then the second target distance between the UAV and the target object can be obtained through geometric operation in combination with the relative positions between each two UWB ground base stations and the phase positions between each UWB ground base station and the target object.

[0063] In the embodiment of the present application, the BP neural network mentioned in the above step S130 is composed of a second input layer, a second output layer and a second hidden layer; the second hidden layer adopts an existing S-type transfer function (i.e. tansig function) as a first activation function, and the corresponding function expression is y1=tansig(w1*p+b1), wherein w1 is the weight value of the second hidden layer, b1 is the threshold value of the second hidden layer, p is the to-be-optimized information input by the second input layer, y1 is the output information of the second hidden layer, and the first activation function is used to collect and convert the output information of each neuron in the second hidden layer into new output information; the second output layer adopts an existing linear function (i.e. purelin function) as a second activation function, and the corresponding function expression is y2=purelin(w2*y1+b2), wherein w2 is the weight value of the second output layer, b2 is the threshold value of the second output layer, and y2 is the output information of the second output layer, and the second activation function is used to collect and convert the output information of each neuron in the second output layer into new output information.

[0064] In the embodiment of the present application, the loss function is constructed in the BP neural network, and the corresponding function expression is loss = |y2-Z UWB | which represents the absolute value of the difference between the target distance y2 output by the BP neural network after each iteration correction and the second target distance Z UWB .

[0065] In the embodiment of the present application, the maximum number of training is set to 500 and the adaptive learning rate is set to 0.001, the BP neural network is iteratively trained and parameter optimized, and the whole training process includes a forward propagation step and a backward propagation step. The output information is obtained after the input information to be optimized input by the second input layer is processed by the second hidden layer and the second output layer in the forward propagation step. If the output information can make the loss function reach a convergence state, the training is ended. If not, the backward propagation step is turned to, and the output result of the loss function is used as a gradient to adjust the related weight value and threshold value by using the gradient descent method. It should be noted that the input information to be optimized used by the BP neural network in the first iteration is the first target distance obtained by the above step S120.

[0066] In the embodiment of the present application, the ranging error of binocular vision will present exponential growth as the measured distance becomes larger. By introducing the BP neural network to compensate and correct the ranging result of the binocular camera, and taking the ranging result of the UWB device as a reference value to construct the loss function of the BP neural network in the process, the ranging accuracy of binocular vision can be improved.

[0067] Based on Figure 1 the provided integrated measuring device, Figure 3 is a component schematic diagram of a UAV ranging system based on UWB and binocular vision provided in the embodiment of the present application, and the system comprises:

[0068] The processing module 210 is configured to input the target image pair collected by the binocular camera into the trained convolutional neural network for processing to obtain the matching cost.

[0069] The determining module 220 is configured to determine the first target distance, and the determination process of the first target distance is related to the parameters of the binocular camera and the matching cost.

[0070] The building module 230 is configured to build a BP neural network, and the loss function used by the building module 230 is constructed in combination with the second target distance measured by the UWB device.

[0071] The correction module 240 is configured to input the first target distance into the BP neural network for iterative correction processing to output the final target distance.

[0072] More specifically, the determining module 220 comprises an analysis unit and a calculation unit, wherein: the analysis unit is configured to perform iterative operation on the matching cost by using a cross-domain cost aggregation algorithm to obtain a final output result, and then analyze the final output result by using a winner-takes-all strategy to obtain disparity information; and the calculation unit is configured to obtain the focal length of the binocular camera and the center distance between the two cameras, and calculate a first target distance in combination with the disparity information.

[0073] It should be noted that the processing module 210, the determining module 220, the building module 230 and the correcting module 240 are all executed by the microcomputer device inside the integrated measuring device.

[0074] The contents in the above method embodiments are all applicable to the present system embodiment, the present system embodiment achieves the same functions as the above method embodiments, and achieves the same beneficial effects as the above method embodiments, which will not be described herein again.

[0075] In addition, the present embodiment further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the above-mentioned UWB and binocular vision based unmanned aerial vehicle ranging method. The computer readable storage medium includes but is not limited to any type of disk (including a floppy disk, a hard disk, an optical disk, a CD-ROM, and a magneto-optical disk), a ROM (Read-Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, a magnetic card or an optical card. That is, the storage device includes any medium that stores or transmits information in a readable form by a device (for example, a computer, a mobile phone, etc.), which can be a read-only memory, a magnetic disk or an optical disk, etc.

[0076] In addition, Figure 4 is a hardware structure schematic diagram of the computer device provided by the present embodiment, and the computer device comprises a processor 320, a memory 330, an input unit 340, a display unit 350 and the like. Those skilled in the art can understand that the computer device can further comprise other components, and the other components will not be described herein. Figure 4The illustrated device structure does not constitute a limitation on all devices, and can include more or fewer components than shown, or combine certain components. The memory 330 can be used to store the computer program 310 and various functional modules, and the processor 320 runs the computer program 310 stored in the memory 330 to perform various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include an internal memory and an external memory. The internal memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random memory. The external memory can include a hard disk, a floppy disk, a ZIP disk, a U disk, a magnetic tape, etc. The memory 330 disclosed in the embodiments of the present application includes but is not limited to the above-mentioned types of memory. The memory 330 disclosed in the embodiments of the present application is only by way of example and not as a limitation.

[0077] The input unit 340 is used to receive the input of signals and receive the keyword input by the user. The input unit 340 can include a touch panel and other input devices. The touch panel can collect the touch operation of the user thereon or nearby (such as the operation of the user on or near the touch panel with a finger, a stylus, or any suitable object or accessory), and drive the corresponding connection device according to the pre-set program; other input devices can include but are not limited to one or more of a physical keyboard, function keys (such as play control buttons, switch buttons, etc.), a trackball, a mouse, a joystick, etc. The display unit 350 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 350 can take the form of a liquid crystal display, an organic light-emitting diode, etc. The processor 320 is the control center of the terminal device, connects all parts of the device through various interfaces and lines, and performs various functions and processes data by running or executing the software program and / or module stored in the memory 330 and calling the data stored in the memory.

[0078] As an embodiment, the computer device includes at least one processor 320, at least one memory 330, and at least one computer program 310, wherein the at least one computer program 310 is stored in the at least one memory 330 and is configured to be executed by the at least one processor 320, and the at least one computer program 310 is configured to perform the UWB and binocular vision based drone ranging method in the above-mentioned embodiments.

[0079] While the application has been described in connection with very specific embodiments, it will be understood that the application is not limited to any of these details or embodiments or any special embodiment, but intends to cover all alternatives, modifications and equivalents as can be included within the scope of the application as defined by the appended claims when interpreted in accordance with the full scope of equivalents, rules of construction and the doctrine of equivalents under 35 U.S.C. § 112. Furthermore, the above description is intended to provide a useful description of the application, and is not intended to limit the application to the particular embodiments or examples described. It is understood that the present application is intended to cover all alternatives, modifications and equivalents as can be included within the scope of the application as defined by the appended claims.

Claims

1. A UAV ranging method based on UWB and binocular vision, characterized in that, The drone is equipped with a binocular camera and a UWB device, and the method includes: The target image acquired by the binocular camera is input into the trained convolutional neural network for processing to obtain the matching cost; The distance to the first target is determined based on the matching cost and the parameters of the binocular camera; A backpropagation (BP) neural network is constructed, and the loss function of the BP neural network is constructed using the second target distance measured by the UWB device. The first target distance is imported into the BP neural network for iterative correction processing, and the final target distance is output.

2. The UAV ranging method based on UWB and binocular vision according to claim 1, characterized in that, The training method for the convolutional neural network is as follows: The drone, equipped with the binocular camera, is used to acquire images of multiple targets in a real-world scene, resulting in an image dataset. After assigning a label to each image pair in the image dataset to represent the matching cost, the image pairs are input into the convolutional neural network for model training and optimization of model weights.

3. The UAV ranging method based on UWB and binocular vision according to claim 1, characterized in that, The convolutional neural network includes a first input layer, a first output layer, and four first hidden layers. Each first hidden layer contains a convolutional kernel with a size of 64*1*3*3, and the output of each first hidden layer is processed by a ReLU function. The first input layer is used to obtain two grayscale image patches with a size of 9*9, and the two grayscale image patches are corresponding to the cropped images from the target image pair; the four first hidden layers are used to perform multi-layer convolution processing on the two grayscale image patches respectively to obtain two feature vectors with a dimension of 64*1; the first output layer is used to perform intra-vector accumulation on the two feature vectors to obtain the matching cost.

4. The UAV ranging method based on UWB and binocular vision according to claim 1, characterized in that, Determining the distance to the first target based on the matching cost and the parameters of the stereo camera includes: The matching cost is iteratively calculated using the cross-domain cost aggregation algorithm, and then the output iterative calculation result is analyzed using the winner-takes-all strategy to obtain disparity information. The focal length of the binocular camera and the center distance between the two cameras are obtained, and the distance to the first target is calculated by combining the parallax information.

5. The UAV ranging method based on UWB and binocular vision according to claim 1, characterized in that, The BP neural network includes a second input layer, a second hidden layer, and a second output layer. The second hidden layer uses a sigmoid transfer function as its activation function, and the second output layer uses a linear function as its activation function.

6. The UAV ranging method based on UWB and binocular vision according to claim 1, characterized in that, The loss function is used to characterize the absolute value of the difference between the target distance output by the BP neural network after each iteration of the correction process and the second target distance.

7. A UAV ranging system based on UWB and binocular vision, characterized in that, The drone is equipped with a binocular camera and UWB devices. The system includes: The processing module is used to process the target image acquired by the binocular camera and input it into the trained convolutional neural network to obtain the matching cost; The determination module is used to determine the distance to the first target based on the matching cost and the parameters of the binocular camera; A module is provided for building a BP neural network and constructing the loss function of the BP neural network using the second target distance measured by the UWB device. The correction module is used to import the first target distance into the BP neural network for iterative correction processing and output the final target distance.

8. The UAV ranging system based on UWB and binocular vision according to claim 7, characterized in that, The determining module includes: The parsing unit is used to perform iterative calculations on the matching cost using the cross-domain cost aggregation algorithm, and then parse the output iterative calculation results in combination with the winner-takes-all strategy to obtain disparity information. The calculation unit is used to obtain the focal length of the binocular camera and the center distance between the two cameras, and to calculate the distance to the first target by combining the parallax information.

9. A computer device, characterized in that, include: At least one memory; At least one processor; The memory stores a computer program, and the processor executes the computer program to implement the UWB and binocular vision-based UAV ranging method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the UWB and binocular vision-based UAV ranging method as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • A stereo image matching method based on a convolutional neural network

    CN109584290A

  • Deep neural network unsupervised learning method based on similarity distances in objects and between objects

    CN111401519A