Multi-rotor unmanned aerial vehicle dual-gimbal control method and device, terminal and medium

Abstract

The embodiment of the application relates to the image processing field, and provides a multi-rotor unmanned aerial vehicle double-gimbal control method and device, a terminal and a medium, the method comprises the following steps: acquiring a first image set of a power exchange platform of an intelligent power exchange base station and acquiring a second image set of the power exchange platform of the intelligent power exchange base station when the multi-rotor unmanned aerial vehicle is subjected to power exchange processing; determining first parking position information of the multi-rotor unmanned aerial vehicle on the power exchange platform through the first image set; if the accuracy of the first parking position information is lower than a preset accuracy threshold, determining second parking position information of the multi-rotor unmanned aerial vehicle on the power exchange platform through the second image set; and performing fusion processing on the first position information and the second position information, target position information, which can improve the accuracy of the multi-rotor unmanned aerial vehicle when determining a power exchange parking position, and further improve the convenience of subsequent power exchange.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a multi-rotor unmanned aerial vehicle dual-gimbal control method and device, a terminal and a medium. BACKGROUND

[0002] The intelligent power exchange base station of the unmanned aerial vehicle is an energy station for providing quick replacement of the power battery of the unmanned aerial vehicle, and the power supply of the unmanned aerial vehicle is realized by directly replacing the power battery pack of the unmanned aerial vehicle, thereby improving the endurance mileage or working time of the unmanned aerial vehicle.

[0003] When the unmanned aerial vehicle performs a long-time flight task, for example, in a fire rescue site, the unmanned aerial vehicle needs to hover over the fire site for a long time, thereby monitoring the change of the fire site in real time. At this time, the battery power of the unmanned aerial vehicle needs to be monitored in real time by the power exchange control system. When the battery power of the unmanned aerial vehicle is low, the unmanned aerial vehicle needs to be guided by the power exchange control system to the intelligent base station in the safe area outside the fire site for power exchange operation. Currently, when the unmanned aerial vehicle is powered, manual cooperation is usually required for power exchange processing, thereby reducing the convenience of power exchange. SUMMARY

[0004] The embodiments of the present application provide a multi-rotor unmanned aerial vehicle dual-gimbal control method, device, terminal and medium, which can improve the accuracy of the multi-rotor unmanned aerial vehicle in determining the power exchange landing position, thereby improving the convenience of subsequent power exchange.

[0005] The first aspect of the embodiments of the present application provides a multi-rotor unmanned aerial vehicle dual-gimbal control method, which is applied to a multi-rotor unmanned aerial vehicle, the multi-rotor unmanned aerial vehicle includes a first gimbal and a second gimbal, the first gimbal carries a first camera, the second gimbal carries a second camera, the shooting resolution of the first camera is lower than that of the second camera, and the shooting angles of the first camera and the second camera are different. The method comprises:

[0006] When the multi-rotor unmanned aerial vehicle is powered, the first image set of the power exchange platform of the intelligent power exchange base station is obtained by controlling the first gimbal and the first camera, and the second image set of the power exchange platform of the intelligent power exchange base station is obtained by the second gimbal and the second camera;

[0007] The first landing position information of the multi-rotor unmanned aerial vehicle on the power exchange platform is determined by the first image set;

[0008] If the accuracy of the first landing position information is lower than a preset accuracy threshold, the second landing position information of the multi-rotor unmanned aerial vehicle on the power exchange platform is determined by the second image set;

[0009] The first position information and the second position information are fused to obtain target position information of the multi-rotor unmanned aerial vehicle when stopping on the power exchange platform during power exchange processing.

[0010] In one possible implementation, the first stopping position information of the multi-rotor unmanned aerial vehicle on the power exchange platform is determined by the first image set, including:

[0011] The first image in the first image set is preprocessed to obtain a first intermediate image set;

[0012] The first intermediate image in the first intermediate image set is used to construct a three-dimensional image of the power exchange platform to obtain a reference three-dimensional image;

[0013] The stopping position in the reference three-dimensional image is extracted to obtain reference stopping position information;

[0014] The reference position corresponding to the reference stopping position information is regionally divided to obtain k sub-reference stopping regions;

[0015] The matching degree between the k sub-reference stopping regions and the battery storage region of the multi-rotor unmanned aerial vehicle is extracted to obtain k first matching degrees;

[0016] According to the k first matching degrees, n sub-target stopping region information is determined from the k sub-reference stopping regions;

[0017] The n sub-target stopping region information is fused to obtain the first stopping position information.

[0018] In one possible implementation, according to the k first matching degrees, n sub-target stopping region information is determined from the k sub-reference stopping regions, including:

[0019] According to the k first matching degrees and the k sub-reference stopping regions, a stopping region matching degree graph is constructed to obtain a first matching degree graph;

[0020] From the first matching degree graph, m reference regions are selected by sliding window to obtain m reference regions;

[0021] The comprehensive matching degree corresponding to the m reference regions is extracted to obtain m comprehensive matching degree information;

[0022] The sub-reference stopping region information of the n reference regions corresponding to the maximum comprehensive matching degree information in the m comprehensive matching degree information is determined as the n sub-target stopping region information.

[0023] In a possible implementation, the first position information and the second position information are fused to obtain target position information of the multi-rotor unmanned aerial vehicle when stopping on the power exchange platform during power exchange processing, comprising:

[0024] extracting a first deviation between the first position information and the second position information;

[0025] if the first deviation is higher than a preset deviation threshold, determining the second position information as the target position information;

[0026] if the first deviation is lower than the preset deviation threshold, determining position information of a median position between the first position information and the second position information as the target position information.

[0027] In a possible implementation, the method further comprises:

[0028] after determining the target position information, instructing the multi-rotor unmanned aerial vehicle to stop at the position indicated by the target position information, and obtaining a target fitting degree between a battery taking-out area of the multi-rotor unmanned aerial vehicle and the power exchange battery clamping device;

[0029] if the target fitting degree is lower than a preset fitting degree threshold, determining a target stopping pose of the multi-rotor unmanned aerial vehicle according to a current stopping pose of the multi-rotor unmanned aerial vehicle and a clamping pose of the power exchange battery clamping device;

[0030] sending the target stopping pose information to the multi-rotor unmanned aerial vehicle to instruct the multi-rotor unmanned aerial vehicle to adjust the current stopping pose to the target stopping pose.

[0031] A second aspect of the embodiment of the application provides a multi-rotor unmanned aerial vehicle dual-gimbal control device, which is applied to a multi-rotor unmanned aerial vehicle, the multi-rotor unmanned aerial vehicle comprising a first gimbal and a second gimbal, the first gimbal carrying a first camera, the second gimbal carrying a second camera, a shooting resolution of the first camera being lower than that of the second camera, and shooting angles of the first camera and the second camera being different, the device comprising:

[0032] an acquisition unit, configured to acquire a first image set of a power exchange platform of an intelligent power exchange base station by the first gimbal and the first camera and a second image set of the power exchange platform of the intelligent power exchange base station by the second gimbal and the second camera when the multi-rotor unmanned aerial vehicle is performing power exchange processing;

[0033] a first determination unit, configured to determine first stopping position information of the multi-rotor unmanned aerial vehicle on the power exchange platform by the first image set;

[0034] The second determining unit is configured to determine, if the accuracy of the first parking position information is lower than a preset accuracy threshold, second parking position information of the multi-rotor unmanned aerial vehicle on the battery swapping platform by using the second image set.

[0035] The fusion unit is configured to perform fusion processing on the first position information and the second position information to obtain target position information of the multi-rotor unmanned aerial vehicle when parking on the battery swapping platform for battery swapping.

[0036] In one possible implementation, the first determining unit is specifically configured to:

[0037] perform image preprocessing on the first images in the first image set to obtain a first intermediate image set;

[0038] perform battery swapping platform three-dimensional image construction by using the first intermediate images in the first intermediate image set to obtain a reference three-dimensional image;

[0039] extract a parking position in the reference three-dimensional image to obtain reference parking position information;

[0040] perform region division on a reference position corresponding to the reference parking position information to obtain k sub-reference parking regions;

[0041] extract a matching degree between the k sub-reference parking regions and a battery storage region of the multi-rotor unmanned aerial vehicle to obtain k first matching degrees;

[0042] determine n sub-target parking region information from the k sub-reference parking regions according to the k first matching degrees;

[0043] perform fusion processing on the n sub-target parking region information to obtain the first parking position information.

[0044] In one possible implementation, in the aspect of determining the n sub-target parking region information from the k sub-reference parking regions according to the k first matching degrees, the first determining unit is specifically configured to:

[0045] perform parking region matching degree map construction according to the k first matching degrees and the k sub-reference parking regions to obtain a first matching degree map;

[0046] perform sliding window selection from the first matching degree map to obtain m reference regions;

[0047] extract a comprehensive matching degree corresponding to the m reference regions to obtain m comprehensive matching degree information;

[0048] determine, as the n sub-target parking region information, sub-reference parking region information of n reference regions corresponding to maximum comprehensive matching degree information in the m comprehensive matching degree information.

[0049] In a possible implementation, the fusion unit is specifically configured to:

[0050] extract a first deviation degree between the first position information and the second position information;

[0051] if the first deviation degree is higher than a preset deviation threshold, determine the second position information as the target position information;

[0052] if the first deviation degree is lower than the preset deviation threshold, determine position information of a median position between the first position information and the second position information as the target position information.

[0053] In a possible implementation, the apparatus is further configured to:

[0054] after the target position information is determined, instruct the multi-rotor unmanned aerial vehicle to stop at a position indicated by the target position information, and then acquire a target fitting degree between a battery taking-out area of the multi-rotor unmanned aerial vehicle and the battery swapping device;

[0055] if the target fitting degree is lower than a preset fitting threshold, determine a target stopping pose of the multi-rotor unmanned aerial vehicle according to a current stopping pose of the multi-rotor unmanned aerial vehicle and a clamping pose of the battery swapping device;

[0056] send the target stopping pose information to the multi-rotor unmanned aerial vehicle, so as to instruct the multi-rotor unmanned aerial vehicle to adjust the current stopping pose to the target stopping pose.

[0057] A third aspect of the embodiment of the application provides a terminal, including a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are connected with each other, wherein the memory is configured to store a computer program, the computer program includes program instructions, the processor is configured to call the program instructions, and execute the step instructions as in the first aspect of the embodiment of the application.

[0058] A fourth aspect of the embodiment of the application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute part or all of the steps described in the first aspect of the embodiment of the application.

[0059] A fifth aspect of the embodiment of the application provides a computer program product, wherein the computer program product includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps described in the first aspect of the embodiment of the application. The computer program product can be a software installation package.

[0060] Implementing the embodiments of the present application has the following beneficial effects:

[0061] When the multi-rotor unmanned aerial vehicle is performing the battery replacement process, the first image set of the battery replacement platform of the intelligent battery replacement base station is acquired through the control of the first holder and the first camera, and the second image set of the battery replacement platform of the intelligent battery replacement base station is acquired through the control of the second holder and the second camera. The first parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform is determined through the first image set. If the accuracy of the first parking position information is lower than the preset accuracy threshold, the second parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform is determined through the second image set. The first position information and the second position information are fused to obtain the target position information of the multi-rotor unmanned aerial vehicle when parking on the battery replacement platform during the battery replacement process. Therefore, accurate position information can be obtained in combination with the multi-cloud platform, and the accuracy of the target position information acquisition is improved. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0063] Figure 1 A flowchart of a multi-rotor unmanned aerial vehicle double-holder control method is provided for the embodiments of the present application.

[0064] Figure 2 A structure diagram of a terminal is provided for the embodiments of the present application.

[0065] Figure 3 A structure diagram of a multi-rotor unmanned aerial vehicle double-holder control device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0067] The terms "first", "second", and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects, not for describing a particular sequential order. The terms "comprises", "comprising", "includes", "including" and the like are to be construed open- ended, meaning that they include the listed steps or elements, but not excluding other steps or elements. For example, a process, method, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements, but can include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

[0068] Reference to "an embodiment" or "the embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive.

[0069] For better understanding of the method of the multi-rotor unmanned aerial vehicle provided by the embodiments of the present application, the multi-rotor unmanned aerial vehicle using the dual-gimbal control method of the multi-rotor unmanned aerial vehicle is first introduced briefly. The multi-rotor unmanned aerial vehicle comprises a first gimbal and a second gimbal, the first gimbal carries a first camera, the second gimbal carries a second camera, the shooting resolution of the first camera is lower than that of the second camera, and the shooting angles of the first camera and the second camera are different. Specifically, in normal operation, the first gimbal is used to drive the first camera to acquire the information of the parking position, and when the accuracy of the acquired information of the parking position is lower than a preset accuracy threshold, a second image set captured by the second camera is used to assist in determining the target position information. The resolution of the first camera is lower than that of the second camera, and the related volume parameters, weight, cost, etc. of the first camera are lower than those of the second camera. The first camera can be used to acquire the first parking position information in normal positioning, for example, when the ambient light is suitable for image acquisition, and the image quality acquired can be normally used for the acquisition of the first parking position information. The gimbal can drive the second camera, which can be a high-definition camera for executing a aerial photography task, and the resolution of the second image captured by the second camera is much higher than that of the first image captured by the first camera, and the second camera is a backup camera.

[0070] Please refer to Figure 1 , Figure 1 A flowchart of the dual-gimbal control method of the multi-rotor unmanned aerial vehicle is provided for the embodiments of the present application. As shown in Figure 1As shown, the method is applied to a multi-rotor unmanned aerial vehicle, the multi-rotor unmanned aerial vehicle includes a first holder and a second holder, the first holder carries a first camera, the second holder carries a second camera, the shooting resolution of the first camera is lower than that of the second camera, the shooting angles of the first camera and the second camera are different, and the method includes:

[0071] 101、When the multi-rotor unmanned aerial vehicle is performing battery replacement processing, the first image set of the battery replacement platform of the intelligent battery replacement base station is obtained by controlling the first holder and the first camera, and the second image set of the battery replacement platform of the intelligent battery replacement base station is obtained by the second holder and the second camera.

[0072] Wherein, the battery replacement processing can be understood as directly replacing the power battery of the multi-rotor unmanned aerial vehicle. The multi-rotor unmanned aerial vehicle needs to be parked on the battery replacement platform of the intelligent battery replacement base station, and the battery replacement platform has a battery replacement area to correspond to different signals of the multi-rotor unmanned aerial vehicle. In a specific battery replacement area, the area of the battery replacement area will be much larger than the multi-rotor unmanned aerial vehicle, so the final target position information needs to be determined from the battery replacement area.

[0073] The second image set of the battery replacement platform of the intelligent battery replacement base station is obtained by the second holder and the second camera when the accuracy of the first parking position is lower than the preset accuracy threshold. If the accuracy of the first parking position information is greater than or equal to the preset accuracy threshold, the second image in the second image set does not need to be collected.

[0074] 102、Determine the first parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform through the first image set.

[0075] When obtaining the first parking position information, the images in the first image set can be preprocessed to obtain intermediate images, three-dimensional image construction is performed using the intermediate images to obtain reference three-dimensional images, and finally the first parking position information is obtained according to the reference three-dimensional images. By constructing a three-dimensional image, the accuracy of determining the first parking position information can be improved.

[0076] 103、If the accuracy of the first parking position information is lower than the preset accuracy threshold, determine the second parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform through the second image set.

[0077] The preset accuracy threshold is set by experience value or historical data. The accuracy of the first parking position information can be obtained by a common method of obtaining accuracy. For example, the accuracy of the first parking position information is obtained by a historical data comparison method. The reason why the first parking position information is inaccurate can be that, in a foggy weather, the visibility is low, and the image quality of the collected image is not high, so that the positioning is inaccurate. At this time, the second cloud platform can be used to drive the second camera to obtain the second image set.

[0078] 104. fusing the first position information and the second position information to obtain target position information of the multi-rotor unmanned aerial vehicle when parking on the power exchange platform during power exchange processing.

[0079] During the fusion processing, the fusion can be performed according to the deviation between the first position information and the second position information. Specifically, if the first deviation is higher than a preset deviation threshold, the second position information is determined as the target position information; if the first deviation is lower than the preset deviation threshold, the position information of the median position between the first position information and the second position information is determined as the target position information. The first deviation can be understood as the distance between the first position information and the second position information. The greater the distance, the higher the deviation. The smaller the distance, the lower the deviation. The preset deviation threshold is set by experience value or historical data.

[0080] The position information of the median position can be understood as the midpoint region position between the first position indicated by the first position information and the second position indicated by the second position information.

[0081] In the example, when the multi-rotor unmanned aerial vehicle is performing power exchange processing, the first cloud platform and the first camera are controlled to obtain the first image set of the power exchange platform of the intelligent power exchange base station, and the second cloud platform and the second camera are controlled to obtain the second image set of the power exchange platform of the intelligent power exchange base station. The first position information of the multi-rotor unmanned aerial vehicle on the power exchange platform is determined by the first image set. If the accuracy of the first parking position information is lower than a preset accuracy threshold, the second parking position information of the multi-rotor unmanned aerial vehicle on the power exchange platform is determined by the second image set. The first position information and the second position information are fused to obtain the target position information of the multi-rotor unmanned aerial vehicle when parking on the power exchange platform during power exchange processing. Therefore, accurate position information can be obtained by combining the multi-cloud platform, and the accuracy of the target position information obtained is improved.

[0082] In one possible implementation, a method for determining the first parking position information of the multi-rotor unmanned aerial vehicle on the power exchange platform by using the first image set includes:

[0083] A1, image pre-processing is performed on the first images in the first image set to obtain a first intermediate image set;

[0084] A2, a three-dimensional image of the battery swapping platform is constructed using the first intermediate image in the first intermediate image set to obtain a reference three-dimensional image;

[0085] A3, the parking position in the reference three-dimensional image is extracted to obtain reference parking position information;

[0086] A4, the reference position corresponding to the reference parking position information is regionally divided to obtain k sub-reference parking areas;

[0087] A5, the matching degree between the k sub-reference parking areas and the battery storage area of the multi-rotor unmanned aerial vehicle is extracted to obtain k first matching degrees;

[0088] A6, according to the k first matching degrees, n sub-target parking area information is determined from the k sub-reference parking areas;

[0089] A7, the n sub-target parking area information is fused to obtain first parking position information.

[0090] The general pre-processing method can be used for image pre-processing to obtain the first intermediate image set. The pre-processing may, for example, be filtering, image enhancement, etc.

[0091] The general three-dimensional image construction method can be used to construct a three-dimensional image using the first intermediate image in the first intermediate image set to obtain a reference three-dimensional image.

[0092] The reference parking position information can be extracted according to the area identifier of the parking position area. The outline of the parking position can also be extracted, and the area enclosed by the outline is determined as the reference parking position information. In the reference parking position information, it can include multiple sub-areas, and the multi-rotor unmanned aerial vehicle can occupy multiple sub-reference parking areas when parking and perform subsequent battery swapping.

[0093] The matching degree between the sub-reference parking area and the battery storage area can be understood as the convenience of the multi-rotor unmanned aerial vehicle when performing battery replacement. In different areas, the convenience and energy consumption of the corresponding battery replacement battery clamping device moving to the area for battery replacement will be different, so the battery replacement battery clamping device needs to be adapted. The battery replacement battery clamping device can cover multiple sub-areas, so when evaluating the matching degree, the current position of the battery replacement battery clamping device, the battery replacement moving distance, etc. can be evaluated. The longer the distance, the lower the matching degree, and the shorter the distance, the higher the matching degree. However, due to the different poses of the multi-rotor unmanned aerial vehicle after parking, the matching degree between the multi-rotor unmanned aerial vehicle and the battery replacement battery clamping device will also deviate. For example, when parking in a certain pose, the battery replacement battery clamping device needs to detour to the battery storage area for clamping operation and subsequent processing.

[0094] The first matching degree map can be constructed according to the k first matching degrees and the k sub-reference parking areas, and then the sliding window method can be used for selection, and finally the sub-reference parking area information in the reference area with the highest comprehensive matching degree is determined as the sub-target parking area information.

[0095] The position information of the area covered by the n sub-target parking area information can be determined as the first parking position information.

[0096] In one possible implementation, a method for determining n sub-target parking area information from k sub-reference parking areas according to k first matching degrees, comprising:

[0097] B1, constructing a parking area matching degree map according to the k first matching degrees and the k sub-reference parking areas, and obtaining a first matching degree map;

[0098] B2, selecting a sliding window from the first matching degree map to obtain m reference areas;

[0099] B3, extracting the comprehensive matching degrees corresponding to the m reference areas to obtain m comprehensive matching degree information;

[0100] B4, determining the sub-reference parking area information of the n reference areas corresponding to the maximum comprehensive matching degree information in the m comprehensive matching degree information as the n sub-target parking area information.

[0101] The corresponding first matching degree can be marked on the k sub-reference parking areas to obtain the first matching degree map. In the sliding window selection, the shape and size of the sliding window are fixed, and the selection is performed in a predetermined order to obtain m reference areas. The shape of the reference area is the same as the shape of the sliding window.

[0102] The mean and root mean square of the matching degree in the sliding window can be extracted, and the n sub-reference parking area information in the reference area with the minimum root mean square and the maximum mean of the matching degree is determined as the n sub-target parking area information.

[0103] In one possible implementation, the method further includes:

[0104] C1, after determining the target position information, instructing the multi-rotor unmanned aerial vehicle to park at the position indicated by the target position information, and acquiring a target fitting degree between a battery taking-out area of the multi-rotor unmanned aerial vehicle and the battery swapping device;

[0105] C2, if the target fitting degree is lower than a preset fitting degree threshold, determining a target parking pose of the multi-rotor unmanned aerial vehicle according to a current parking pose of the multi-rotor unmanned aerial vehicle and a clamping pose of the battery swapping device;

[0106] C3, sending the target parking pose information to the multi-rotor unmanned aerial vehicle to instruct the multi-rotor unmanned aerial vehicle to adjust the current parking pose to the target parking pose.

[0107] The target fitting degree can be understood as the difficulty of cooperation between the battery area and the battery swapping device for taking out the battery. The higher the difficulty, the lower the fitting degree, and the lower the difficulty, the higher the fitting degree.

[0108] The preset fitting degree threshold is set by an empirical value or historical data. The optimal parking pose can be determined according to the current parking pose and the clamping pose of the battery swapping device, and the target parking pose is obtained. It can be solved by a general optimal parking pose optimization algorithm to obtain the target parking pose.

[0109] For the above embodiments, please refer to Figure 2 , Figure 2 A structure schematic diagram of a terminal provided by the embodiments of the present application, as shown in Figure 2 , including a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are connected with each other, wherein the memory is used for storing a computer program, the computer program includes program instructions, the processor is configured to invoke the program instructions, the above program includes instructions for executing the following steps;

[0110] When the multi-rotor unmanned aerial vehicle is performing battery swapping processing, the first image set of the battery swapping platform of the intelligent battery swapping base station is acquired by controlling the first holder and the first camera, and the second image set of the battery swapping platform of the intelligent battery swapping base station is acquired by controlling the second holder and the second camera;

[0111] determining first parking position information of the multi-rotor unmanned aerial vehicle on the battery swap platform through the first image set;

[0112] if the accuracy of the first parking position information is lower than a preset accuracy threshold, determining second parking position information of the multi-rotor unmanned aerial vehicle on the battery swap platform through the second image set;

[0113] performing fusion processing on the first position information and the second position information to obtain target position information of the multi-rotor unmanned aerial vehicle when parking on the battery swap platform for battery swap processing.

[0114] The above mainly describes the scheme of the embodiments of the present application from the perspective of the method execution process. It can be understood that the terminal includes a hardware structure and / or a software module corresponding to the execution of each function in order to implement the above functions. Those skilled in the art should easily realize that, in combination with the unit and algorithm steps of each example described in the embodiments provided herein, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0115] The embodiments of the present application can divide the functional units of the terminal according to the above method examples. For example, each functional unit can be divided according to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical function division. Actual implementation can have another division method.

[0116] Consistent with the above, please refer to Figure 3 , Figure 3 A structural schematic diagram of a multi-rotor unmanned aerial vehicle dual-gimbal control device is provided in the embodiments of the present application. As shown in Figure 3 , the device is applied to a multi-rotor unmanned aerial vehicle, the multi-rotor unmanned aerial vehicle includes a first gimbal and a second gimbal, the first gimbal carries a first camera, the second gimbal carries a second camera, the shooting resolution of the first camera is lower than that of the second camera, the shooting angles of the first camera and the second camera are different, and the device includes:

[0117] The acquisition unit 301 is configured to acquire, by the first holder and the first camera, a first image set of a battery replacement platform of the intelligent battery replacement base station when the multi-rotor unmanned aerial vehicle is performing battery replacement processing, and acquire, by the second holder and the second camera, a second image set of the battery replacement platform of the intelligent battery replacement base station.

[0118] The first determination unit 302 is configured to determine first parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform by the first image set.

[0119] The second determination unit 303 is configured to determine second parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform by the second image set if the accuracy of the first parking position information is lower than a preset accuracy threshold.

[0120] The fusion unit 304 is configured to perform fusion processing on the first position information and the second position information to obtain target position information of the multi-rotor unmanned aerial vehicle when parking on the battery replacement platform during battery replacement processing.

[0121] In one possible implementation, the first determination unit 302 is specifically configured to:

[0122] perform image preprocessing on the first images in the first image set to obtain a first intermediate image set;

[0123] perform three-dimensional image construction of the battery replacement platform by using the first intermediate images in the first intermediate image set to obtain a reference three-dimensional image;

[0124] extract a parking position in the reference three-dimensional image to obtain reference parking position information;

[0125] perform regional division on a reference position corresponding to the reference parking position information to obtain k sub-reference parking regions;

[0126] extract a matching degree between the k sub-reference parking regions and a battery storage region of the multi-rotor unmanned aerial vehicle to obtain k first matching degrees;

[0127] determine n sub-target parking region information from the k sub-reference parking regions according to the k first matching degrees;

[0128] perform fusion processing on the n sub-target parking region information to obtain the first parking position information.

[0129] In one possible implementation, in terms of determining n sub-target parking region information from the k sub-reference parking regions according to the k first matching degrees, the first determination unit 302 is specifically configured to:

[0130] According to the k first matching degrees and the k sub-reference stopover areas, a stopover area matching degree graph is constructed to obtain a first matching degree graph;

[0131] A sliding window selection is performed from the first matching degree graph to obtain m reference areas;

[0132] An integrated matching degree corresponding to the m reference areas is extracted to obtain m integrated matching degree information;

[0133] The sub-reference stopover area information of the n reference areas corresponding to the maximum integrated matching degree information in the m integrated matching degree information is determined as n sub-target stopover area information.

[0134] In one possible implementation, the fusion unit 304 is specifically configured to:

[0135] A first deviation between the first position information and the second position information is extracted;

[0136] If the first deviation is higher than a preset deviation threshold, the second position information is determined as target position information;

[0137] If the first deviation is lower than the preset deviation threshold, position information of a median position between the first position information and the second position information is determined as target position information.

[0138] In one possible implementation, the apparatus is further configured to:

[0139] After the target position information is determined, the multi-rotor unmanned aerial vehicle is instructed to stop at a position indicated by the target position information, and a target fit degree between a battery taking-out area of the multi-rotor unmanned aerial vehicle and the battery swapping device is obtained;

[0140] If the target fit degree is lower than a preset fit degree threshold, a target stopover pose of the multi-rotor unmanned aerial vehicle is determined according to a current stopover pose of the multi-rotor unmanned aerial vehicle and a clamping pose of the battery swapping device;

[0141] The target stopover pose information is sent to the multi-rotor unmanned aerial vehicle to instruct the multi-rotor unmanned aerial vehicle to adjust the current stopover pose to the target stopover pose.

[0142] Embodiments of the present application also provide a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes a computer to execute part or all steps of any one of the multi-rotor unmanned aerial vehicle dual-gimbal control methods described in the above method embodiments.

[0143] The embodiment of the present application further provides a computer program product, which comprises a non-transitory computer-readable storage medium storing a computer program, and the computer program causes a computer to execute some or all of the steps of any of the multi-rotor unmanned aerial vehicle dual-gimbal control methods described in the above method embodiments.

[0144] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0145] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0146] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical or other forms.

[0147] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0148] In addition, each functional unit in each embodiment of the application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software program module.

[0149] If the integrated unit is realized in the form of a software program module and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0150] A person of ordinary skill in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer readable memory, and the memory can include a flash disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0151] The embodiments of the present application are described in detail above, and specific examples are applied in this paper to describe the principles and implementation modes of the present application. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the embodiments should not be understood as a limitation of the present application.

Claims

1. A dual-gimbal control method for a multi-rotor unmanned aerial vehicle (UAV), characterized in that, The method is applied to a multi-rotor unmanned aerial vehicle, the multi-rotor unmanned aerial vehicle comprises a first holder and a second holder, the first holder carries a first camera, the second holder carries a second camera, a shooting resolution of the first camera is lower than that of the second camera, and shooting angles of the first camera and the second camera are different, and the method comprises: When the multi-rotor unmanned aerial vehicle is subjected to battery replacement processing, a first image set of a battery replacement platform of an intelligent battery replacement base station is acquired through the first holder and the first camera, and a second image set of the battery replacement platform of the intelligent battery replacement base station is acquired through the second holder and the second camera; First parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform is determined through the first image set; If an accuracy of the first parking position information is lower than a preset accuracy threshold, second parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform is determined through the second image set; The first position information and the second position information are subjected to fusion processing to obtain target position information of the multi-rotor unmanned aerial vehicle when parking on the battery replacement platform during battery replacement processing; The first parking position information of the multi-rotor unmanned aerial vehicle on the battery replacement platform is determined through the first image set, comprising: First images in the first image set are subjected to filtering and image enhancement processing to obtain a first intermediate image set; A reference three-dimensional image is obtained through three-dimensional image construction of the battery replacement platform by using first intermediate images in the first intermediate image set; A reference parking position is extracted from the reference three-dimensional image to obtain reference parking position information; A reference position corresponding to the reference parking position information is subjected to regional division to obtain k sub-reference parking regions; Matching degrees between the k sub-reference parking regions and a battery storage region of the multi-rotor unmanned aerial vehicle are extracted to obtain k first matching degrees; n sub-target parking region information is determined from the k sub-reference parking regions according to the k first matching degrees; The n sub-target parking region information is subjected to fusion processing to obtain the first parking position information; The n sub-target parking region information is determined from the k sub-reference parking regions according to the k first matching degrees, comprising: A first matching degree map is obtained through parking region matching degree map construction according to the k first matching degrees and the k sub-reference parking regions; m reference regions are obtained through sliding window selection from the first matching degree map; m comprehensive matching degree information is obtained by extracting comprehensive matching degrees corresponding to the m reference regions; n reference region sub-reference parking region information corresponding to maximum comprehensive matching degree information in the m comprehensive matching degree information is determined as the n sub-target parking region information; The first position information and the second position information are subjected to fusion processing to obtain target position information of the multi-rotor unmanned aerial vehicle when parking on the battery replacement platform during battery replacement processing, comprising: A first deviation between the first position information and the second position information is extracted; If the first deviation is higher than a preset deviation threshold, the second position information is determined as the target position information. If the first deviation degree is lower than a preset deviation degree threshold, position information of a median position between the first position information and the second position information is determined as target position information.

2. The dual-gimbal control method for a multi-copter drone of claim 1, wherein, The method further comprises: After the target position information is determined, the multi-rotor unmanned aerial vehicle is instructed to stop at the position indicated by the target position information, and then a target fitting degree between a battery taking-out area of the multi-rotor unmanned aerial vehicle and the battery swapping device is obtained. If the target fitting degree is lower than a preset fitting degree threshold, a target stopping pose of the multi-rotor unmanned aerial vehicle is determined according to a current stopping pose of the multi-rotor unmanned aerial vehicle and a clamping pose of the battery swapping device. The target stopping pose information is sent to the multi-rotor unmanned aerial vehicle to instruct the multi-rotor unmanned aerial vehicle to adjust the current stopping pose to the target stopping pose.

3. A multi-rotor unmanned aerial vehicle dual-gimbal control device, characterized in that, The device is applied to a multi-rotor unmanned aerial vehicle, the multi-rotor unmanned aerial vehicle comprising a first gimbal and a second gimbal, the first gimbal carrying a first camera, and the second gimbal carrying a second camera, the first camera having a lower shooting resolution than the second camera, and the first camera and the second camera having different shooting angles, the device comprising: An acquisition unit is configured to acquire, when the multi-rotor unmanned aerial vehicle is undergoing battery swapping, a first image set of a battery swapping platform of an intelligent battery swapping station by controlling the first gimbal and the first camera, and a second image set of the battery swapping platform of the intelligent battery swapping station by the second gimbal and the second camera; A first determination unit is configured to determine first stopping position information of the multi-rotor unmanned aerial vehicle on the battery swapping platform by the first image set; A second determination unit is configured to determine second stopping position information of the multi-rotor unmanned aerial vehicle on the battery swapping platform by the second image set if an accuracy of the first stopping position information is lower than a preset accuracy threshold; A fusion unit is configured to perform fusion processing on the first position information and the second position information to obtain target position information of the multi-rotor unmanned aerial vehicle when stopping on the battery swapping platform during battery swapping; The first determination unit is specifically configured to: perform filtering and image enhancement processing on the first images in the first image set to obtain a first intermediate image set; perform three-dimensional image construction of the battery swapping platform by using the first intermediate images in the first intermediate image set to obtain a reference three-dimensional image; extract a stopping position in the reference three-dimensional image to obtain reference stopping position information; perform region division on a reference position corresponding to the reference stopping position information to obtain k sub-reference stopping regions; extract matching degrees between the k sub-reference stopping regions and a battery storage area of the multi-rotor unmanned aerial vehicle to obtain k first matching degrees; determine n sub-target stopping region information from the k sub-reference stopping regions according to the k first matching degrees; perform fusion processing on the n sub-target stopping region information to obtain the first stopping position information; In terms of determining the n sub-target stopping region information from the k sub-reference stopping regions according to the k first matching degrees, the first determination unit is specifically configured to: According to the k first matching degrees and the k sub reference stop regions, a stop region matching degree map is constructed to obtain a first matching degree map; A sliding window is selected from the first matching degree map to obtain m reference regions; An integrated matching degree corresponding to the m reference regions is extracted to obtain m integrated matching degree information; The sub reference stop region information of the n reference regions corresponding to the maximum integrated matching degree information in the m integrated matching degree information is determined as n sub target stop region information; The first position information and the second position information are fused to obtain target position information of the multi-rotor unmanned aerial vehicle when stopping on the power exchange platform during power exchange processing, including: A first deviation between the first position information and the second position information is extracted; If the first deviation is higher than a preset deviation threshold, the second position information is determined as the target position information; If the first deviation is lower than the preset deviation threshold, the position information of a median position between the first position information and the second position information is determined as the target position information.

4. A terminal, characterized by comprising: The computer readable storage medium stores a computer program, and the computer program includes program instructions. The program instructions, when executed by a processor, cause the processor to perform the multi-rotor unmanned aerial vehicle double-gimbal control method according to any one of claims 1-2.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program includes program instructions. The program instructions, when executed by a processor, cause the processor to perform the multi-rotor unmanned aerial vehicle double-gimbal control method according to any one of claims 1-2.

Images