Method and system for improving visual precise landing effect of unmanned aerial vehicle, and electronic equipment

Abstract

The invention relates to the technical field of unmanned aerial vehicle visual precise landing scheme design, in particular to a method and system for improving the visual precise landing effect of an unmanned aerial vehicle and electronic equipment. In the landing process of the unmanned aerial vehicle, the camera arranged on the unmanned aerial vehicle is used for obtaining the target image containing the target positioning code arranged on the hangar, the target image is processed, the shadow of the target image is removed, the positioning code in the target image is recognized, and then the target position information contained in the target positioning code is obtained. And controlling the unmanned aerial vehicle to land by using the target position information. The technical problem of low identification efficiency caused by interference on identification of the positioning code due to the fact that the shadow of the unmanned aerial vehicle body is easily projected to the hangar platform and overlapped with the positioning code under the irradiation of sunlight is solved, the identification efficiency of the positioning code is greatly improved, and the effect and the intelligent degree of visual precise landing of the unmanned aerial vehicle are improved to a great extent.

Description

Technical Field

[0001] The present invention relates to the technical field of drone vision precise landing scheme design, and particularly to a method and system for improving the drone vision precise landing effect, and an electronic device. Background Art

[0002] For hangar drones, they often need to land very precisely on the hangar. The currently common technical implementation method is to paste positioning codes on the hangar platform. During the drone landing process, the positioning codes are recognized through camera vision, and according to the relative position between the drone and the positioning codes, the position of the drone relative to the hangar is calculated, and the position of the drone is dynamically adjusted, so as to achieve precise landing on the hangar platform.

[0003] The biggest problem with vision landing is that the recognition effect is easily affected by sunlight. Especially when the drone is landing, under the illumination of sunlight, the body shadow of the drone is easily projected onto the hangar platform and overlaps with the positioning codes, thus interfering with the recognition of the positioning codes and resulting in low recognition efficiency. The existing technology urgently needs a solution to improve the drone vision precise landing effect to solve the problems existing in the existing technology.

[0004] Therefore, the existing technology still needs to be further developed. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above technical deficiencies, and provide a method and system for improving the drone vision precise landing effect, and an electronic device, so as to solve the problems existing in the existing technology.

[0006] To achieve the above technical purpose, according to the first aspect of the present invention, the present invention provides a method for improving the drone vision precise landing effect, and the method includes: S100. During the landing process of the drone, use the camera set on the drone to obtain a target image including the target positioning code set on the hangar; S200. Process the target image to remove the shadow of the target image; S300. Recognize the positioning code in the target image, and then obtain the target position information included in the target positioning code, and use the target position information to control the landing of the drone.

[0007] Specifically, the processing of the target image to remove the shadow in the target image includes: S210. Convert the target image from RGB to a color space, and then separate the illumination and color information of the target image; S210. Detect the target image through a preset threshold to determine whether there is a shadow area in the target image; S220. If it is determined that there is a shadow area in the target image, locally smooth the brightness of the shadow area in the target image, so that the brightness of the shadow area gradually smooths to the brightness of the surrounding area adjacent to the shadow area.

[0008] Specifically, detecting the target image through a preset threshold includes: Using the lowerBound function and upperBound in the opencv library to define the preset upper boundary and preset lower boundary of the color range of the shadow area of the target image, and then detecting the target image using the preset upper boundary and preset lower boundary of the color range, positioning the pixels located between the preset upper boundary and preset lower boundary as shadow pixels, and positioning the set of all shadow pixels as the shadow area.

[0009] Specifically, locally smoothing the brightness of the shadow area in the target image, so that the brightness of the shadow area gradually smooths to the brightness of the surrounding area adjacent to the shadow area, includes: Using Gaussian blur to estimate the illumination distribution of the target image, and then separating the reflection component and illumination component of the target image.

[0010] Specifically, the using Gaussian blur to estimate the illumination distribution of the target image and then separating the reflection component and illumination component of the target image includes: Converting the target image into a grayscale image, smoothing the grayscale image through Gaussian filtering to estimate the illumination distribution of the image, obtaining the Gaussian-blurred image, performing Retinex processing on the grayscale image and the Gaussian-blurred image to obtain the enhanced target image, and converting the enhanced target image back to the color format.

[0011] Specifically, locally smoothing the brightness of the shadow area in the target image, so that the brightness of the shadow area gradually smooths to the brightness of the surrounding area adjacent to the shadow area, includes: Creating a Gaussian kernel of a preset size, using the created Gaussian kernel to locally smooth the brightness of the shadow area in the enhanced target image, and replacing the shadow area in the enhanced target image with the result of the local smoothing adjustment.

[0012] Specifically, the using the created Gaussian kernel to locally smooth the brightness of the shadow area in the enhanced target image includes: Performing a convolution operation on the enhanced target image using the filter2D function of OpenCV, and the convolution kernel of the convolution operation is the created Gaussian kernel of the preset size.

[0013] Specifically, the process of performing Retinex processing on the grayscale image and the Gaussian-blurred image to obtain the enhanced target image includes: Subtract the grayscale image from the Gaussian-blurred image to obtain the enhanced target image.

[0014] According to a second aspect of the present invention, there is provided a system for improving the visual precise landing effect of an unmanned aerial vehicle (UAV), including: An acquisition module, configured to, during the landing process of the UAV, use a camera disposed on the UAV to acquire a target image including a target positioning code set on a hangar; A control module, configured to process the target image to remove the shadow of the target image; identify the positioning code in the target image, and further obtain the target position information included in the target positioning code, and use the target position information to control the landing of the UAV.

[0015] According to a third aspect of the present invention, there is provided an electronic device, including: a memory; and a processor, where computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, the method for improving the visual precise landing effect of the UAV as described above is implemented.

[0016] Beneficial effects: During the landing process of the UAV, the present invention uses a camera disposed on the UAV to acquire a target image including a target positioning code set on a hangar, processes the target image to remove the shadow of the target image, identifies the positioning code in the target image, and further obtains the target position information included in the target positioning code, and uses the target position information to control the landing of the UAV. This solves the technical problem that when the UAV body is irradiated by sunlight, the body shadow is easily projected onto the hangar platform and overlaps with the positioning code, thereby interfering with the recognition of the positioning code and resulting in low recognition efficiency. It greatly improves the recognition efficiency of the positioning code and significantly improves the visual precise landing effect and the degree of intelligence of the UAV. Description of the drawings

[0017] Figure 1 is a schematic flowchart of the method for improving the visual precise landing effect of the UAV provided in a specific embodiment of the present invention; Figure 2 is a schematic diagram of the system composition of the system for improving the visual precise landing effect of the UAV provided in a specific embodiment of the present invention. Detailed implementation manners

[0018] To enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Based on the embodiments in this application, other similar embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application. In addition, the directional terms mentioned in the following embodiments, such as "upper", "lower", "left", "right", etc., are only references to the directions in the accompanying drawings. Therefore, the directional terms used are for illustration rather than limiting the present invention.

[0019] The present invention will be further described below in conjunction with the accompanying drawings and preferred embodiments.

[0020] Please refer to Figure 1 , the present invention provides a method for improving the visual precise landing effect of an unmanned aerial vehicle, including: S100. During the landing process of the unmanned aerial vehicle, a camera disposed on the unmanned aerial vehicle is used to obtain a target image including a target positioning code set on the hangar.

[0021] S200. Process the target image to remove the shadow of the target image.

[0022] Specifically, the process of processing the target image to remove the shadow in the target image includes: S210. Convert the target image from RGB to a color space, and then separate the illumination and color information of the target image.

[0023] Specifically, the color space is the HSV or Lab color space.

[0024] S210. Detect the target image through a preset threshold to determine whether there is a shadow area in the target image.

[0025] Specifically, detecting the target image through a preset threshold includes: Use the lowerBound function and upperBound in the opencv library to define the preset upper boundary and preset lower boundary of the color range of the shadow area of the target image, and then use the preset upper boundary and preset lower boundary of the color range to detect the target image, locate the pixels between the preset upper boundary and preset lower boundary as shadow pixels, and locate the set of all shadow pixels as the shadow area.

[0026] Specifically, the software code for the above process is as follows: fun detectShadows(hsvImage: Mat): Mat { val lowerBound = Scalar(0.0, 0.0, 0.0) val upperBound = Scalar(180.0, 255.0, 100.0) val shadowMask = Mat() Core.inRange(hsvImage, lowerBound, upperBound, shadowMask) return shadowMask } It can be understood that the above code realizes screening out shadow pixels in the target image whose hue H is in the range of 0 - 180, saturation S is close to the maximum value of 255, and brightness V is less than or equal to 100.

[0027] S220. If it is determined that there is a shadow area in the target image, locally smooth the brightness of the shadow area in the target image, so that the brightness of the shadow area gradually smooths to the brightness of the surrounding area adjacent to the shadow area.

[0028] Specifically, locally smoothing the brightness of the shadow area in the target image, so that the brightness of the shadow area gradually smooths to the brightness of the surrounding area adjacent to the shadow area, includes: Estimate the illumination distribution of the target image using Gaussian blur, and then separate the reflection component and illumination component of the target image.

[0029] Specifically, the process of estimating the illumination distribution of the target image using Gaussian blur and then separating the reflection component and illumination component of the target image includes: Convert the target image to a grayscale image, perform smoothing processing on the grayscale image through Gaussian filtering to estimate the illumination distribution of the image, obtain the Gaussian-blurred image, perform Retinex processing on the grayscale image and the Gaussian-blurred image to obtain the enhanced target image, and convert the enhanced target image back to the color format.

[0030] Specifically, the code for the above process is as follows; fun applyRetinex(image: Mat): Mat { val grayImage = Mat() Imgproc.cvtColor(image, grayImage, Imgproc.COLOR_BGR2GRAY) val gaussianBlurred = Mat() Imgproc.GaussianBlur(grayImage, gaussianBlurred, Size(15.0,15.0), 0.0) val retinexImage = Mat() Core.subtract(grayImage, gaussianBlurred, retinexImage) val resultImage = Mat() Imgproc.cvtColor(retinexImage, resultImage, Imgproc.COLOR_GRAY2BGR) return resultImage } It can be understood that the above code implements a simplified Retinex image enhancement algorithm based on Gaussian filtering. The main steps include: 1. Grayscale conversion: Convert the color image to a grayscale image for simplified processing.

[0031] 2. Illumination estimation: Estimate the illumination distribution of the image through Gaussian blur.

[0032] 3. Retinex processing: Enhance the details and contrast of the image by subtracting the illumination estimation.

[0033] 4. Color restoration: Convert the processed grayscale image back to the color format for easy display.

[0034] Specifically, the local smoothing adjustment of the brightness of the shadow area in the target image, so that the brightness of the shadow area gradually smooths to the brightness of the surrounding area adjacent to the shadow area, includes: Create a Gaussian kernel of a preset size, and use the created Gaussian kernel to perform local smoothing adjustment on the brightness of the shadow area in the enhanced target image, and replace the shadow area in the enhanced target image with the result of the smoothing adjustment.

[0035] Specifically, the use of the created Gaussian kernel to perform local smoothing adjustment on the brightness of the shadow area in the enhanced target image includes: Use the filter2D function of OpenCV to perform a convolution operation on the enhanced target image, and the convolution kernel of the convolution operation is the created Gaussian kernel of the preset size.

[0036] Specifically, the Retinex processing of the grayscale image and the Gaussian-blurred image to obtain the enhanced target image includes: Subtract the grayscale image from the Gaussian-blurred image to obtain the enhanced target image.

[0037] Specifically, the code for the above process is as follows: fun adjustLocalBrightness(image: Mat, shadowMask: Mat): Mat { val shadowAdjusted = image.clone() val kernel = Mat.ones(15, 15, CvType.CV_32F) / 225.0 val blurredImage = Mat() Imgproc.filter2D(image, blurredImage, -1, kernel) for (i in 0 until shadowMask.rows()) { for (j in 0 until shadowMask.cols()) { if (shadowMask.get(i, j)[0]>0) { shadowAdjusted.put(i,j, blurredImage.get(i, j)) } } } return shadowAdjusted } It can be understood that the function of the above code is to perform brightness smoothing on the shadow areas in the image. The specific steps are as follows: 1. Clone the input image.

[0038] 2. Create a Gaussian kernel with a size of 15 * 15 for smoothing.

[0039] 3. Smooth the input image to obtain the blurred image.

[0040] 4. Traverse the shadow mask image to find the shadow areas and replace the pixels in these areas with the corresponding pixels of the smoothed image.

[0041] 5. Return the processed image.

[0042] It should be noted here that all the code in the present invention is written in Kotlin.

[0043] S300. Identify the positioning code in the target image, and then obtain the target position information contained in the target positioning code, and use the target position information to control the landing of the drone.

[0044] It can be understood that in the process of the drone landing, the present invention uses a camera disposed on the drone to obtain a target image including the target positioning code set on the hangar, processes the target image, removes the shadow of the target image, identifies the positioning code in the target image, and then obtains the target position information contained in the target positioning code, and uses the target position information to control the landing of the drone. It solves the technical problem that when the drone body is irradiated by sunlight, the body shadow is easily projected onto the hangar platform and overlaps with the positioning code, thereby interfering with the identification of the positioning code and resulting in low identification efficiency. It greatly improves the positioning code identification efficiency and greatly improves the effect and intelligence level of the drone's visual precise landing.

[0045] Please refer to Figure 2 , the present invention provides another embodiment. The present embodiment provides a system for improving the visual precise landing effect of a drone. The system for improving the visual precise landing effect of the drone includes: An acquisition module 100, configured to use a camera disposed on the drone to obtain a target image including a target positioning code set on the hangar during the landing process of the drone; A control module 200, configured to process the target image to remove the shadow of the target image; configured to identify the positioning code in the target image, and then obtain the target position information contained in the target positioning code, and use the target position information to control the landing of the drone.

[0046] Here, it should be noted that in the process of the drone landing, the present invention uses a camera disposed on the drone to obtain a target image including the target positioning code set on the hangar, processes the target image, removes the shadow of the target image, identifies the positioning code in the target image, and then obtains the target position information contained in the target positioning code, and uses the target position information to control the landing of the drone. It solves the technical problem that when the drone body is irradiated by sunlight, the body shadow is easily projected onto the hangar platform and overlaps with the positioning code, thereby interfering with the identification of the positioning code and resulting in low identification efficiency. It greatly improves the positioning code identification efficiency and greatly improves the effect and intelligence level of the drone's visual precise landing.

[0047] In a preferred embodiment, the present application further provides an electronic device, and the electronic device includes: A memory; and a processor, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, the method for improving the visual precise landing effect of the unmanned aerial vehicle is implemented. The computer device can be generally a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, a memory, a network interface, a communication interface, etc. connected through a system bus. The processor of the computer device can be used to provide necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory can provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and the communication interface of the computer device can be used to connect and communicate with external devices through a network. When the computer program is executed by the processor, the steps of the method of the present invention are executed.

[0048] The present invention can be implemented as a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method of the embodiments of the present invention are caused to be executed. In one embodiment, the computer program is distributed on a plurality of network-coupled computer devices or processors, so that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, can be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations can be executed by one or more computer devices or processors, and one or more other method steps / operations can be executed by one or more other computer devices or processors. One or more computer devices or processors can execute a single method step / operation, or execute two or more method steps / operations.

[0049] Those of ordinary skill in the art can understand that the method steps of the present invention can be instructed by a computer program to complete the relevant hardware such as a computer device or a processor. The computer program can be stored in a non-transitory computer-readable storage medium, and when the computer program is executed, the steps of the present invention are caused to be executed. Depending on the situation, any reference to a memory, a storage, a database, or other medium in this article may include non-volatile and / or volatile memories. Examples of non-volatile memories include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memories include random access memory (RAM), external cache memory, etc.

[0050] It is understandable that in the process of the UAV landing, the present invention uses a camera provided on the UAV to obtain a target image including a target positioning code set on the hangar, processes the target image, eliminates the shadow of the target image, identifies the positioning code in the target image, and further obtains the target position information included in the target positioning code, and uses the target position information to control the UAV to land. This solves the technical problem that when the UAV body is irradiated by sunlight, the body shadow is easily projected onto the hangar platform and overlaps with the positioning code, thus interfering with the recognition of the positioning code and resulting in low recognition efficiency. It greatly improves the positioning code recognition efficiency and largely improves the effect and intelligence level of the UAV's visual precise landing.

[0051] The above-described technical features can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification as long as such a combination is not contradictory.

[0052] The specific embodiments of the present invention described above do not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and deformations made according to the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for improving the visual precision landing effect of a drone, characterized in that: The method comprises: S100, during the landing process of the UAV, using a camera disposed on the UAV to obtain a target image including a target positioning code disposed on the hangar; S200, processing the target image to remove the shadow of the target image; S300, identifying the positioning code in the target image, and then obtaining the target position information contained in the target positioning code, and using the target position information to control the landing of the UAV.

2. The method for improving the visual precision landing effect of a drone according to claim 1, characterized in that: The step of processing the target image to remove shadows in the target image includes: S210, converting the target image from RGB to a color space, thereby separating illumination and color information of the target image; S210, detecting the target image by using a preset threshold to determine whether there is a shadow area in the target image; S220: If it is determined that there is a shadow area in the target image, locally smooth the brightness of the shadow area in the target image, so that the brightness of the shadow area is gradually smoothed to the brightness of the surrounding area adjacent to the shadow area.

3. The method for improving the visual precision landing effect of a drone according to claim 2, characterized in that: Detect the target image through the preset threshold, including: The lowerBound function and upperBound in the opencv library are used to define the preset upper boundary and preset lower boundary of the color range of the shadow area of ​​the target image, and then the preset upper boundary and preset lower boundary of the color range are used to detect the target image, and the pixels between the preset upper boundary and the preset lower boundary are located as shadow pixels, and the set of all shadow pixels is located as the shadow area.

4. The method for improving the visual precision landing effect of a drone according to claim 2, characterized in that: The brightness of the shadow area in the target image is locally and smoothly adjusted, so that the brightness of the shadow area is gradually smoothed to the brightness of the surrounding area adjacent to the shadow area, including: Gaussian blur is used to estimate the illumination distribution of the target image, and then the reflection component and illumination component of the target image are separated.

5. The method for improving the visual precision landing effect of a drone according to claim 4, characterized in that: The method of estimating the illumination distribution of the target image by using Gaussian blur, and then separating the reflection component and the illumination component of the target image, comprises: The target image is converted into a grayscale image, the grayscale image is smoothed by Gaussian filtering to estimate the illumination distribution of the image, and a Gaussian blurred image is obtained. The grayscale image and the Gaussian blurred image are retinexed to obtain an enhanced target image, and the enhanced target image is converted back to a color format.

6. The method for improving the visual precision landing effect of a drone according to claim 5, characterized in that: The locally smoothing adjustment of the brightness of the shadow area in the target image, so as to gradually smooth the brightness of the shadow area to the brightness of the surrounding area adjacent to the shadow area, includes: A Gaussian kernel of a preset size is created, and the brightness of the shadow area in the enhanced target image is locally smoothly adjusted by using the created Gaussian kernel, and the shadow area in the enhanced target image is replaced by the result of the smooth adjustment.

7. The method for improving the visual precision landing effect of a drone according to claim 6, characterized in that: The method of using the created Gaussian kernel to locally and smoothly adjust the brightness of the shadow area in the enhanced target image includes: The filter2D function of OpenCV is used to perform a convolution operation on the enhanced target image. The convolution kernel of the convolution operation is the created Gaussian kernel of the preset size.

8. The method for improving the visual precision landing effect of a drone according to claim 5, characterized in that: The grayscale image and the Gaussian blurred image are subjected to Retinex processing to obtain an enhanced target image, including: Subtract the grayscale image from the Gaussian blurred image to obtain the enhanced target image.

9. A system for improving the visual precision landing effect of a drone, characterized in that: include: An acquisition module is used to acquire a target image including a target positioning code set on the hangar by using a camera set on the drone during the landing process of the drone; A control module, used for processing the target image and removing the shadow of the target image; It is used to identify the positioning code in the target image, and then obtain the target position information contained in the target positioning code, and use the target position information to control the landing of the UAV.

10. An electronic device, characterized in that: include: Memory; and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the method for improving the visual precision landing effect of a drone according to any one of claims 1 to 8 is implemented.

Images