Unmanned aerial vehicle light detection method based on machine vision and big data

Abstract

The invention discloses an unmanned aerial vehicle light detection method based on machine vision and big data. The method comprises the steps that S1, video collection is carried out through a video collection module; s2, carrying out image acquisition and processing through an image acquisition module; s3, performing light feature extraction through an image processing module, wherein the light feature extraction comprises brightness feature extraction and color feature extraction; s4, light analysis is carried out through a calculation and analysis module; and S5, outputting a result, and outputting a detection result of the navigation light of the tested unmanned aerial vehicle according to a light analysis result. According to the unmanned aerial vehicle light detection method based on the machine vision and the big data, the night flight video of the unmanned aerial vehicle is collected through the camera simulating the human eyes, the night flight video is converted into the digital image through the image processing module, and navigation light light is analyzed and judged through a computer vision algorithm and big data analysis; and reliable detection of the unmanned aerial vehicle navigation light is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) testing, and in particular relates to a UAV light detection method based on machine vision and big data. Background Art

[0002] Lighting testing is one of the inspection items for civil drones. The current common testing method, based on Section 5.15 of GB 42590-2023, "Safety Requirements for Civilian Unmanned Aircraft Systems," involves flying the drone at a relative distance of 120 meters from the inspector at night (one hour after sunset and one hour before sunrise) with a visibility range of 10 to 20 kilometers. The inspector then observes the navigation lights with the naked eye at angles of 30°, 45°, and 60° to determine their color.

[0003] The problem with this testing method is that different people have different visual conditions, including vision, field of view, and color discrimination ability. Therefore, differences in visual conditions and subjective factors may lead to controversial test results. It is impossible to accurately and objectively determine the visibility and color of drone navigation lights at night.

[0004] Therefore, further improvements are made to the above problems. Summary of the Invention

[0005] The main purpose of this invention is to provide a drone light detection method based on machine vision and big data. The drone night flight video is collected by a camera that simulates the human eye, and converted into a digital image using an image processing module. The navigation light is analyzed and judged through computer vision algorithms and big data analysis, thereby realizing reliable detection of drone navigation lights.

[0006] To achieve the above objectives, the present invention provides a method for detecting UAV lights based on machine vision and big data, comprising the following steps:

[0007] Step S1: Capture video using a video acquisition module, using a binocular camera to shoot the drone at night, and obtain a video of the drone at a preset distance and angle;

[0008] Step S2: performing image acquisition and processing through the image acquisition module, processing the acquired video, including frame extraction, grayscale conversion and image smoothing;

[0009] Step S3: Light feature extraction is performed using an image processing module, including brightness feature extraction and color feature extraction. A threshold segmentation algorithm is used to accurately extract the light area in the grayscale image. A color space conversion algorithm is used to convert the color image obtained by the binocular camera from the first color space to the second color space, thereby obtaining the color features of the navigation lights.

[0010] Step S4: Performing lighting analysis through the calculation and analysis module and further analyzing the extracted color features, including visibility judgment and color matching;

[0011] Step S5: Output the results. According to the light analysis results, output the detection results of the navigation lights of the tested UAV.

[0012] As a further preferred technical solution of the above technical solution, for step S1:

[0013] A binocular camera simulating the human eye was used to film the test drone at night when visibility met the standard. The camera remained stationary and hovered 120 meters away from the camera and focused at angles of 30°, 45°, and 60°, respectively. Each of the three angles was filmed for 10 seconds and the videos were saved.

[0014] As a further preferred technical solution of the above technical solution, step S2 is specifically implemented as follows:

[0015] Step S2.1: extract each frame image from each video and save it as the image to be processed;

[0016] Step S2.2: Convert the extracted color image into a grayscale image f1(x,y), i.e., a matrix with pixels as elements, where n*m represents the number of pixels in the image. Grayscale images contain only brightness information, not color information. Subsequently, an algorithm is used to calculate the visibility of different colored lights. The RGB values directly obtained by the camera are converted to grayscale values using a floating-point algorithm, i.e., Grey = 0.299*R + 0.587*G + 0.114*B.

[0017] Step S2.3: Image smoothing and denoising of the grayscale image.

[0018] As a further preferred technical solution of the above technical solution, step S3 is specifically implemented as the following steps:

[0019] Step S3.1: Apply the threshold segmentation algorithm to the matrix f1(x,y) to extract the light area f2 and the remaining dark background area f3. The specific method is to set the grayscale threshold as t, and the entire grayscale image is divided into two parts [0, t] and [t, 255]. The grayscale means of the parts [0, t] and [t, 255] are m1 and m2 respectively. The grayscale mean of the entire image is mg. The probability of a pixel being divided into [0, t] and [t, 255] are p1 and p2 respectively. Let n i is the total number of pixels with grayscale i in the image, and the probability that a pixel in the image has grayscale i is: And there is have to:

[0020]

[0021] Solve the between-class variance The maximized gray level k is the maximum inter-class variance threshold t;

[0022] Step 3.2: Since grayscale images do not contain color information, color feature extraction is required for the RGB image obtained by the camera. First, the light area obtained after brightness feature extraction is used to extract the pixel points in the RGB image. Then, a color space conversion algorithm is used to convert the color image from the RGB color space to the CIE1931 XYZ color space. The X, Y, and Z tristimulus values of each pixel are obtained and calculated using the matrix:

[0023] As a further preferred technical solution of the above technical solution, step S4 is specifically implemented as the following steps:

[0024] Step S4.1: Determine visibility. The visibility of the drone's navigation lights needs to consider four factors: light source size, brightness, contrast, and atmospheric visibility. To determine whether it can be observed at night, the dimensional analysis method is used to calculate light visibility. The specific implementation is as follows:

[0025] Step S4.1.1: Calculate the number a of pixels in the extracted light area f2;

[0026] Step S4.1.2: Calculate the average brightness of the light area sum is the sum of the grayscale values of each pixel in the light area;

[0027] Step S4.1.3: Calculate the contrast between the light and the dark background. Use the RGB image to calculate the contrast. First, calculate the contrast of all pixels in the f3 area. Then calculate the f2 region

[0028] Contrast

[0029] Step S4.1.4: Use a visibility meter to measure the atmospheric visibility distance l and calculate the visibility

[0030] Step S4.1.5: Calculate the visibility of the drone’s navigation lights for each image Take the maximum value V among the three angles max ;

[0031] Step S4.1.6: Accurately determine the visibility threshold V0 of the drone’s navigation lights through big data analysis. The specific method is:

[0032] Step S4.1.6.1: Collect the results of different UAV lighting detection projects. Use the same model of binocular camera to collect the V of each UAV using the above steps S1 to S4.1.5. max value;

[0033] Step S4.1.6.2: Delineate the visible and invisible areas based on the test results. The area above the red line is the visible area, and the area below the red line is the invisible area. The V max The minimum value is V max,yes,min , V in the visible region max The average value is V in the invisible area max The maximum value is V max,no,max , V in the invisible area max The average value is

[0034] Step S4.1.7: When the visibility of the navigation lights of a certain type of UAV is V max ≥V0, it is judged as a navigation light visible to the human eye, otherwise it is judged as a navigation light invisible to the human eye;

[0035] Step S4.2: Perform color matching and calculate the X, Y, and Z values of each pixel in the light area obtained in step S3.2. And converted into color coordinates, the conversion method is Accurately display the color of navigation lights.

[0036] As a further preferred technical solution of the above technical solution, for step S5, the light detection results are exported and printed in color, and the information includes the test angles and photos corresponding to each angle, the CIE1931 chromaticity diagram with the navigation light color coordinate points, and the calculated visibility V max and V0 value, as well as visibility judgment result visible or invisible.

[0037] The beneficial effects of the present invention are:

[0038] 1. A binocular camera that simulates the human eye is used instead of direct observation with the naked eye, eliminating the influence of differences in vision between people and subjective factors. This is beneficial to protecting the eyesight of inspectors during high-intensity inspection operations.

[0039] 2. Accurately determine the visibility threshold V0 of drone navigation lights based on big data analysis, and eliminate disputes over light visibility caused by abnormal vision or subjective factors of inspectors in subsequent drone navigation light inspection projects.

[0040] 3. Analyze the navigation lights using machine vision algorithms, converting optical signals to electrical signals and then to digital signals. The visibility V of the drone's navigation lights is calculated based on factors such as light source size, brightness, contrast, and atmospheric visibility. This value is then compared with the threshold V0 to determine the visibility of the drone's navigation lights. The RGB color space is converted to the CIE1931 XYZ color space and then to color coordinates, accurately reflecting the navigation light color in the CIE1931 chromaticity diagram.

[0041] 4. During the test, test photos, images and other materials can be retained for reference, which can avoid disputes over test results due to the test personnel forgetting to retain test materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic flow diagram of the present invention.

[0043] Figure 2 It is the RGB color space to grayscale image matrix conversion diagram of the present invention.

[0044] Figure 3 This is a classification diagram of the visibility test results of the UAV navigation lights of the present invention.

[0045] Figure 4 It is a schematic diagram of the result of the color coordinates of the present invention being reflected in the CIE 1931 chromaticity diagram. DETAILED DESCRIPTION

[0046] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are for illustrative purposes only, and those skilled in the art will readily appreciate other obvious variations. The basic principles of the present invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

[0047] In the preferred embodiment of the present invention, those skilled in the art should note that the test drone and the like involved in the present invention may be regarded as prior art.

[0048] Preferred embodiment.

[0049] like Figure 1-4 As shown, the present invention discloses a method for detecting UAV lights based on machine vision and big data, comprising the following steps:

[0050] Step S1: Capture video using a video acquisition module, using a binocular camera to shoot the drone at night, and obtain a video of the drone at a preset distance and angle;

[0051] Step S2: performing image acquisition and processing through the image acquisition module, processing the acquired video, including frame extraction, grayscale conversion and image smoothing;

[0052] Step S3: Light feature extraction is performed using an image processing module, including brightness feature extraction and color feature extraction. A threshold segmentation algorithm is used to accurately extract the light area in the grayscale image. Furthermore, a color space conversion algorithm is used to convert the color image obtained by the binocular camera from the first color space (RGB) to the second color space (CIE1931 XYZ), thereby obtaining the color features of the navigation lights.

[0053] Step S4: Performing lighting analysis through the calculation and analysis module and further analyzing the extracted color features, including visibility judgment and color matching;

[0054] Step S5: Output the results. According to the light analysis results, output the detection results of the navigation lights of the tested UAV.

[0055] Specifically, for step S1:

[0056] A binocular camera simulating the human eye was used to film the test drone at night when visibility met the standard. The camera remained stationary during the filming (the camera's photosensitive element should preferably use a charge-coupled device image sensor (CCD) with high sensitivity and better imaging quality. The drone was fully functional and had sufficient power during the filming). The camera was hovered at a distance of 120 m from the camera and focused at angles of 30°, 45°, and 60°, respectively. The three angles were filmed for 10 seconds each, and the videos were saved.

[0057] More specifically, step S2 is implemented as follows:

[0058] Step S2.1: (Using video processing software, such as OpenCV, etc.) extract each frame image from each video and save it as the image to be processed;

[0059] Step S2.2: Convert the extracted color image into a grayscale image f1(x,y), i.e. a matrix with pixels as elements, such as Figure 2 As shown, n*m is the pixel of the image. The grayscale image only contains brightness information, not color information. The visibility of different colored lights is subsequently calculated through an algorithm. The RGB values directly obtained by the camera are converted to grayscale values using a floating-point algorithm, that is, Grey = 0.299*R + 0.587*G + 0.114*B;

[0060] Step S2.3: Image smoothing (to improve image quality), denoising the grayscale image (image smoothing can be performed using algorithms such as median filtering, mean filtering, and Gaussian filtering. In this embodiment, Gaussian filtering is used).

[0061] Furthermore, step S3 is specifically implemented as the following steps:

[0062] Step S3.1: Apply the threshold segmentation algorithm to the matrix f1(x,y) to extract the light area f2, and the remaining dark background area is f3 (this embodiment uses the maximum inter-class variance (Otsu) method to determine the grayscale threshold). Specifically, the grayscale threshold is set to t, and the entire grayscale image is divided into two parts [0, t] and [t, 255]. The grayscale means of the parts [0, t] and [t, 255] are m1 and m2 respectively. The grayscale mean of the entire image is mg. The probability of a pixel being assigned to [0, t] and [t, 255] is p1 and p2 respectively. Let n i is the total number of pixels with grayscale i in the image, and the probability that a pixel in the image has grayscale i is: And there is have to:

[0063]

[0064] Solve the between-class variance The maximized gray level k is the maximum inter-class variance threshold t;

[0065] Step 3.2: Since grayscale images do not contain color information, color feature extraction is required for the RGB image obtained by the camera. First, the light area obtained after brightness feature extraction is used to extract the pixel points in the RGB image. Then, a color space conversion algorithm is used to convert the color image from the RGB color space to the CIE1931 XYZ color space. The X, Y, and Z tristimulus values of each pixel are obtained and calculated using the matrix:

[0066] Furthermore, step S4 is specifically implemented as the following steps:

[0067] Step S4.1: Determine visibility. The visibility of the drone's navigation lights needs to consider four factors: light source size, brightness, contrast, and atmospheric visibility. To determine whether it can be observed at night, the dimensional analysis method is used to calculate light visibility. The specific implementation is as follows:

[0068] Step S4.1.1: Calculate the number a of pixels in the extracted light area f2;

[0069] Step S4.1.2: Calculate the average brightness of the light area sum is the sum of the grayscale values of each pixel in the light area;

[0070] Step S4.1.3: Calculate the contrast between the light and the dark background. Use the RGB image to calculate the contrast. First, calculate the contrast of all pixels in the f3 area. Then calculate the f2 region

[0071] Step S4.1.4: Use a visibility meter to measure the atmospheric visibility distance l (km) and calculate the visibility

[0072] Step S4.1.5: Calculate the visibility of the drone’s navigation lights for each image Take the maximum value V among the three angles max ;

[0073] Step S4.1.6: Accurately determine the visibility threshold V0 of the drone’s navigation lights through big data analysis. The specific method is:

[0074] Step S4.1.6.1: Collect the results of different UAV lighting detection projects. Use the same model of binocular camera to collect the V of each UAV using the above steps S1 to S4.1.5. max value;

[0075] Step S4.1.6.2: Delineate visible and invisible areas based on the test results, e.g. Figure 3 As shown, the area above the red line is the visible area, and the area below the red line is the invisible area. The V max The minimum value is V max,yes,min , V in the visible region max The average value is V in the invisible area max The maximum value is V max,no,max , V in the invisible area max The average value is

[0076]

[0077] Step S4.1.7: When the visibility of the navigation lights of a certain type of UAV is V max ≥V0, it is judged as a navigation light visible to the human eye, otherwise it is judged as a navigation light invisible to the human eye;

[0078] Step S4.2: Perform color matching and calculate the X, Y, and Z values of each pixel in the light area obtained in step S3.2. And converted into color coordinates, the conversion method is (like Figure 4 As shown, the navigation light color is accurately displayed using CIE1931xy software reflected in the CIE1931 chromaticity diagram.

[0079] Preferably, for step S5, the light detection results are exported and printed in color, including the test angles and photos corresponding to each angle, the CIE1931 chromaticity diagram with the navigation light color coordinates, and the calculated visibility V max and V0 value, as well as visibility judgment result visible or invisible.

[0080] It is worth mentioning that the technical features such as the test drone involved in the patent application of this invention should be regarded as prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected by conventional options in the field, and should not be regarded as the inventive point of this patent. This patent will not be further elaborated.

[0081] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A drone lighting detection method based on machine vision and big data, characterized in that: The following steps are involved: Step S1: Capture video using a video acquisition module, using a binocular camera to shoot the drone at night, and obtain a video of the drone at a preset distance and angle; Step S2: performing image acquisition and processing through the image acquisition module, processing the acquired video, including frame extraction, grayscale conversion and image smoothing; Step S3: Light feature extraction is performed using an image processing module, including brightness feature extraction and color feature extraction. A threshold segmentation algorithm is used to accurately extract the light area in the grayscale image. A color space conversion algorithm is used to convert the color image obtained by the binocular camera from the first color space to the second color space, thereby obtaining the color features of the navigation lights. Step S4: Performing lighting analysis through the calculation and analysis module and further analyzing the extracted color features, including visibility judgment and color matching; Step S5: Output the results. According to the light analysis results, output the detection results of the navigation lights of the tested UAV.

2. The method for detecting UAV lighting based on machine vision and big data according to claim 1, characterized in that: For step S1: A binocular camera simulating the human eye was used to film the test drone at night when visibility met the standard. The camera remained stationary and hovered 120 meters away from the camera and focused at angles of 30°, 45°, and 60°, respectively. Each of the three angles was filmed for 10 seconds and the videos were saved.

3. The method for detecting UAV lighting based on machine vision and big data according to claim 2, characterized in that: Step S2 is specifically implemented as follows: Step S2.1: extract each frame image from each video and save it as the image to be processed; Step S2.2: Convert the extracted color image into a grayscale image f1(x,y), i.e., a matrix with pixels as elements, where n*m represents the number of pixels in the image. Grayscale images contain only brightness information, not color information. Subsequently, an algorithm is used to calculate the visibility of different colored lights. The RGB values directly obtained by the camera are converted to grayscale values using a floating-point algorithm, i.e., Grey = 0.299*R + 0.587*G + 0.114*B. Step S2.3: Image smoothing and denoising of the grayscale image.

4. The method for detecting UAV lighting based on machine vision and big data according to claim 3, characterized in that: Step S3 is specifically implemented as follows: Step S3.1: Apply the threshold segmentation algorithm to the matrix f1(x,y) to extract the light area f2 and the remaining dark background area f3. The specific method is to set the grayscale threshold as t, and the entire grayscale image is divided into two parts [0, t] and [t, 255]. The grayscale means of the parts [0, t] and [t, 255] are m1 and m2 respectively. The grayscale mean of the entire image is mg. The probability of a pixel being divided into [0, t] and [t, 255] are p1 and p2 respectively. Let n i is the total number of pixels with grayscale i in the image, and the probability that a pixel in the image has grayscale i is: And there is have to: Solve the between-class variance The maximized gray level k is the maximum inter-class variance threshold t; Step 3.2: Since grayscale images do not contain color information, color feature extraction is required for the RGB image obtained by the camera. First, the light area obtained after brightness feature extraction is used to extract the pixel points in the RGB image. Then, a color space conversion algorithm is used to convert the color image from the RGB color space to the CIE1931 XYZ color space. The X, Y, and Z tristimulus values of each pixel are obtained and calculated using the matrix:

5. The method for detecting UAV lighting based on machine vision and big data according to claim 4, characterized in that: Step S4 is specifically implemented as follows: Step S4.1: Determine visibility. The visibility of the drone's navigation lights needs to consider four factors: light source size, brightness, contrast, and atmospheric visibility. To determine whether it can be observed at night, the dimensional analysis method is used to calculate light visibility. The specific implementation is as follows: Step S4.1.1: Calculate the number a of pixels in the extracted light area f2; Step S4.1.2: Calculate the average brightness of the light area sum is the sum of the grayscale values of each pixel in the light area; Step S4.1.3: Calculate the contrast between the light and the dark background. Use the RGB image to calculate the contrast. First, calculate the contrast of all pixels in the f3 area. Then calculate the f2 region Contrast Step S4.1.4: Use a visibility meter to measure the atmospheric visibility distance l and calculate the visibility Step S4.1.5: Calculate the visibility of the drone’s navigation lights for each image Take the maximum value V among the three angles max ; Step S4.1.6: Accurately determine the visibility threshold V0 of the drone’s navigation lights through big data analysis. The specific method is: Step S4.1.6.1: Collect the results of different UAV lighting detection projects. Use the same model of binocular camera to collect the V of each UAV using the above steps S1 to S4.1.

5. max value; Step S4.1.6.2: Delineate the visible and invisible areas based on the test results. The area above the red line is the visible area, and the area below the red line is the invisible area. The V max The minimum value is V max,yes,min , V in the visible region max The average value is V in the invisible area max The maximum value is V max,no,max , V in the invisible area max The average value is Step S4.1.7: When the visibility of the navigation lights of a certain type of UAV is V max ≥V0, it is judged as a navigation light visible to the human eye, otherwise it is judged as a navigation light invisible to the human eye; Step S4.2: Perform color matching and calculate the X, Y, and Z values of each pixel in the light area obtained in step S3.

2. And converted into color coordinates, the conversion method is Accurately display the color of navigation lights.

6. The method for detecting UAV lighting based on machine vision and big data according to claim 5, characterized in that: For step S5, the light detection results are exported and printed in color. The information includes the test angles and photos corresponding to each angle, the CIE1931 chromaticity diagram with the navigation light color coordinates, and the calculated visibility V max and V0 value, as well as visibility judgment result visible or invisible.

Images