A visual detection method and system based on drone protection
By using drone visual inspection methods to filter images, determine obstacle information and operational parameters, and enable drones to quickly and effectively avoid obstacles, the problem of resource consumption and performance impact in existing drone protection is solved, and the convenience and accuracy of protection are improved.
Patent Information
- Application Number
- CN202411194359.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-08-29
AI Technical Summary
Existing drone protection methods involve installing equipment or modifying the drone's structure, which results in high resource consumption and performance impact, and are difficult to effectively prevent damage caused by untimely obstacle avoidance.
The UAV visual inspection method is used to acquire images for quality screening, extract obstacle information, determine the probability and degree of collision, confirm the degree of interference in combination with operating parameters, and select reference images to be transmitted to the control system to perform obstacle avoidance operations.
It improves the convenience and accuracy of drone protection, reduces useless image interference, provides fast and effective obstacle avoidance reference, and avoids drone damage.
Smart Images

Figure CN118711093B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of visual detection, and in particular to a visual detection method and system based on drone protection. Background Art
[0002] Drones, also known as unmanned aerial vehicles, are aircraft that do not require direct human control. They use advanced remote control technology, autonomous flight control systems, and sensor technology to perform various tasks in the air, such as reconnaissance, photography, and cargo transportation. The application of drones in the civilian field has brought many conveniences and benefits, such as aerial photography, farmland inspections, forest fire prevention, and so on. However, during the operation of drones, due to unexpected situations such as untimely obstacle avoidance, drones are easily damaged, resulting in economic losses and dangerous accidents. Therefore, the protection of drones is crucial.
[0003] Currently, the protection function of drones is mainly achieved by installing equipment, devices or modifying the drone structure. Using these methods to achieve drone protection not only consumes a lot of resources, but also easily affects the performance of the drone, causing inconvenience to the operation of the drone. Summary of the invention
[0004] The purpose of the present invention is to provide a visual detection method and system based on drone protection to solve the problems raised in the above background technology.
[0005] In the first aspect, the present application provides a visual detection method based on drone protection, which adopts the following technical solution:
[0006] Acquire the images taken by the drone, perform quality screening on the images, and obtain valid images;
[0007] Extract obstacle information from valid images and determine the collision probability of the drone based on the obstacle information;
[0008] Obtain the operating parameters of the drone, and determine the impact degree based on the operating parameters of the drone and obstacle information;
[0009] The impact degree of the obstacle on the UAV is determined based on the obstacle information and recorded as the operational impact degree. The interference degree of the obstacle on the UAV is confirmed by combining the collision probability and collision degree.
[0010] Extracting shooting information of the effective image, and confirming the accuracy of the information of the effective image according to the shooting information;
[0011] Obstacle avoidance requirement of the UAV is obtained according to the information accuracy and interference degree, and the valid image whose obstacle avoidance requirement reaches the preset obstacle avoidance requirement threshold is used as the reference image;
[0012] The reference image is transmitted to the control system of the UAV to execute the obstacle avoidance and protection operation of the UAV.
[0013] Preferably, the step of acquiring images taken by the drone and screening the quality of the images taken to obtain valid images is specifically as follows:
[0014] Obtaining the time point and shooting location of the image taken by the drone, and setting standard parameters of the image according to the shooting time point and shooting location, wherein the standard parameters include standard definition, standard brightness, and standard resolution;
[0015] Acquire images captured by the drone and extract image parameters of the captured images, wherein the image parameters include image clarity, image brightness, and image resolution;
[0016] Screen and remove images whose image parameters do not meet the standard parameters to obtain standard images;
[0017] Things that have no impact on the operation of the drone are recorded as invalid information, and things that have an impact on the operation of the drone are recorded as valid information;
[0018] The effective information ratio on the standard image is extracted, a threshold of the effective information ratio is set, and the standard images whose effective information ratio reaches the threshold are selected as effective images.
[0019] Preferably, the step of extracting obstacle information from the effective image and determining the collision probability of the drone according to the obstacle information is specifically as follows:
[0020] Extracting obstacle information from a valid image, wherein the obstacle information includes obstacle category and obstacle position;
[0021] Determine whether the obstacle is fixed based on the obstacle type. If the obstacle is fixed, obtain the flight direction of the drone and determine the collision probability based on the obstacle position and flight direction.
[0022] If the obstacle is not fixed, determine whether the obstacle has an autonomous obstacle avoidance function based on the obstacle type;
[0023] If the obstacle does not have an autonomous obstacle avoidance function, the obstacle running speed is extracted, and the obstacle running route is formed according to the obstacle running speed and obstacle position;
[0024] Obtain the flight speed of the drone, determine the flight path of the drone based on the flight speed and flight direction, and obtain the collision probability based on the obstacle route and the drone flight path;
[0025] If the obstacle has an autonomous obstacle avoidance function, the collision probability is determined according to the obstacle category.
[0026] Preferably, if the obstacle has an autonomous obstacle avoidance function, the step of determining the collision probability according to the obstacle category is specifically as follows:
[0027] Obtain the behavior of the obstacle according to the obstacle category, and obtain the obstacle avoidance sensitivity of the obstacle at different times according to the obstacle behavior matching;
[0028] Obtain the corresponding obstacle avoidance sensitivity based on the time point matching, and obtain the corresponding obstacle avoidance speed based on the obstacle avoidance sensitivity;
[0029] Determine whether the flight speed of the drone exceeds the avoidance speed. If it exceeds the avoidance speed, obtain the original collision probability based on the obstacle route and the drone flight route, and use the original collision probability as the collision probability;
[0030] If the flight speed does not exceed the obstacle avoidance speed, the speed difference between the flight speed and the avoidance speed is calculated, and the obstacle avoidance probability is obtained based on the speed difference;
[0031] The collision probability is obtained by subtracting the obstacle avoidance probability from the original collision probability.
[0032] Preferably, the step of obtaining the operating parameters of the drone and determining the impact degree according to the operating parameters of the drone and the obstacle information is specifically:
[0033] The volume of the obstacle is extracted based on the effective image, and the obstacle hardness and average moving speed are extracted based on the obstacle category;
[0034] Obtain the fuselage hardness of the drone, and combine the fuselage hardness, flight speed, obstacle hardness and average moving speed to obtain the degree of damage to the obstacle and the degree of damage to the drone;
[0035] The impact point is estimated based on the obstacle's running route and the drone's flight route, and the impact area is obtained based on the obstacle's volume and the impact point;
[0036] The impact weight ratios of the obstacle damage degree, the drone damage degree and the impact area are set respectively, and the impact degree is calculated according to the impact weight ratio.
[0037] Preferably, the step of determining the degree of influence of the obstacle on the UAV according to the obstacle information and recording it as the degree of influence on the operation is specifically:
[0038] Get the direction of the drone's light source, and determine whether the obstacle blocks the drone's light source based on the obstacle's running route and the drone's flight route;
[0039] If the obstacle does not block the light source of the drone, the degree of impact of the obstacle on the drone is obtained according to the obstacle category and recorded as the first impact degree, and the first impact degree is used as the operation impact degree;
[0040] If an obstacle blocks the drone’s light source, the blocking time is determined based on the obstacle’s running route and the drone’s flight route;
[0041] On the basis of the first impact degree, the impact degree of the occlusion time on the UAV is added as the operational impact degree.
[0042] Preferably, if the obstacle does not block the light source of the drone, the step of obtaining the degree of influence of the obstacle on the drone according to the obstacle category and recording it as the first degree of influence is specifically:
[0043] If the obstacle does not block the drone's light source, the probability of adhesion between the obstacle and the drone after collision is obtained according to the obstacle category;
[0044] Obstacle weight is obtained according to obstacle volume and obstacle category, and the influence degree of UAV flight balance is obtained according to obstacle weight and impact point and recorded as balance interference degree;
[0045] The balance influence of the UAV is calculated by multiplying the adhesion probability, the balance interference degree and the collision probability;
[0046] Determine whether the obstacle interferes with the drone's filming based on the impact point. If the obstacle interferes with the drone's filming, obtain the percentage of the obstacle's blocked area in the filming area.
[0047] Set the influence weight ratio of the blocked area ratio and the balance influence degree, calculate the influence degree of the obstacle on the drone according to the influence weight ratio and record it as the first influence degree.
[0048] Preferably, the step of extracting the shooting information of the effective image and confirming the accuracy of the information of the effective image according to the shooting information is specifically:
[0049] Extracting shooting information of a valid image, wherein the shooting information includes a real-time shooting angle, a real-time shooting environment, and a real-time lens blur;
[0050] Obtain the best shooting angle of the effective features of the obstacle, and calculate the angle difference between the best shooting angle and the real-time shooting angle;
[0051] Extracting real-time shooting brightness, real-time shooting light direction and real-time shooting background according to the real-time shooting environment;
[0052] Obtaining the best shooting environment for drone photography, wherein the best shooting environment includes the best shooting brightness, the best shooting light direction, and the best shooting background;
[0053] The environmental difference between the real-time shooting environment and the optimal shooting environment is calculated, and the information accuracy is calculated by combining the angle difference and the lens blur.
[0054] Preferably, the step of extracting shooting information of the effective image, wherein the shooting information includes a real-time shooting angle, a real-time shooting environment, and a real-time lens blur, is specifically as follows:
[0055] Get the flight altitude of the drone and combine it with the shooting location to get the air quality at the drone's location;
[0056] Obtain the weather at the location of the drone to determine whether the weather causes lens pollution. If the weather causes lens pollution, obtain the degree of weather pollution.
[0057] Extract the particulate matter content in the air quality, calculate the air pollution degree based on the particulate matter content, and superimpose the meteorological pollution degree to obtain the lens blur;
[0058] If the weather does not cause lens pollution, the degree of air pollution will be used as the lens blur.
[0059] In the second aspect, the present application provides a visual inspection system based on drone protection, which adopts the following technical solutions:
[0060] A visual inspection system based on drone protection includes: an effective image module, which obtains images taken by the drone, performs quality screening on the taken images, and obtains effective images;
[0061] The collision probability module extracts obstacle information from the effective image and determines the collision probability of the drone based on the obstacle information;
[0062] The impact degree module obtains the operating parameters of the drone and determines the impact degree based on the operating parameters of the drone and obstacle information;
[0063] The interference degree module determines the impact degree of the obstacle on the drone based on the obstacle information and records it as the operation impact degree. It also determines the interference degree of the obstacle on the drone by combining the collision probability and collision degree.
[0064] The information accuracy module extracts the shooting information of the valid image and confirms the information accuracy of the valid image based on the shooting information;
[0065] The obstacle avoidance requirement module obtains the obstacle avoidance requirement of the drone based on the information accuracy and interference level, and uses the valid image whose obstacle avoidance requirement reaches the preset obstacle avoidance requirement threshold as the reference image;
[0066] The obstacle avoidance and protection module transmits the reference image to the control system of the UAV and performs the obstacle avoidance and protection operations of the UAV.
[0067] In summary, the present application includes at least one of the following beneficial technical effects:
[0068] 1. Perform quality screening on the images taken by the drone, analyze the screened images, and analyze the collision probability and degree of the obstacle and the drone based on the extracted obstacle information, as well as the impact of the obstacle on the operation of the drone, so as to determine the interference degree of the obstacle. The image information accuracy is obtained based on the shooting information of the image, and the obstacle avoidance requirement is obtained in combination with the interference degree, so as to confirm whether the image needs to be transmitted to the drone control system and provide reference data for the drone obstacle avoidance. By analyzing the drone images, the image screening is realized, so as to provide a reference basis for the drone obstacle avoidance, reduce useless images, and improve the convenience of visual detection based on drone protection.
[0069] 2. Using the obstacle information extracted from the image, determine the collision probability between the obstacle and the drone based on whether the obstacle is fixed, whether the obstacle has an autonomous obstacle avoidance function, and the flight speed and direction of the drone. Then, the collision degree is obtained based on the volume and collision position of the obstacle, and the information contained in the image is analyzed to obtain the collision situation between the obstacle and the drone. At the same time, the degree of influence of the obstacle on the operation of the drone is obtained based on the occlusion and shooting effects of the obstacle on the drone, and finally the degree of interference of the obstacle is determined. The interference effect of the obstacle on the drone is obtained by considering multiple aspects, which improves the accuracy of visual detection based on drone protection.
[0070] 3. Analyze the information accuracy of the captured image according to the shooting angle, shooting environment and lens blur of the effective image, determine the obstacle avoidance requirement of the information contained in the image according to the information accuracy and the interference degree of the obstacle, and judge whether the image needs to be transmitted to the control system of the UAV according to the obstacle avoidance requirement. That is, analyze whether the image information is useful for the UAV obstacle avoidance, thereby improving the convenience and intelligence of visual detection based on UAV protection. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 It is a schematic diagram of the specific steps of an embodiment of a visual detection method based on drone protection of the present invention.
[0072] Figure 2 It is a module connection diagram of an embodiment of a visual detection system based on drone protection of the present invention. DETAILED DESCRIPTION
[0073] Below is a combination of the embodiments and Figure 1-Figure 2 The present invention is further described in detail, but the embodiments of the present invention are not limited thereto.
[0074] The present invention discloses a visual detection method based on drone protection, which specifically comprises the following steps:
[0075] Step S1, obtaining images taken by the drone, and performing quality screening on the images to obtain valid images.
[0076] Step S2: extract obstacle information from the effective image and determine the collision probability of the drone based on the obstacle information.
[0077] Step S3, obtaining the operating parameters of the UAV, and determining the impact degree according to the operating parameters of the UAV and the obstacle information.
[0078] Step S4: determine the degree of influence of the obstacle on the UAV based on the obstacle information and record it as the operation influence degree, and confirm the interference degree of the obstacle on the UAV by combining the collision probability and the collision degree.
[0079] The weight ratios of the operational impact degree, collision probability and collision degree are set respectively, and the interference degree is calculated according to the weight ratio.
[0080] Step S5, extracting the shooting information of the effective image, and confirming the accuracy of the information of the effective image according to the shooting information.
[0081] Step S6, obtaining the obstacle avoidance requirement of the UAV according to the information accuracy and the interference degree, and taking the valid image whose obstacle avoidance requirement reaches the preset obstacle avoidance requirement threshold as the reference image.
[0082] Subtract the percentage of information accuracy from 1 to get the information inaccuracy. Multiply the information inaccuracy by the interference level to get the interference float. Add the interference float to the interference level to get the obstacle avoidance requirement. For example, if the information accuracy and interference level are 60% and 40% respectively, then (1-60%) × 40% + 40% = 56%.
[0083] Step S7, transmitting the reference image to the control system of the UAV to execute the obstacle avoidance protection operation of the UAV.
[0084] In actual use, the application of drones has brought convenience to people. At the same time, since drones do not require human control, the protection of drones has become crucial. General drone protection is achieved by installing equipment and devices on drones, which not only increases the weight of drones, but also affects the operation of drones. The protection of drones through visual detection is achieved through the existing functions of drones and does not affect the operation of drones. By analyzing and processing the images taken by drones, drones can be helped to avoid obstacles, thereby achieving the effect of protecting drones. Since drones will take a large number of images, not all images are conducive to drone obstacle avoidance and route planning. Therefore, by analyzing the images and selecting images with reference value for drone obstacle avoidance, not only can the obstacle avoidance route be obtained quickly and conveniently, but also the interference of unnecessary images can be reduced.
[0085] The steps of obtaining images taken by drones and screening the quality of the images to obtain valid images are as follows:
[0086] Step S11, obtaining the time point and shooting location of the drone shooting image, and setting standard parameters of the image according to the shooting time point and shooting location, the standard parameters including standard clarity, standard brightness and standard resolution.
[0087] Step S12, acquiring the image captured by the drone, and extracting image parameters of the captured image, where the image parameters include image clarity, image brightness, and image resolution.
[0088] Step S13, screening and removing images whose image parameters do not meet the standard parameters to obtain a standard image.
[0089] Different times and locations have different requirements for shooting. For example, the natural brightness is higher when shooting during the day, and lower when shooting at night. If you take a photo with very low brightness during the day, it means that there is a problem with the shooting and it is not a suitable photo at the moment. When shooting at night, it is also not possible to take a very bright photo because it is dark. Set the shooting image standard at the current time point according to the weather conditions at the shooting location.
[0090] In step S14, things that have no impact on the operation of the drone are recorded as invalid information, and things that have an impact on the operation of the drone are recorded as valid information.
[0091] Step S15, extracting the effective information ratio on the standard image, setting an effective information ratio threshold, and screening the standard images whose effective information ratio reaches the effective information ratio threshold as effective images.
[0092] In actual use, not all images taken by drones can be used as references for obstacle avoidance. Some images are blurred due to shooting jitter, and some images are dark due to occlusion. These will affect the subsequent analysis of the images. Therefore, it is necessary to preliminarily screen the images and screen images that can extract effective information. For example, there are obstacles in the image, but because the image is blurred, it is difficult to identify the obstacle information, so such images are difficult to provide effective information, so they need to be deleted. In addition, the information contained in some images is not useful information for drone obstacle avoidance, so they also need to be screened out. For example, the drone captures lakes and skies, which do not hinder the drone and affect the drone's route. A large number of images interfere with the drone's obstacle avoidance route planning, so birds, balloons and other things that affect the drone's flight affect the effective information. At the same time, if the effective information accounts for a small area and is difficult to confirm, it also needs to be screened out. For example, there is a black object in the image, but the area is very small, and it is difficult to determine whether it is a bird, plastic or something else. It does not play an effective role in obstacle avoidance and needs to be screened out.
[0093] The steps of extracting obstacle information from the effective image and determining the collision probability of the drone based on the obstacle information are as follows:
[0094] Step S21, extracting obstacle information of the effective image, where the obstacle information includes obstacle category and obstacle position.
[0095] Step S22, judging whether the obstacle is fixed according to the obstacle category, if the obstacle is fixed, obtaining the flight direction of the UAV, and determining the collision probability according to the obstacle position and flight direction.
[0096] Step S23: If the obstacle is not stationary, determine whether the obstacle has an autonomous obstacle avoidance function according to the obstacle type.
[0097] Step S24: If the obstacle does not have an autonomous obstacle avoidance function, the obstacle running speed is extracted, and the obstacle running route is formed according to the obstacle running speed and the obstacle position.
[0098] Step S25, obtaining the flight speed of the UAV, determining the flight path of the UAV according to the flight speed and the flight direction, and obtaining the collision probability according to the obstacle running path and the UAV flight path.
[0099] Step S26: If the obstacle has an autonomous obstacle avoidance function, the collision probability is determined according to the obstacle type.
[0100] In actual use, current drones avoid obstacles when they encounter them, but not all obstacles need to be avoided. For non-living things that do not have the function of autonomous obstacle avoidance, such as balloons, plastic bags, etc., since they cannot avoid obstacles, drones need to avoid obstacles. For things that have autonomous obstacle avoidance, such as birds, they will autonomously adjust their flight direction after discovering drones to avoid drones. In this case, the drone can avoid obstacles without changing its route. Secondly, whether the obstacle is fixed or not also affects the probability of collision between the obstacle and the drone. If the obstacle is fixed, then if the obstacle is located on the drone's route, there will be a 100% collision, and the collision probability is 100%. If the obstacle moves, but there is no autonomous obstacle avoidance function, then the obstacle will have a certain operating law, and the operating route will be formed according to the operating law. The flight route of the drone and the obstacle's operating route are used to determine whether they will arrive at a certain point at the same time, thereby colliding. The collision probability between the drone and the obstacle is closely related to the type of obstacle, and specific problems should be analyzed specifically.
[0101] If the obstacle has an autonomous obstacle avoidance function, the steps to determine the collision probability are as follows:
[0102] Step S261, obtaining the behavior of the obstacle according to the obstacle category, and obtaining the obstacle avoidance sensitivity of the obstacle at different times according to the habit matching of the obstacle.
[0103] Step S262, obtaining a corresponding obstacle avoidance sensitivity according to the time point matching, and obtaining a corresponding obstacle avoidance speed according to the obstacle avoidance sensitivity.
[0104] Step S263, determine whether the flight speed of the drone exceeds the avoidance speed. If it exceeds the avoidance speed, obtain the original collision probability according to the obstacle running route and the drone flight route, and use the original collision probability as the collision probability.
[0105] Step S264: If the flight speed does not exceed the obstacle avoidance speed, the speed difference between the flight speed and the avoidance speed is calculated, and the obstacle avoidance probability is obtained according to the speed difference.
[0106] Step S265, subtracting the obstacle avoidance probability from the original collision probability to obtain the collision probability.
[0107] In actual use, general organisms or some technological products have the function of autonomous obstacle avoidance, that is, general organisms will autonomously avoid obstacles when they see them. Different objects have different obstacle avoidance sensitivities. On the one hand, the reaction speed of the object determines the obstacle avoidance sensitivity, and on the other hand, the obstacle avoidance speed of the object also affects the obstacle avoidance sensitivity. For example, some birds are more vigilant and move faster, so they can avoid slingshot projection to a large extent. Some domestic chickens are less vigilant, have slow reaction speeds, and move slower, so the hit rate of slingshot projection is higher. However, if the throwing speed is slower, domestic chickens can also avoid it. Therefore, according to the obstacle avoidance sensitivity and the distance between the drone and the obstacle, the obstacle avoidance speed that the object can avoid autonomously can be matched. If the flight speed exceeds the avoidance speed, it means that the obstacle is difficult to avoid the drone, and the collision probability is obtained according to the route. If the flight speed does not exceed the obstacle avoidance speed, the greater the difference, the greater the probability that the obstacle will avoid itself, and the smaller the collision probability.
[0108] The steps of obtaining the operating parameters of the drone and determining the impact degree according to the operating parameters of the drone and the obstacle information are as follows:
[0109] Step S31, extracting the volume of the obstacle according to the effective image, and obtaining the obstacle hardness and average moving speed according to the obstacle category.
[0110] Step S32, obtaining the fuselage hardness of the drone, combining the fuselage hardness, flight speed, obstacle hardness and average moving speed to obtain the degree of damage to the obstacle and the degree of damage to the drone.
[0111] Step S33, the impact point is estimated based on the obstacle running route and the UAV flight route, and the impact area is obtained based on the obstacle volume and the impact point.
[0112] Step S34, respectively setting the impact weight ratios of the obstacle damage degree, the drone damage degree and the impact area, and calculating the impact degree according to the impact weight ratios.
[0113] In actual use, not all obstacles will be damaged after collision. The harder the obstacle and the drone are, the greater the damage to both parties when the collision speed is faster. When the hardness is not large enough, such as when a plastic bag collides with a drone, it is not easy for both parties to be damaged. The degree of damage caused by the collision also needs to consider the area of the collision. The larger the area of the collision, the larger the area of damage, so the more serious the collision. Set the collision weight ratios of the obstacle damage degree, the drone damage degree and the collision area respectively, and then multiply them one by one and superimpose them to obtain the collision degree. For example, the collision weight ratios are 20%, 50%, and 30% respectively, and the obstacle damage degree, drone damage degree and collision area are 40%, 20% and 9 square centimeters respectively. The calculation result is 20%×40%+50%×20%+30%×0.09=20.7%.
[0114] The steps of determining the degree of influence of obstacles on the UAV based on the obstacle information and recording it as the degree of influence on the operation are as follows:
[0115] Step S41, obtaining the direction of the light source of the drone, and judging whether the obstacle blocks the light source of the drone according to the obstacle's running route and the drone's flight route.
[0116] Step S42: If the obstacle does not block the light source of the drone, the degree of influence of the obstacle on the drone is obtained according to the obstacle category and recorded as the first influence degree, and the first influence degree is used as the operation influence degree.
[0117] Step S43: If an obstacle blocks the light source of the drone, the blocking time is determined according to the obstacle's running route and the drone's flight route.
[0118] Step S44, based on the first impact degree, the impact degree of the shielding time on the UAV is added as the operation impact degree.
[0119] In actual use, some obstacles will not collide with the drone, but it does not mean that they will not affect the flight of the drone. Similarly, some obstacles collide with the drone, but do not cause any damage, but will have other effects on the drone. For example, some obstacles collide with the drone, although they do not damage the drone, but if they are attached to the drone's fuselage, they will affect the actual flight of the drone. If the obstacle blocks the light of the drone, it will have an occlusion effect on the drone. For example, the obstacle will not collide with the drone, but the obstacle blocks the light source of the drone, which will affect the subsequent image shooting and analysis of the drone. The degree of influence of the occlusion time on the drone is obtained according to the length of the occlusion time and the degree of brightness reduction of the occlusion.
[0120] If the obstacle does not block the light source of the drone, the step of obtaining the degree of influence of the obstacle on the drone according to the obstacle category and recording it as the first degree of influence is as follows:
[0121] Step S421: If the obstacle does not block the light source of the drone, the probability of adhesion between the obstacle and the drone after collision is obtained according to the obstacle category.
[0122] The adhesion probability is obtained based on the viscosity between the drone and the obstacle and the impact speed. The greater the viscosity, the greater the adhesion probability, and the greater the impact speed, the greater the adhesion probability.
[0123] Step S422, obtaining the weight of the obstacle according to the obstacle volume and obstacle type, and obtaining the degree of influence on the flight balance of the UAV according to the obstacle weight and the impact point and recording it as the balance interference degree.
[0124] Step S423, the balance influence of the UAV is calculated by multiplying the adhesion probability, the balance interference degree and the collision probability.
[0125] Step S424, judging whether the obstacle interferes with the shooting of the drone according to the impact point, if the obstacle interferes with the shooting of the drone, the blocking area ratio of the obstacle in the shooting area is obtained.
[0126] Step S425, setting the influence weight ratio of the blocked area ratio and the balance influence degree, and calculating the influence degree of the obstacle on the drone according to the influence weight ratio and recording it as the first influence degree.
[0127] In actual use, some objects will stick to the drone after colliding with it, causing the drone to be unbalanced. For example, after a plastic bag collides with a drone, although it will not cause any damage to the drone, it is easy to stick to the drone's body and affect the drone's flight. If the object is stuck to the center of the drone's body, the impact on the balance will be smaller. If it is stuck to the corners of the drone, it is more likely to cause imbalance. The heavier the object, the greater the impact on the drone. For example, a dragonfly with a smaller weight will cause the drone to tilt in flight, but a bird with a larger weight may cause the drone to fall. If the collision probability is 0, there will be no sticking. Therefore, the collision probability needs to be considered when calculating the balance interference. Some objects will also block the drone's camera lens, affecting the drone's subsequent image shooting and subsequent image analysis, thereby affecting the drone's obstacle avoidance protection. For example, some small insects, although the impact of the collision is low and the impact on the drone's flight is also low, it does not mean that you don't need to care about these small insects. Their blocking of the lens also greatly affects the drone's protection.
[0128] The steps of extracting the shooting information of the valid image and confirming the accuracy of the information of the valid image according to the shooting information are specifically as follows:
[0129] Step S51, extracting shooting information of the effective image, where the shooting information includes real-time shooting angle, real-time shooting environment and real-time lens blur.
[0130] Step S52, obtaining the best shooting angle of the effective features of the obstacle, and calculating the angle difference between the best shooting angle and the real-time shooting angle.
[0131] The best shooting angle is obtained based on the characteristic position of the obstacle, and the angle closest to the real-time shooting angle is used for calculation when calculating the difference. For example, the head of the red-crowned crane needs to be photographed, and both the front and side angles can be photographed. The front and side angles are different, so the shooting angle closest to the real-time shooting angle is selected for calculation.
[0132] Step S53, extracting the real-time shooting brightness, the real-time shooting light direction and the real-time shooting background according to the real-time shooting environment.
[0133] The shooting light direction refers to the position of the shooting position and the light source. For example, a backlit position will cause the picture to be overexposed or produce strong shadows, affecting the clarity and detail of the image.
[0134] Step S54, obtaining the best shooting environment for drone photography, where the best shooting environment includes the best shooting brightness, the best shooting light direction, and the best shooting background.
[0135] Step S55, calculating the difference between the real-time shooting environment and the optimal shooting environment, combining the angle difference and the lens blur, and calculating the information accuracy.
[0136] Select the best environment value that is closest to the parameters of the best shooting environment and the real-time shooting environment, calculate the difference and superimpose them to get the environment difference. Then set the weight ratio of the angle difference, environment difference and lens blur respectively, multiply them one by one and superimpose them to get the information accuracy.
[0137] In actual use, all analyses are based on images taken by drones. If it is difficult to extract accurate information from the images, the accuracy of the image analysis results will be greatly reduced. For example, drones will encounter many different kinds of birds during flight, and different birds have different obstacle avoidance sensitivities. If you only shoot bird wings and extract information based on bird wings, the obstacle category obtained will not be very accurate. The accuracy of the information is obtained based on the shooting angle, environmental conditions, and lens blur. For example, the red-crowned crane is characterized by a red head, so the best shooting angle is to include the head. Crows are all black, so they are difficult to identify against a black background, and the background color needs to be different from the crow's color.
[0138] The steps of extracting shooting information of a valid image, where the shooting information includes a real-time shooting angle, a real-time shooting environment, and a real-time lens blur, are as follows:
[0139] Step S511, obtaining the flight altitude of the drone, and combining the shooting location to obtain the air quality at the location of the drone.
[0140] The air quality data is obtained through the air quality data released by the Meteorological Bureau. If the drone has its own air quality sensor, the data is collected based on the sensor.
[0141] Step S512, obtaining the weather at the location of the drone, determining whether the weather causes lens pollution, and if the weather causes lens pollution, obtaining the degree of weather pollution.
[0142] Some weather conditions will not cause lens pollution, such as sunny and cloudy days, and there will be no additional weather products blocking the drone lens.
[0143] The degree of meteorological pollution is obtained based on the percentage of lens occlusion area caused by weather. For example, if the percentage of lens occlusion caused by rain is 20%, the degree of meteorological pollution is 20%.
[0144] Step S513, extracting the particulate matter content in the air quality, calculating the air pollution degree according to the particulate matter content, and superimposing the meteorological pollution degree to obtain the lens blur.
[0145] The percentage of the blocked area of particles is calculated based on the particle content and the lens area, thus obtaining the degree of air pollution. The higher the particle content, the larger the percentage of the blocked area.
[0146] Step S514: If the weather does not cause lens pollution, the degree of air pollution is used as the lens blur.
[0147] In actual use, lens blur will also affect the quality of drone images and the accuracy of information. For example, when it rains or snows, rain and snow will fall on the drone lens, blocking part of the shot and causing the lens to be unclear. At the same time, tiny particles such as dust, sand, and pollen in the air are easily attached to the surface of the lens, especially when shooting outdoors. These particles are brought to the lens by the wind or shooting activities. When the lens is blocked, some parts of the image will be blocked, resulting in inaccurate extraction of effective information, which in turn leads to inaccurate image analysis results, and ultimately affects the protection of the drone.
[0148] A visual inspection system based on drone protection, by applying a visual inspection method based on drone protection as described above, includes: an effective image module, which obtains images taken by the drone, performs quality screening on the taken images, and obtains effective images.
[0149] The collision probability module extracts obstacle information from the effective image and determines the collision probability of the drone based on the obstacle information.
[0150] The impact degree module obtains the operating parameters of the drone and determines the impact degree based on the operating parameters of the drone and obstacle information.
[0151] The interference degree module determines the impact degree of the obstacle on the drone based on the obstacle information and records it as the operation impact degree. It also determines the interference degree of the obstacle on the drone by combining the collision probability and collision degree.
[0152] The information accuracy module extracts the shooting information of the valid image and confirms the information accuracy of the valid image based on the shooting information.
[0153] The obstacle avoidance requirement module obtains the obstacle avoidance requirement of the drone based on the information accuracy and interference level, and uses the valid image whose obstacle avoidance requirement reaches the preset obstacle avoidance requirement threshold as the reference image.
[0154] The obstacle avoidance and protection module transmits the reference image to the control system of the UAV and performs the obstacle avoidance and protection operations of the UAV.
[0155] In actual use, by obtaining images taken by drones, after preliminary screening, effective images that are helpful for drone obstacle avoidance are obtained, obstacle information of effective images is extracted, and the collision probability, collision degree and operation impact degree are obtained by combining the operation parameters of the drone, and the interference degree of obstacles is calculated according to the weight ratio. At the same time, according to the shooting information of the effective image, the information accuracy of the image is analyzed, and the obstacle avoidance requirement of the image is obtained according to the information accuracy and interference degree. The effective images that reach the threshold are screened and uploaded as reference images for drone obstacle avoidance to form an obstacle avoidance route.
[0156] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.
Claims
1. A visual detection method based on drone protection, It is characterized in that The following steps are involved: Acquire the image taken by the drone, obtain the time point and location of the image and set the corresponding standard parameters, extract the image parameters of the image, filter and obtain the standard image according to the standard parameters and image parameters, record the things that affect the operation of the drone as valid information, and obtain the valid image according to the proportion of valid information on the standard image; Extract obstacle information from valid images and determine the collision probability of the drone based on the obstacle information; Obtain the operating parameters of the drone, and determine the impact degree based on the operating parameters of the drone and obstacle information; The impact degree of the obstacle on the UAV is determined based on the obstacle information and recorded as the operational impact degree. The interference degree of the obstacle on the UAV is confirmed by combining the collision probability and collision degree. The step of determining the degree of influence of the obstacle on the UAV according to the obstacle information and recording it as the degree of influence on the operation is specifically as follows: Get the direction of the drone's light source, and determine whether the obstacle blocks the drone's light source based on the obstacle's running route and the drone's flight route; If the obstacle does not block the light source of the drone, the degree of impact of the obstacle on the drone is obtained according to the obstacle category and recorded as the first impact degree, and the first impact degree is used as the operation impact degree; If an obstacle blocks the drone’s light source, the blocking time is determined based on the obstacle’s running route and the drone’s flight route; On the basis of the first impact degree, the impact degree of the occlusion time on the drone is added as the operational impact degree; If the obstacle does not block the light source of the drone, the step of obtaining the degree of influence of the obstacle on the drone according to the obstacle category and recording it as the first degree of influence is specifically as follows: If the obstacle does not block the drone's light source, the probability of adhesion between the obstacle and the drone after collision is obtained according to the obstacle category; Obstacle weight is obtained according to obstacle volume and obstacle category, and the influence degree of UAV flight balance is obtained according to obstacle weight and impact point and recorded as balance interference degree; The balance influence of the UAV is calculated by multiplying the adhesion probability, the balance interference degree and the collision probability; Determine whether the obstacle interferes with the drone's filming based on the impact point. If the obstacle interferes with the drone's filming, obtain the percentage of the obstacle's blocked area in the filming area. Set the influence weight ratio of the blocked area ratio and the balance influence degree, and calculate the influence degree of the obstacle on the drone based on the influence weight ratio and record it as the first influence degree; Extracting shooting information of the effective image, and confirming the accuracy of the information of the effective image according to the shooting information; Obstacle avoidance requirement of the UAV is obtained according to the information accuracy and interference degree, and the valid image whose obstacle avoidance requirement reaches the preset obstacle avoidance requirement threshold is used as the reference image; The reference image is transmitted to the control system of the UAV to execute the obstacle avoidance and protection operation of the UAV.
2. According to claim 1, a visual detection method based on drone protection, It is characterized in that The steps of obtaining the image captured by the drone, obtaining the time point and the shooting location of the image and setting the corresponding standard parameters, extracting the image parameters of the captured image, screening the standard image according to the standard parameters and the image parameters, recording the things that affect the operation of the drone as valid information, and obtaining the valid image according to the proportion of valid information on the standard image are specifically as follows: Obtaining the time point and shooting location of the image taken by the drone, and setting standard parameters of the image according to the shooting time point and shooting location, wherein the standard parameters include standard definition, standard brightness, and standard resolution; Acquire images captured by the drone and extract image parameters of the captured images, wherein the image parameters include image clarity, image brightness, and image resolution; Screen and remove images whose image parameters do not meet the standard parameters to obtain standard images; Things that have no impact on the operation of the drone are recorded as invalid information, and things that have an impact on the operation of the drone are recorded as valid information; The effective information ratio on the standard image is extracted, a threshold of the effective information ratio is set, and the standard images whose effective information ratio reaches the threshold are selected as effective images.
3. According to the visual detection method based on drone protection according to claim 2, It is characterized in that The step of extracting obstacle information from the effective image and determining the collision probability of the drone according to the obstacle information is specifically as follows: Extracting obstacle information from a valid image, wherein the obstacle information includes obstacle category and obstacle position; Determine whether the obstacle is fixed based on the obstacle type. If the obstacle is fixed, obtain the flight direction of the drone and determine the collision probability based on the obstacle position and flight direction. If the obstacle is not fixed, determine whether the obstacle has an autonomous obstacle avoidance function based on the obstacle type; If the obstacle does not have an autonomous obstacle avoidance function, the obstacle running speed is extracted, and the obstacle running route is formed according to the obstacle running speed and obstacle position; Obtain the flight speed of the drone, determine the flight path of the drone based on the flight speed and flight direction, and obtain the collision probability based on the obstacle route and the drone flight path; If the obstacle has an autonomous obstacle avoidance function, the collision probability is determined according to the obstacle category.
4. According to the visual detection method based on drone protection according to claim 3, It is characterized in that If the obstacle has an autonomous obstacle avoidance function, the step of determining the collision probability according to the obstacle category is specifically as follows: Obtain the behavior of the obstacle according to the obstacle category, and obtain the obstacle avoidance sensitivity of the obstacle at different times according to the obstacle behavior matching; Obtain the corresponding obstacle avoidance sensitivity based on the time point matching, and obtain the corresponding obstacle avoidance speed based on the obstacle avoidance sensitivity; Determine whether the flight speed of the drone exceeds the avoidance speed. If it exceeds the avoidance speed, obtain the original collision probability based on the obstacle route and the drone flight route, and use the original collision probability as the collision probability; If the flight speed does not exceed the obstacle avoidance speed, the speed difference between the flight speed and the avoidance speed is calculated, and the obstacle avoidance probability is obtained based on the speed difference; The collision probability is obtained by subtracting the obstacle avoidance probability from the original collision probability.
5. According to claim 4, a visual detection method based on drone protection, It is characterized in that The step of obtaining the operating parameters of the drone and determining the impact degree according to the operating parameters of the drone and the obstacle information is specifically as follows: The volume of the obstacle is extracted based on the effective image, and the obstacle hardness and average moving speed are extracted based on the obstacle category; Obtain the fuselage hardness of the drone, and combine the fuselage hardness, flight speed, obstacle hardness and average moving speed to obtain the degree of damage to the obstacle and the degree of damage to the drone; The impact point is estimated based on the obstacle's running route and the drone's flight route, and the impact area is obtained based on the obstacle's volume and the impact point; The impact weight ratios of the obstacle damage degree, the drone damage degree and the impact area are set respectively, and the impact degree is calculated according to the impact weight ratio.
6. A visual detection method based on drone protection according to claim 5, It is characterized in that The step of extracting the shooting information of the effective image and confirming the accuracy of the information of the effective image according to the shooting information is specifically as follows: Extracting shooting information of a valid image, wherein the shooting information includes a real-time shooting angle, a real-time shooting environment, and a real-time lens blur; Obtain the best shooting angle of the effective features of the obstacle, and calculate the angle difference between the best shooting angle and the real-time shooting angle; Extracting real-time shooting brightness, real-time shooting light direction and real-time shooting background according to the real-time shooting environment; Obtaining the best shooting environment for drone photography, wherein the best shooting environment includes the best shooting brightness, the best shooting light direction, and the best shooting background; The environmental difference between the real-time shooting environment and the optimal shooting environment is calculated, and the information accuracy is calculated by combining the angle difference and the lens blur.
7. A visual detection method based on drone protection according to claim 6, It is characterized in that The step of extracting shooting information of the effective image, wherein the shooting information includes a real-time shooting angle, a real-time shooting environment, and a real-time lens blur, is specifically as follows: Get the flight altitude of the drone and combine it with the shooting location to get the air quality at the drone's location; Obtain the weather at the location of the drone to determine whether the weather causes lens pollution. If the weather causes lens pollution, obtain the degree of weather pollution. Extract the particulate matter content in the air quality, calculate the air pollution degree based on the particulate matter content, and superimpose the meteorological pollution degree to obtain the lens blur; If the weather does not cause lens pollution, the degree of air pollution will be used as the lens blur.
8. A visual inspection system based on drone protection, It is characterized in that By applying a visual detection method based on drone protection as described in any one of claims 1 to 7, comprising: The effective image module obtains the images taken by the drone, screens the quality of the images, and obtains effective images; The collision probability module extracts obstacle information from the effective image and determines the collision probability of the drone based on the obstacle information; The impact degree module obtains the operating parameters of the drone and determines the impact degree based on the operating parameters of the drone and obstacle information; The interference degree module determines the impact degree of the obstacle on the drone based on the obstacle information and records it as the operation impact degree. It also determines the interference degree of the obstacle on the drone by combining the collision probability and collision degree. The information accuracy module extracts the shooting information of the valid image and confirms the information accuracy of the valid image based on the shooting information; The obstacle avoidance requirement module obtains the obstacle avoidance requirement of the drone based on the information accuracy and interference level, and uses the valid image whose obstacle avoidance requirement reaches the preset obstacle avoidance requirement threshold as the reference image; The obstacle avoidance and protection module transmits the reference image to the control system of the UAV and performs the obstacle avoidance and protection operations of the UAV.
Citation Information
Patent Citations
Unmanned aerial vehicle navigation control system and method based on big data analysis
CN112799426A
Obstacle avoidance system of unmanned aerial vehicle
CN115599119A
Unmanned aerial vehicle obstacle avoidance method, obstacle avoidance device and unmanned aerial vehicle
CN118034352A
Obstacle detection device
JP2022065347A
Methods and systems for bird deterrence and maintenance thereof
US20200156784A1
