An Automatic Recognition Method and System for Bare Soil Based on Real-Time Images of UAVs

Abstract

This application relates to the field of image recognition technology, and specifically relates to a method and system for automatic recognition of bare soil based on real-time images of unmanned aerial vehicles. The method includes: during the inspection process of the unmanned aerial vehicle, obtaining real-time monitoring screen images, and obtaining several frames of images containing uncovered areas of bare soil to form an image set; obtaining a plurality of image blocks; classifying all image blocks into dark blocks and bright blocks; calculating the first evaluation value, the second evaluation value, and the discrimination coefficient of each image block; screening all dark blocks to obtain each shadow block; after removing the shadows of all shadow blocks in each frame of image, obtaining each frame of image after shadow removal, training the target detection model, and performing bare soil recognition on the real-time monitoring screen images. This application can accurately remove the shadows generated by occlusion in the image, improve the accuracy of the target detection model in recognizing bare soil in the image, and is beneficial to accurately recognizing the bare soil area in the real-time monitoring screen.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and particularly relates to a method and system for automatically identifying bare soil based on real-time images of unmanned aerial vehicles (UAVs). Background Art

[0002] The rapid advancement of urbanization is a process of territorial space transformation. During the urbanization construction process, there are often uncovered bare ground surfaces. As the main pollution source of the urban environment, if the bare ground surfaces cannot be processed in time, dust is easily generated, which will then harm the health of the public. Therefore, it is necessary to identify and supervise the bare ground surfaces within the city.

[0003] Currently, the monitoring and identification of bare soil on urban ground mainly rely on manual inspections. However, this method has obvious defects. It not only consumes human resources but also causes the bare ground surfaces within the city to not be discovered in time. When using image recognition technology to identify bare soil, due to the complex internal environment of the city, the real-time video images obtained by UAV inspections will have shadow areas caused by various obstructions, resulting in a low accuracy rate for identifying bare soil in the images and a high false detection rate for bare soil identification. Summary of the Invention

[0004] To solve the above technical problems, a method and system for automatically identifying bare soil based on real-time images of UAVs are provided to solve the existing problems.

[0005] The solution of this application to solve the technical problems is to provide a method and system for automatically identifying bare soil based on real-time images of UAVs, including the following steps:

[0006] In the first aspect, an embodiment of this application provides a method for automatically identifying bare soil based on real-time images of UAVs. The method includes the following steps:

[0007] During the UAV inspection process, real-time monitoring video images are obtained, and several frames of images containing uncovered bare soil areas are obtained to form an image set;

[0008] Perform color space transformation on each frame of image in the image set, extract the brightness information, and based on the brightness of each frame of image, segment each frame of image to obtain multiple image blocks;

[0009] Based on the average level of the brightness of the pixel points within each image block, all the image blocks in each frame of image are classified into dark blocks and bright blocks;

[0010] Calculate the first evaluation value of each image block through the energy distribution characteristics of the brightness of the pixel points within each image block in the frequency domain and the change trend of the brightness difference between different pixel points in the spatial domain;

[0011] Analyze the distribution difference of the brightness of the pixel points located at the boundary within each image block in the local area, as well as the texture characteristics of the local areas in different image blocks on both sides of the pixel points located at the boundary, and calculate the second evaluation value of each image block;

[0012] Fuse the first evaluation value and the second evaluation value to obtain the discrimination coefficient of each image block;

[0013] Based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of image, screen all dark blocks in each frame of image to obtain each shadow block;

[0014] After removing the shadows from all shadow blocks in each frame of image, perform an inverse color space transformation to obtain each frame of image after shadow removal. Train the target detection model with all the images after shadow removal in the image set, and perform bare soil recognition on the real-time monitoring screen images.

[0015] Preferably, the color space transformation of each frame of image in the image set and the extraction of brightness information include: performing a HIS forward transformation on each frame of image, converting the image from the RGB space to the HIS space, and extracting the I component.

[0016] Preferably, the classification of all image blocks in each frame of image into each dark block and each bright block includes:

[0017] Calculate the mean value of the I components of all pixel points in each image block, denoted as the average brightness;

[0018] Obtain the segmentation threshold of the average brightness of all image blocks in each frame of image. Denote the image blocks with the average brightness less than or equal to the segmentation threshold as each dark block, and vice versa, denote them as each bright block.

[0019] Preferably, the calculation of the first evaluation value of each image block includes:

[0020] Perform frequency domain analysis on the I components of all pixel points in each image block to obtain a spectrogram, extract the low-frequency energy in the spectrogram, and calculate the proportion of the low-frequency energy in the spectrogram, denoted as the low-frequency energy ratio;

[0021] Calculate the distance between the position coordinates of each pixel point and the central pixel point in each image block, and the difference in the I component, denoted as the relative distance and the brightness difference respectively; form a two-dimensional array with the relative distance and the brightness difference, and obtain the slope after linear fitting of all two-dimensional arrays in each image block, denoted as the brightness change rate;

[0022] The first evaluation value is the product of the low-frequency energy ratio and the brightness change rate.

[0023] Preferably, the calculation of the second evaluation value of each image block includes:

[0024] Denote the pixel points located at the boundaries within each image block as boundary pixel points; construct a local window with a preset size centered on any boundary pixel point within each image block.

[0025] Perform clustering on the I components of all pixel points within the local window to obtain two clustering clusters; calculate the difference in the I components between the clustering centers of the two clustering clusters, denoted as the brightness distribution difference.

[0026] Analyze the difference in the texture features of the local regions within the image blocks on both sides of any boundary pixel point to obtain the texture consistency of the any boundary pixel point.

[0027] The second evaluation value is the sum of the products of the brightness distribution difference and the texture consistency of all boundary pixel points within each image block.

[0028] Preferably, obtaining the texture consistency of the any boundary pixel point includes:

[0029] Denote the remaining image blocks adjacent to the any boundary pixel point in each image block as neighboring image blocks.

[0030] Construct windows with a preset size within the two image blocks on both sides of the any boundary pixel point respectively. Denote the window on one side within the image block to which the any boundary pixel point belongs as the internal window, and denote the window on one side within the neighboring image block as the external window. Among them, both the internal window and the external window contain the any boundary pixel point.

[0031] Grayscale each frame of the image to obtain a grayscale image; extract the texture features of the corresponding regions of the internal window and the external window within the grayscale image to obtain the texture feature values of the internal window and the external window respectively.

[0032] Take the difference in the texture feature values between the internal window and the external window as the texture consistency of the any boundary pixel point.

[0033] Preferably, the discrimination coefficient of each image block is the ratio of the first evaluation value to the second evaluation value.

[0034] Preferably, the further method for obtaining each shadow block is:

[0035] Calculate the upper quartile of the discrimination coefficients of all bright blocks in each frame of the image, denoted as the judgment threshold; denote the dark blocks with discrimination coefficients less than the judgment threshold in each frame of the image as each shadow block.

[0036] Preferably, each frame of image after shadow removal includes: performing HIS inverse transformation after removing all shadow blocks in each frame of image through a geodesic shadow removal algorithm to obtain each frame of image after shadow removal.

[0037] In a second aspect, an embodiment of the present application further provides a bare soil automatic recognition system based on real-time images of an unmanned aerial vehicle, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned bare soil automatic recognition methods based on real-time images of an unmanned aerial vehicle are implemented.

[0038] The present application has at least the following beneficial effects:

[0039] In the present application, by performing color space transformation on each frame of image and extracting the brightness in each frame of image, each frame of image is segmented into multiple image blocks. Then, based on the average brightness of the pixel points in the image blocks, the dark blocks and the bright blocks are distinguished. The beneficial effect is that the image blocks are preliminarily screened according to the brightness degree of the image blocks to preliminarily clarify the shadow area and the non-shadow area. Secondly, the first evaluation value of each image block is calculated. The beneficial effect is that it takes into account the low-frequency energy situation in the image block and the variation characteristics between the brightness and distance of different pixel points, and reflects the possibility of the image block being a shadow area through the variation characteristics of the pixel point brightness in the frequency domain and the spatial domain. The second evaluation value of each image block is calculated. The beneficial effect is that it takes into account the distribution characteristics of the brightness within the local range of the pixel points located at the boundary of the image block to illustrate the gradual change of the brightness of the boundary pixel points, analyzes the consistency of the texture characteristics of the local areas within two adjacent image blocks on both sides of the pixel points located at the boundary, reflects the possibility that the local areas on both sides belong to the same type of object, and thus reflects the possibility of the corresponding image block being a shadow area. The discrimination coefficient of each image block is obtained, and based on the discrimination coefficients of all the bright blocks and all the dark blocks in each frame of image, all the dark blocks in each frame of image are screened to obtain each shadow block. The beneficial effect is that through the discrimination coefficient of the bright block corresponding to the non-shadow area, further screening is performed on the dark blocks that may be shadow areas, and the interference of dark-colored buildings or other objects with low brightness inside the city on the shadow area is eliminated, improving the discrimination result between the shadow area and the non-shadow area in the image. After removing the shadows from all the shadow blocks in each frame of image and performing color space inverse transformation, each frame of image after shadow removal is obtained. The target detection model is trained through all the images after shadow removal in the image set, and the bare soil in the real-time monitoring screen image is recognized. The beneficial effect is that by removing the shadows generated due to occlusion in the image and then training the target detection model, the accuracy of the target detection model in recognizing the bare soil in the image is improved, which is beneficial to accurately recognizing the bare soil area in the real-time monitoring screen. Description of the Drawings

[0040] The following further elaborates in detail a method for automatically identifying bare soil based on real-time images of an unmanned aerial vehicle (UAV) in conjunction with the accompanying drawings.

[0041] Figure 1 It is a flowchart of the steps of a method for automatically identifying bare soil based on real-time images of an unmanned aerial vehicle (UAV) provided in an embodiment of the present application;

[0042] Figure 2 It is a flowchart of the steps of a method for obtaining the first evaluation value of each image block provided in an embodiment of the present application;

[0043] Figure 3 It is a flowchart of the steps of a method for obtaining each shadow block provided in an embodiment of the present application. Detailed implementation manners

[0044] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the following further elaborates in detail a method and system for automatically identifying bare soil based on real-time images of an unmanned aerial vehicle (UAV) in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0046] Please refer to Figure 1 , which shows a flowchart of the steps of a method for automatically identifying bare soil based on real-time images of an unmanned aerial vehicle (UAV) provided in an embodiment of the present application. The method includes the following steps:

[0047] Step 1, during the UAV inspection process, real-time monitoring screen images are obtained, and several frames of images containing uncovered bare soil areas are obtained to form an image set.

[0048] Through the UAV inspection, video data containing uncovered bare soil is captured from different heights and angles. After key frame extraction from the video data and operations such as rotation, translation, and scaling, several frames of images containing uncovered bare soil areas are screened out to form an image set;

[0049] In this embodiment, the ffmpeg tool is used to perform key frame extraction on the video data, followed by operations such as rotation, translation, and scaling, and manual screening is carried out to obtain each frame of image. The ffmpeg tool is a well-known technology and will not be elaborated here.

[0050] During the real-time monitoring video of the UAV inspection, it is transmitted to a streaming media server, and the ffmpeg tool is used to extract key frames to obtain real-time monitoring screen images.

[0051] So far, the monitored video image and the image set are obtained.

[0052] Step 2: Perform color space transformation on each frame of the image set, extract the luminance information, segment each frame of the image based on the luminance of each frame to obtain multiple image blocks; divide all the image blocks in each frame into dark blocks and bright blocks based on the average level of the luminance of the pixel points within each image block; calculate the first evaluation value of each image block through the energy distribution characteristics of the luminance of the pixel points within each image block in the frequency domain and the change trend of the luminance difference between different pixel points in the spatial domain.

[0053] Since the internal environment of the city is relatively complex, objects such as buildings and trees will cast shadows in the images captured by the UAV, and these shadows will interfere with the recognition of bare soil and reduce the recognition accuracy. Therefore, in order to improve the accuracy of bare soil recognition, it is necessary to preprocess the images before bare soil recognition to remove the influence of shadows. The traditional geodesic shadow removal algorithm can effectively process large semi-shadow areas by adjusting the multiplicative factor to enhance the contrast of the shadow area. In practical applications, when distinguishing between the shadow area and the non-shadow area in the image, it is generally necessary to manually set the division threshold and cannot be flexibly adjusted according to the actual situation in the image, resulting in insufficient flexibility and adaptability of the algorithm. To solve the limitations of the traditional algorithm, the geodesic shadow removal algorithm is improved to improve the accuracy of bare soil recognition.

[0054] Secondly, shadows are usually closely related to the luminance information of the image. Therefore, by performing HIS transformation on each frame of the image, the luminance channel is extracted, specifically:

[0055] Perform the forward HIS transformation on each frame of the image to convert the image from the RGB space to the HIS space and extract the I component; segment each frame of the image to obtain multiple image blocks;

[0056] In this embodiment, the I components of all pixel points in each frame of the image are used as the input, and the watershed segmentation algorithm is used to segment each frame of the image to obtain multiple image blocks. Among them, the watershed segmentation algorithm is a well-known technology and will not be elaborated here.

[0057] It should be noted that the forward HIS transformation is a well-known technology and will not be elaborated here; among them, in the HIS space, the H component represents hue, the I component represents luminance, and the S component represents saturation.

[0058] Furthermore, since the shadow area in the image is only affected by scattered light, the luminance value of the shadow area is relatively low, while the non-shadow area is directly illuminated by light, so the luminance value of the non-shadow area is relatively high. Therefore, the image blocks are initially screened through the luminance information of each image block, specifically:

[0059] Calculate the mean value of the I components of all pixel points within each image block, denoted as the average luminance;

[0060] Adopt a threshold segmentation algorithm to obtain the segmentation threshold of the average luminance of all image blocks in each frame of the image. Denote the image blocks with the average luminance less than or equal to the segmentation threshold as each dark block, and vice versa, denote them as each bright block;

[0061] In this embodiment, the Otsu threshold segmentation algorithm is adopted to obtain the segmentation threshold. Among them, the Otsu threshold segmentation algorithm is a well-known technology and will not be elaborated here. As other implementation manners, implementers can adopt other methods of existing technologies, for example, constructing a histogram method, etc. This embodiment does not make special restrictions on this.

[0062] It should be noted that if the average luminance of the image block is less than or equal to the segmentation threshold, it indicates that the corresponding image block is more likely to be a shadow area.

[0063] Secondly, since there are dark objects in each frame of the image, including but not limited to dark soil, dark building surfaces, etc., the dark objects themselves have a low reflectivity and may show similarity in luminance with the shadow area, resulting in difficulty in accurately distinguishing the shadow area and the non-shadow area only relying on the segmentation threshold. It is necessary to further analyze whether the image block is a shadow area.

[0064] In addition, since the shadow area usually has relatively small luminance changes due to the diffuse reflection and occlusion of light, that is, the luminance change of the shadow area is relatively smooth and mainly shows low-frequency components in the frequency domain of the image. While the non-shadow area is directly irradiated by light, and factors such as the material, color, and texture of the object surface will cause the luminance of pixel points in the image to change rapidly, presenting more edges and detail features, and thus showing more high-frequency components in the frequency domain of the image. Therefore, perform frequency domain analysis on the image block to reflect the possibility that the corresponding image block belongs to the shadow area.

[0065] Among them, for the convenience of further screening the dark blocks later and analyzing the possibility that the bright blocks and dark blocks belong to the shadow area, therefore, calculate the first evaluation value of each image block, including the first evaluation value of each dark block and the first evaluation value of each bright block. The step flowchart of the method for obtaining the first evaluation value of each image block provided in the embodiment of the present application is as Figure 2 shown, and specifically includes:

[0066] Perform frequency domain analysis on the I components of all pixel points within each image block to obtain a spectrogram;

[0067] In this embodiment, the fast Fourier transform is adopted to obtain the spectrogram. Among them, the fast Fourier transform is a well-known technology and will not be elaborated here.

[0068] Extract the low-frequency energy in the spectrogram through a low-pass filter, calculate the proportion of the low-frequency energy in the spectrogram, and obtain the low-frequency energy ratio of each image block.

[0069] It should be noted that the low-pass filter is a well-known technology and will not be elaborated here. As other implementation manners, the implementer can calculate the low-frequency energy ratio by other methods. For example, extract the high-frequency energy in the spectrogram through a high-pass filter, calculate the proportion of the high-frequency energy in the spectrogram, and thus indirectly obtain the low-frequency energy ratio. This embodiment does not make special restrictions on this.

[0070] It should be noted that the larger the low-frequency energy ratio, the larger the proportion of the low-frequency information component in the corresponding image block, the smoother the change in brightness within the image block, and the more likely it belongs to the shadow area.

[0071] Furthermore, if the image block belongs to the shadow area, the closer it is to the shadow center, the more serious the light occlusion phenomenon is, and the weaker the light diffuse reflection phenomenon is. As a result, the brightness of the pixel points closer to the center position is smaller. Therefore, in the spatial domain, by analyzing the brightness change of the pixel points in each image block, calculate the brightness change rate to evaluate the possibility that the corresponding image block belongs to the shadow area. Specifically:

[0072] Calculate the distance between the position coordinates of each pixel point and the central pixel point within each image block, denoted as the relative distance.

[0073] In this embodiment, the distance is measured by calculating the Euclidean distance between the position coordinates of each pixel point and the central pixel point within each image block. As other implementation manners, the implementer can adopt other methods in the prior art, such as the Manhattan distance, etc. This embodiment does not make special restrictions on this.

[0074] Calculate the difference in the I component between each pixel point and the central pixel point within each image block, denoted as the brightness difference.

[0075] In this embodiment, calculate the absolute value of the difference in the I component between each pixel point and the central pixel point within each image block, denoted as the brightness difference.

[0076] Form a two-dimensional array with the relative distance and the brightness difference, and obtain the slope after linear fitting of all two-dimensional arrays within each image block, denoted as the brightness change rate of each image block.

[0077] In this embodiment, the least squares method is used for linear fitting. Among them, the least squares method is a well-known technology and will not be elaborated here.

[0078] It should be noted that the larger the brightness change rate, the weaker the brightness at the position closer to the center in the corresponding image block, and the more likely it belongs to the shadow area.

[0079] Further, based on the low-frequency energy ratio and the brightness change rate, a first evaluation value is determined, specifically as follows:

[0080] The product of the low-frequency energy ratio and the brightness change rate is used as the first evaluation value for each image block;

[0081] It should be noted that if an image block belongs to a shadow area, due to occlusion or light diffuse reflection in the shadow area, the image block contains more low-frequency component information, so the larger the low-frequency energy ratio; at the same time, since the shadow area is not directly irradiated by light and is only affected by light diffuse reflection, and there is energy loss during the diffuse reflection process, the closer to the center of the shadow area, the smaller the brightness of the shadow area, so the larger the brightness change rate. Therefore, the larger the obtained first evaluation value, the more likely the corresponding image block belongs to the shadow area.

[0082] Thus, the first evaluation value of each image block is obtained.

[0083] Step 3: Analyze the distribution difference of the brightness of the pixel points located at the boundary within each image block in the local area, and the texture characteristics of the local areas in different image blocks on both sides of the pixel points located at the boundary, and calculate the second evaluation value of each image block.

[0084] Further, due to the influence of light diffuse reflection, the boundary of the shadow area is in a gradual change state, so the boundary between the shadow area and the non-shadow area is relatively blurred, while the boundary of the area of dark building objects or trees and other objects is relatively obvious. Therefore, the brightness distribution difference is calculated through the brightness change of the pixel points at the boundary, specifically as follows:

[0085] The pixel points located at the boundary within each image block are denoted as boundary pixel points; a local window with a preset size is constructed centered on any boundary pixel point within each image block;

[0086] In this embodiment, a 5×5 local window is constructed. As other implementation manners, the implementer can set it according to the actual situation.

[0087] Cluster the I components of all pixel points in the local window to obtain two clustering clusters;

[0088] In this embodiment, the K-means algorithm is used for clustering. Among them, the K-means algorithm is a well-known technology and will not be elaborated here. As other implementation manners, the implementer can adopt other methods of existing technologies, such as the DBSCAN clustering algorithm, etc. This embodiment does not make special restrictions on this.

[0089] Calculate the difference in the I components between the clustering centers of the two clustering clusters, and denote it as the brightness distribution difference of the any boundary pixel point;

[0090] In this embodiment, the absolute value of the difference in the I component between the cluster centers of two clusters is calculated and denoted as the luminance distribution difference of any boundary pixel point.

[0091] It should be noted that the smaller the luminance distribution difference, the blurrier the shadow boundary at the position corresponding to the boundary pixel point, and thus the more likely it belongs to the boundary of the shadow area.

[0092] Secondly, since the occluding shadow only changes the luminance of the shadow area without changing the texture features of the shadow area, and the two sides of the boundary between the shadow area and the non-shadow area usually belong to the same type of object, the shadow area shows a high similarity with its adjacent image blocks at the boundary. Therefore, the texture consistency is calculated as follows:

[0093] The remaining image blocks adjacent to any boundary pixel point in each image block are denoted as adjacent image blocks;

[0094] Windows with a preset size are constructed in the two image blocks on both sides of any boundary pixel point. The window on one side within the image block to which the boundary pixel point belongs is denoted as the internal window, and the window on one side within the adjacent image block is denoted as the external window, where both the internal window and the external window contain the boundary pixel point;

[0095] In this embodiment, a window with a preset size of 3×3 is constructed. As other implementation manners, the implementer can set it according to the actual situation.

[0096] Each frame of the image is grayscaled to obtain a grayscale image; the entropy of the gray-level co-occurrence matrix of the corresponding region of the internal window in the grayscale image is calculated as the texture feature value of the internal window;

[0097] The entropy of the gray-level co-occurrence matrix of the corresponding region of the external window in the grayscale image is calculated as the texture feature value of the external window;

[0098] It should be noted that the gray-level co-occurrence matrix is a well-known technology and will not be elaborated here. As other implementation manners, the implementer can use other methods of the existing technology to extract texture features, such as the LBP (Local binary patterns) method, etc. This embodiment does not make special restrictions on this.

[0099] The difference in texture feature values between the internal window and the external window is used as the texture consistency of any boundary pixel point;

[0100] In this embodiment, the absolute value of the difference in texture feature values between the internal window and the external window is used as the texture consistency of any boundary pixel point.

[0101] It should be noted that the smaller the texture consistency, the more consistent the textures on both sides of the corresponding boundary pixel points, and the more likely it is that both sides belong to the same type of object. Therefore, the lower brightness of the corresponding image block is more likely to be caused by occluding shadows.

[0102] Furthermore, based on the brightness distribution difference and the texture consistency, a second evaluation value is determined, specifically as follows:

[0103] The sum of the products of the brightness distribution differences and the texture consistencies of all boundary pixel points within each image block is used as the second evaluation value of each image block;

[0104] In this embodiment, the calculation formula for the second evaluation value of each image block in each frame of image is: , where is the second evaluation value of the th image block in the th frame of image, is the brightness distribution difference of the th boundary pixel point within the th image block in the th frame of image, is the texture consistency of the th boundary pixel point within the th image block in the th frame of image, is the th frame of image, is the number of all boundary pixel points within the

[0105] It should be noted that when the image block is a shadow area, due to the influence of light diffuse reflection, the boundary between the shadow area and the non-shadow area is in a gradual change state, so the brightness distribution difference is smaller; at the same time, since the shadow area generated by occlusion only changes the brightness of the shadow area and does not change the texture characteristics of the shadow area, therefore, the two sides of the boundary between the shadow area and the non-shadow area usually belong to the same type of object, so the obtained texture consistency is smaller, and thus the second evaluation value is smaller, indicating that the corresponding image block is more likely to belong to the shadow area.

[0106] Thus, the second evaluation value of each image block is obtained.

[0107] Step 4: Fuse the first evaluation value and the second evaluation value to obtain the discrimination coefficient of each image block; based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of image, screen all dark blocks in each frame of image to obtain each shadow block; perform color space inverse transformation after removing the shadows from all shadow blocks in each frame of image to obtain each frame of image after shadow removal, and train the target detection model through all the images after shadow removal in the image set to perform bare soil recognition on the real-time monitoring screen image.

[0108] Based on the first evaluation value and the second evaluation value, determine a discrimination coefficient, specifically:

[0109] Take the ratio of the first evaluation value to the second evaluation value as the discrimination coefficient of each image block;

[0110] It should be noted that the larger the discrimination coefficient, the greater the possibility that the corresponding image block is a shadow area.

[0111] Thus, the discrimination coefficients of each bright block and each dark block can be obtained;

[0112] Furthermore, based on the discrimination coefficients of the bright blocks and the dark blocks, screen the dark blocks to obtain shadow blocks, specifically:

[0113] Record the upper quartile of the discrimination coefficients of all bright blocks in each frame of image as the judgment threshold;

[0114] Record the dark blocks in each frame of image with discrimination coefficients less than the judgment threshold as each shadow block;

[0115] Among them, the step flow chart of the method for obtaining each shadow block provided in the embodiments of the present application is as Figure 3 shown.

[0116] Secondly, according to the obtained shadow blocks, remove the shadows in the images, specifically:

[0117] After removing the shadows of all shadow blocks in each frame of image through the geodesic shadow removal algorithm, perform HIS inverse transformation to obtain each frame of image after shadow removal;

[0118] It should be noted that the geodesic shadow removal algorithm and the HIS inverse transformation are well-known technologies and will not be elaborated here.

[0119] Perform image annotation on each frame of image after shadow removal to form an image set;

[0120] In this embodiment, the Label Studio image annotation tool is used for image annotation. Among them, the Label Studio image annotation tool is a well-known technology and will not be elaborated here.

[0121] Train the target detection model through the image set to obtain the trained target detection model;

[0122] Perform shadow removal on the monitoring screen images obtained by the drone in real time through the above method, identify the bare soil uncovered areas in the real-time screen images after shadow removal through the trained target detection model, perform real-time monitoring on them, and if there are bare soil uncovered areas in the real-time screen, perform target tracking and warning on them.

[0123] In this embodiment, the YoloV8 object detection algorithm is used for model training. Among them, the YoloV8 object detection algorithm is a well-known technology and will not be elaborated here. Secondly, if there is an uncovered bare soil area in the real-time video, the DeepSport algorithm is used to track the target. Among them, the DeepSport algorithm is a well-known technology and will not be elaborated here.

[0124] Based on the same inventive concept as the above method, an embodiment of the present application also provides a bare soil automatic recognition system based on real-time images of an unmanned aerial vehicle, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above methods for automatically recognizing bare soil based on real-time images of an unmanned aerial vehicle.

[0125] It should be understood that although Figure 1 the steps in the flowchart of Figure 1 are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0126] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as the scope recorded in this specification.

[0127] The above-described embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limitations on the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made. Therefore, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application all belong to the protection scope of the technical solution of the present application.

Claims

1. A bare soil automatic identification method based on real-time images from unmanned aerial vehicles, characterized in that: The method comprises the following steps: Through the drone inspection process, the monitoring screen images are obtained in real time, and several frames of images containing bare soil uncovered areas are obtained to form an image set; Performing color space transformation on each frame of the image in the image set to extract brightness information, where the brightness information is the I component after the space transformation, and segmenting each frame of the image based on the brightness of each frame of the image to obtain multiple image blocks; Based on the average brightness of the pixels in each image block, all image blocks in each frame of the image are divided into dark blocks and bright blocks; Extract low-frequency energy in the spectrum of each image block through a low-pass filter, calculate the proportion of low-frequency energy in the spectrum, and obtain the low-frequency energy ratio of each image block; calculate the difference of I component between each pixel point and the central pixel point in each image block, and record it as brightness difference; calculate the first evaluation value of each image block based on the low-frequency energy ratio and the change trend of the brightness difference in the spatial domain; The pixel points at the boundary of each image block are recorded as boundary pixel points; a local window of a preset size is constructed with any boundary pixel point in each image block as the center; Cluster the I components of all pixels in the local window to obtain two clusters; calculate the difference in the I components between the cluster centers of the two clusters, which is recorded as the brightness distribution difference; Analyze the difference in texture features of local areas in the image blocks on both sides of any boundary pixel point to obtain the texture consistency of any boundary pixel point; The second evaluation value is the sum of the products of the brightness distribution difference and the texture consistency of all boundary pixels in each image block; fusing the first evaluation value and the second evaluation value to obtain a discrimination coefficient of each image block; Based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of image, all dark blocks in each frame of image are screened to obtain each shadow block; After removing the shadows of all shadow blocks in each frame of the image, an inverse color space transform is performed to obtain each frame of the image after the shadows are removed. The target detection model is trained through all the images after the shadows are removed in the image set, and bare soil recognition is performed on the real-time monitoring screen images.

2. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The color space transformation is performed on each frame of the image in the image set to extract the brightness information, including: performing HIS positive transformation on each frame of the image, converting the image from the RGB space to the HIS space, and extracting the I component.

3. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 2, characterized in that: The step of dividing all image blocks in each frame of image into dark blocks and bright blocks includes: Calculate the mean value of the I component of all pixels in each image block and record it as the average brightness; The segmentation threshold of the average brightness of all image blocks in each frame of image is obtained, and image blocks with average brightness less than or equal to the segmentation threshold are recorded as dark blocks, and vice versa, they are recorded as bright blocks.

4. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 2, characterized in that: The calculating the first evaluation value of each image block comprises: Calculate the distance between the position coordinates of each pixel point and the central pixel point in each image block, and record it as the relative distance; form the relative distance and the brightness difference into a two-dimensional array, and obtain the slope of all two-dimensional arrays in each image block after linear fitting, and record it as the brightness change rate; The first evaluation value is the product of the low-frequency energy ratio and the brightness change rate.

5. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The obtaining of the texture consistency of any boundary pixel point comprises: The remaining image blocks adjacent to any boundary pixel point in each image block are recorded as adjacent image blocks; Constructing windows of preset sizes in two image blocks on both sides of any boundary pixel point, respectively, recording the window located on one side of the image block to which any boundary pixel point belongs as an internal window, and recording the window located on one side of the adjacent image block as an external window, wherein both the internal window and the external window contain any boundary pixel point; Graying each frame of the image to obtain a grayscale image; extracting texture features of the internal window and the external window in the grayscale image corresponding to the region to obtain texture feature values ​​of the internal window and the external window respectively; The difference in the texture feature value between the inner window and the outer window is used as the texture consistency of any boundary pixel point.

6. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The discrimination coefficient of each image block is a ratio of the first evaluation value to the second evaluation value.

7. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The further acquisition method of each shadow block is as follows: The upper quartile of the discrimination coefficients of all bright blocks in each frame of image is calculated and recorded as the discrimination threshold; the dark blocks in each frame of image whose discrimination coefficients are less than the discrimination threshold are recorded as shadow blocks.

8. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The step of obtaining each frame of image after shadow removal includes: removing shadows from all shadow blocks in each frame of image by using a geodesic shadow removal algorithm and then performing an HIS inverse transformation to obtain each frame of image after shadow removal.

9. A bare soil automatic identification system based on real-time images from unmanned aerial vehicles, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method for automatic identification of bare soil based on real-time images of unmanned aerial vehicles as described in any one of claims 1 to 8 are implemented.

Images