Automatic bare soil identification method and system based on real-time image of unmanned aerial vehicle

Abstract

The invention relates to the technical field of image recognition, in particular to a bare soil automatic recognition method and system based on unmanned aerial vehicle real-time images, and the method comprises the steps: obtaining a monitoring picture image in real time through an unmanned aerial vehicle inspection process, and obtaining a plurality of frames of images containing a bare soil uncovered area, and forming an image set; obtaining a plurality of image blocks; dividing all the image blocks into dark blocks and bright blocks; calculating a first evaluation value, a second evaluation value and a discrimination coefficient of each image block; screening all the dark blocks to obtain shadow blocks; and performing shadow removal on all shadow blocks in each frame of image to obtain each frame of image after shadow removal, training the target detection model, and performing bare soil identification on a real-time monitoring picture image. According to the method, the shadow generated by shielding in the image can be accurately removed, the accuracy of identifying bare soil in the image by the target detection model is improved, and accurate identification of the bare soil area in the real-time monitoring image is facilitated.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and system for automatically identifying bare soil based on real-time images from unmanned aerial vehicles. Background Art

[0002] The rapid advancement of urbanization is a process of regional space transformation. In the process of urbanization construction, there are often uncovered bare land surfaces. As the main source of pollution in the urban environment, bare land surfaces are very likely to generate dust if not handled in time, which in turn harms public health. Therefore, it is necessary to identify and supervise bare land surfaces within the city.

[0003] At present, the monitoring and identification of bare soil on the urban ground mainly relies on manual inspections. However, this method has obvious defects. It not only consumes human resources, but also causes the bare soil surface in the city to not be discovered in time. When using image recognition technology to identify bare soil, due to the complex internal environment of the city, the real-time image of the drone inspection will have shadow areas due to various occlusions, resulting in a low accuracy rate of bare soil identification in the image, resulting in a high false detection rate of bare soil identification. Summary of the invention

[0004] In order to solve the above technical problems, a bare soil automatic identification method and system based on real-time images of unmanned aerial vehicles are provided to solve the existing problems.

[0005] The solution to the technical problem of this application is to provide a method and system for automatically identifying bare soil based on real-time images of unmanned aerial vehicles, including the following steps: In a first aspect, an embodiment of the present application provides a method for automatically identifying bare soil based on real-time images of a drone, the method comprising the following steps: Through the drone inspection process, the monitoring screen images are obtained in real time, and several frames of images containing bare soil uncovered areas are obtained to form an image set; Performing color space transformation on each frame of the image in the image set, extracting brightness information, and segmenting each frame of the image based on the brightness of each frame of the image to obtain multiple image blocks; Based on the average brightness of the pixels in each image block, all image blocks in each frame of the image are divided into dark blocks and bright blocks; Calculate the first evaluation value of each image block by using the energy distribution characteristics of the brightness of the pixel points in each image block in the frequency domain and the change trend of the brightness difference of different pixel points in the spatial domain; Analyze the distribution difference of brightness of pixels located at the boundary in the local area of ​​each image block, and the texture characteristics of the local areas in different image blocks on both sides of the pixels located at the boundary, and calculate the second evaluation value of each image block; fusing the first evaluation value and the second evaluation value to obtain a discrimination coefficient of each image block; Based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of image, all dark blocks in each frame of image are screened to obtain each shadow block; After removing the shadows of all shadow blocks in each frame of the image, an inverse color space transform is performed to obtain each frame of the image after the shadows are removed. The target detection model is trained through all the images after the shadows are removed in the image set, and bare soil recognition is performed on the real-time monitoring screen images.

[0006] Preferably, performing color space transformation on each frame of the image in the image set to extract brightness information includes: performing HIS forward transformation on each frame of the image to convert the image from RGB space to HIS space and extracting the I component.

[0007] Preferably, dividing all image blocks in each frame of image into dark blocks and bright blocks comprises: Calculate the mean value of the I component of all pixels in each image block and record it as the average brightness; The segmentation threshold of the average brightness of all image blocks in each frame of image is obtained, and image blocks with average brightness less than or equal to the segmentation threshold are recorded as dark blocks, and vice versa, they are recorded as bright blocks.

[0008] Preferably, the calculating the first evaluation value of each image block includes: Perform frequency domain analysis on the I components of all pixels in each image block to obtain a spectrum, extract the low-frequency energy in the spectrum, and calculate the proportion of the low-frequency energy in the spectrum, which is recorded as the low-frequency energy ratio; Calculate the distance between the position coordinates of each pixel point and the central pixel point in each image block and the difference in I component, which are recorded as relative distance and brightness difference respectively; form the relative distance and the brightness difference into a two-dimensional array, obtain the slope of all two-dimensional arrays in each image block after linear fitting, and record it as brightness change rate; The first evaluation value is the product of the low-frequency energy ratio and the brightness change rate.

[0009] Preferably, the calculating the second evaluation value of each image block includes: The pixel points at the boundary of each image block are recorded as boundary pixel points; a local window of a preset size is constructed with any boundary pixel point in each image block as the center; Cluster the I components of all pixels in the local window to obtain two clusters; calculate the difference in the I components between the cluster centers of the two clusters, which is recorded as the brightness distribution difference; Analyze the difference in texture features of local areas in the image blocks on both sides of any boundary pixel point to obtain the texture consistency of any boundary pixel point; The second evaluation value is the sum of the products of the brightness distribution differences of all boundary pixels in each image block and the texture consistency.

[0010] Preferably, obtaining the texture consistency of any boundary pixel point includes: The remaining image blocks adjacent to any boundary pixel point in each image block are recorded as adjacent image blocks; Constructing windows of preset sizes in two image blocks on both sides of any boundary pixel point, respectively, recording the window located on one side of the image block to which any boundary pixel point belongs as an internal window, and recording the window located on one side of the adjacent image block as an external window, wherein both the internal window and the external window contain any boundary pixel point; Graying each frame of the image to obtain a grayscale image; extracting texture features of the internal window and the external window in the grayscale image corresponding to the region to obtain texture feature values ​​of the internal window and the external window respectively; The difference in texture feature values ​​between the inner window and the outer window is used as the texture consistency of any boundary pixel point.

[0011] Preferably, the discrimination coefficient of each image block is a ratio of the first evaluation value to the second evaluation value.

[0012] Preferably, a further method for obtaining each shadow block is: The upper quartile of the discrimination coefficients of all bright blocks in each frame of image is calculated and recorded as the discrimination threshold; the dark blocks in each frame of image whose discrimination coefficients are less than the discrimination threshold are recorded as shadow blocks.

[0013] Preferably, obtaining each frame of image after shadow removal includes: removing shadows from all shadow blocks in each frame of image by using a geodesic shadow removal algorithm and then performing an HIS inverse transform to obtain each frame of image after shadow removal.

[0014] In the second aspect, an embodiment of the present application also provides a system for automatically identifying bare soil based on real-time images of unmanned aerial vehicles, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned methods for automatically identifying bare soil based on real-time images of unmanned aerial vehicles are implemented.

[0015] This application has at least the following beneficial effects: The present application extracts the brightness in each frame of the image by performing a color space transformation on each frame of the image, thereby dividing each frame of the image into multiple image blocks, and distinguishing dark blocks and bright blocks according to the average brightness of the pixels in the image blocks. The beneficial effect is that the image blocks are preliminarily screened according to their brightness levels to preliminarily clarify the shadow area and the non-shadow area; secondly, the first evaluation value of each image block is calculated, which has the beneficial effect of considering the low-frequency energy contained in the image block and the variation characteristics between the brightness and distance of different pixels, and further reflecting the possibility that the image block is a shadow area through the variation characteristics of the brightness of the pixel points in the frequency domain and the spatial domain; the second evaluation value of each image block is calculated, which has the beneficial effect of considering the distribution characteristics of the brightness in the local range of the pixel points located at the boundary of the image block, so as to illustrate the gradual change of the brightness of the boundary pixel points, and analyze the consistency of the texture features of the local areas in the two adjacent image blocks on both sides of the pixel points located at the boundary, reflecting the possibility that the local areas on both sides belong to the same type of object, thereby Reflect the possibility that the corresponding image block is a shadow area; obtain the discrimination coefficient of each image block, and based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of the image, screen all dark blocks in each frame of the image to obtain each shadow block. Its beneficial effect is that through the discrimination coefficient of the bright block corresponding to the non-shadow area, the dark blocks that may be the shadow area are further screened, and the interference of dark buildings or other objects with low brightness inside the city on the shadow area is eliminated, so as to improve the distinction between the shadow area and the non-shadow area in the image; after removing the shadows of all shadow blocks in each frame of the image, an inverse color space transform is performed to obtain each frame of the image after the shadows are removed, and the target detection model is trained through all images after the shadows are removed in the image set, and the real-time monitoring screen image is subjected to bare soil recognition. Its beneficial effect is that by removing the shadows caused by occlusion in the image and then training the target detection model, the accuracy of the target detection model in identifying bare soil in the image is improved, which is conducive to the accurate recognition of bare soil areas in the real-time monitoring screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The following is a further detailed description of a bare soil automatic identification method based on real-time images of unmanned aerial vehicles in the present application in conjunction with the accompanying drawings.

[0017] Figure 1 A flowchart of a method for automatically identifying bare soil based on real-time images of drones provided in an embodiment of the present application; Figure 2 A flowchart of the steps of a method for obtaining a first evaluation value of each image block provided in an embodiment of the present application; Figure 3 A flowchart of the steps of the method for obtaining each shadow block provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clear, the following is a further detailed description of a bare soil automatic identification method and system based on real-time drone images proposed in the present application in combination with the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0020] See also Figure 1 , which shows a flowchart of a method for automatically identifying bare soil based on real-time images of drones provided by an embodiment of the present application, the method comprising the following steps: Step 1: Through the drone inspection process, real-time monitoring screen images are obtained, as well as several frames of images containing bare soil uncovered areas to form an image set.

[0021] Through drone inspection, video data containing uncovered bare soil is captured from different heights and angles. After key frames are extracted from the video data, several frames of images containing uncovered bare soil areas are selected through operations such as rotation, translation and scaling to form an image set. In this embodiment, the ffmpeg tool is used to extract key frames from the video data, and then rotate, translate, zoom, and other operations are performed, and each frame of the image is obtained through manual screening. The ffmpeg tool is a well-known technology and will not be described in detail here.

[0022] The real-time monitoring video during drone inspection is transmitted to the streaming media server, and the key frames are extracted using the ffmpeg tool to obtain the monitoring screen image in real time.

[0023] At this point, the monitoring screen images and image sets are obtained.

[0024] Step 2, perform color space transformation on each frame of the image in the image set, extract brightness information, and segment each frame of the image based on the brightness of each frame of the image to obtain multiple image blocks; based on the average level of brightness of the pixels in each image block, divide all image blocks in each frame of the image into dark blocks and bright blocks; calculate the first evaluation value of each image block through the energy distribution characteristics of the brightness of the pixels in each image block in the frequency domain and the changing trend of the brightness difference of different pixels in the spatial domain.

[0025] Due to the complex internal environment of the city, objects such as buildings and trees will produce shadows in the images taken by drones. These shadows will interfere with the recognition of bare soil and reduce the accuracy of recognition. Therefore, in order to improve the accuracy of bare soil recognition, it is necessary to preprocess the image before bare soil recognition to remove the influence of shadows. The traditional geodesic shadow removal algorithm can effectively handle larger half-shadow areas and enhance the contrast of shadow areas by adjusting the multiplicative factor. In practical applications, when distinguishing between shadow areas and non-shadow areas in the image, it is generally necessary to manually set the division threshold, which cannot be flexibly adjusted according to the actual situation in the image, resulting in insufficient flexibility and adaptability of the algorithm. In order to solve the limitations of the traditional algorithm, the geodesic shadow removal algorithm is improved to improve the accuracy of bare soil recognition.

[0026] Secondly, shadows are usually closely related to the brightness information of the image. Therefore, the brightness channel is extracted by performing HIS transformation on each frame of the image, specifically: Perform HIS positive transformation on each frame of image, convert the image from RGB space to HIS space, and extract the I component; segment each frame of image to obtain multiple image blocks; In this embodiment, the I component of all pixels in each frame of image is taken as input, and each frame of image is segmented using a watershed segmentation algorithm to obtain multiple image blocks, wherein the watershed segmentation algorithm is a well-known technology and will not be described in detail here.

[0027] It should be noted that HIS positive transformation is a well-known technology and will not be described in detail here; in the HIS space, the H component represents the hue, the I component represents the brightness, and the S component represents the saturation.

[0028] Furthermore, since the shadow area in the image is only affected by scattered light, the brightness value of the shadow area is low, while the non-shadow area is directly illuminated by light, so the brightness value of the non-shadow area is high. Therefore, the image blocks are initially screened according to the brightness information of each image block, specifically: Calculate the mean value of the I component of all pixels in each image block and record it as the average brightness; A threshold segmentation algorithm is used to obtain a segmentation threshold of the average brightness of all image blocks in each frame of the image, and image blocks whose average brightness is less than or equal to the segmentation threshold are recorded as dark blocks, and vice versa, are recorded as bright blocks; In this embodiment, the Otsu threshold segmentation algorithm is used to obtain the segmentation threshold, wherein the Otsu threshold segmentation algorithm is a well-known technology and will not be described in detail here. As other implementation methods, the implementer can adopt other methods of the prior art, such as constructing a histogram method, etc. This embodiment does not impose any special restrictions on this.

[0029] It should be noted that if the average brightness of an image block is less than or equal to the segmentation threshold, it means that the possibility that the corresponding image block is a shadow area is greater.

[0030] Secondly, since there are dark objects in each frame of the image, including but not limited to dark soil, dark building surfaces, etc., the dark objects themselves have a lower reflectivity and may show similarity in brightness with the shadow area, making it difficult to accurately distinguish between the shadow area and the non-shadow area by relying solely on the segmentation threshold. Further analysis is needed to determine whether the image block is a shadow area.

[0031] In addition, since the shadow area usually has a relatively small brightness change due to diffuse reflection and occlusion of light, that is, the brightness change in the shadow area is relatively smooth, and it mainly appears as a low-frequency component in the frequency domain of the image, while the non-shadow area is directly exposed to light, and factors such as the material, color and texture of the object surface will cause the brightness of the pixels in the image to change rapidly, showing more edges and details, and then showing more high-frequency components in the frequency domain of the image. Therefore, the frequency domain analysis of the image block is performed to reflect the possibility that the corresponding image block belongs to the shadow area.

[0032] In order to facilitate further screening of dark blocks in the future and analyze the possibility of bright blocks and dark blocks belonging to the shadow area, the first evaluation value of each image block is calculated, including the first evaluation value of each dark block and the first evaluation value of each bright block. The step flow chart of the method for obtaining the first evaluation value of each image block provided in the embodiment of the present application is as follows: Figure 2 As shown, specifically including: Perform frequency domain analysis on the I components of all pixels in each image block to obtain a spectrum diagram; In this embodiment, a fast Fourier transform is used to obtain a frequency spectrum diagram, wherein the fast Fourier transform is a well-known technology and will not be described in detail herein.

[0033] The low-frequency energy in the spectrum is extracted by a low-pass filter, the proportion of the low-frequency energy in the spectrum is calculated, and the low-frequency energy ratio of each image block is obtained; It should be noted that the low-pass filter is a well-known technology and will not be elaborated here. As other implementation methods, the implementer can calculate the low-frequency energy ratio by other methods. For example, the high-frequency energy in the spectrum graph can be extracted by a high-pass filter, and the proportion of high-frequency energy in the spectrum graph can be calculated, thereby indirectly obtaining the low-frequency energy ratio. This embodiment does not impose any special restrictions on this.

[0034] It should be noted that, the larger the low-frequency energy ratio is, the larger the proportion of low-frequency information components in the corresponding image block is, and the smoother the change of brightness in the image block is, and the more likely it is to belong to the shadow area.

[0035] Furthermore, if the image block belongs to the shadow area, the closer to the center of the shadow, the more serious the light blocking phenomenon, the weaker the diffuse reflection phenomenon of the light, and thus the brightness of the pixel points closer to the center position is smaller. Therefore, in the spatial domain, by analyzing the brightness change of the pixel points in each image block, the brightness change rate is calculated to evaluate the possibility that the corresponding image block belongs to the shadow area, specifically: Calculate the distance between the position coordinates of each pixel point in each image block and the central pixel point, and record it as the relative distance; In this embodiment, the distance is measured by calculating the Euclidean distance between the position coordinates of each pixel point in each image block and the central pixel point. As other implementation methods, the implementer can adopt other methods of the prior art, such as Manhattan distance, etc. This embodiment does not impose any special restrictions on this.

[0036] Calculate the difference in I component between each pixel in each image block and the central pixel, and record it as brightness difference; In this embodiment, the absolute value of the difference between the I component of each pixel point in each image block and the central pixel point is calculated and recorded as the brightness difference.

[0037] The relative distance and the brightness difference are combined into a two-dimensional array, and the slope of all the two-dimensional arrays in each image block after linear fitting is obtained, which is recorded as the brightness change rate of each image block; In this embodiment, the least square method is used for linear fitting, wherein the least square method is a well-known technology and will not be described in detail here.

[0038] It should be noted that, the greater the brightness change rate is, the weaker the brightness is at the center position of the corresponding image block, and the more likely it is to belong to the shadow area.

[0039] Further, based on the low-frequency energy ratio and the brightness change rate, a first evaluation value is determined, specifically: Taking the product of the low-frequency energy ratio and the brightness change rate as the first evaluation value of each image block; It should be noted that if the image block belongs to a shadow area, the shadow area is blocked or diffusely reflected by light, so that the image block contains more low-frequency component information, and the low-frequency energy ratio is larger; at the same time, since the shadow area is not directly exposed to light, but only diffusely reflected by light, and there is energy loss in the diffuse reflection process, the closer the shadow area is to the center, the smaller the brightness of the shadow area, and the greater the brightness change rate. Therefore, the larger the first evaluation value obtained, it means that the corresponding image block may belong to the shadow area.

[0040] At this point, the first evaluation value of each image block is obtained.

[0041] Step 3: Analyze the distribution difference of brightness in the local area of ​​the pixels located at the boundary in each image block, and the texture characteristics of the local areas in different image blocks on both sides of the pixels located at the boundary, and calculate the second evaluation value of each image block.

[0042] Furthermore, due to the influence of diffuse reflection of light, the boundary of the shadow area is in a gradual state, so the boundary between the shadow area and the non-shadow area is relatively fuzzy, while the boundary of the area of ​​dark-colored buildings or trees is relatively obvious. Therefore, the brightness distribution difference is calculated by the brightness change of the pixels at the boundary, which is specifically: The pixel points at the boundary of each image block are recorded as boundary pixel points; a local window of a preset size is constructed with any boundary pixel point in each image block as the center; In this embodiment, a 5×5 local window is constructed. As other implementation methods, the implementer can set it according to the actual situation.

[0043] Cluster the I components of all pixels in the local window to obtain two clusters; In this embodiment, K-means algorithm is used for clustering, wherein K-means algorithm is a well-known technology and will not be described in detail here. As other implementation methods, implementers can adopt other methods of the prior art, such as DBSCAN clustering algorithm, etc. This embodiment does not impose any special restrictions on this.

[0044] Calculate the difference of the I component between the cluster centers of the two clusters, and record it as the brightness distribution difference of any boundary pixel point; In this embodiment, the absolute value of the difference of the I component between the cluster centers of two clusters is calculated and recorded as the brightness distribution difference of any boundary pixel point.

[0045] It should be noted that the smaller the brightness distribution difference is, the more blurred the shadow boundary at the location of the corresponding boundary pixel point is, and the more likely it is to belong to the boundary of the shadow area.

[0046] Secondly, since the occlusion shadow only changes the brightness of the shadow area but does not change the texture characteristics of the shadow area, and the two sides of the boundary between the shadow area and the non-shadow area usually belong to the same type of objects, the shadow area shows a high similarity with its adjacent image blocks at the boundary. Therefore, the texture consistency is calculated as follows: The remaining image blocks adjacent to any boundary pixel point in each image block are recorded as adjacent image blocks; Constructing windows of preset sizes in two image blocks on both sides of any boundary pixel point, respectively, recording the window located on one side of the image block to which any boundary pixel point belongs as an internal window, and recording the window located on one side of the adjacent image block as an external window, wherein both the internal window and the external window contain any boundary pixel point; In this embodiment, a window of a preset size of 3×3 is constructed. As other implementation modes, the implementer can set it according to the actual situation.

[0047] Graying each frame of the image to obtain a grayscale image; calculating the entropy of the grayscale co-occurrence matrix of the corresponding area of ​​the internal window in the grayscale image as the texture eigenvalue of the internal window; Calculating the entropy of the gray level co-occurrence matrix of the area corresponding to the external window in the gray image as the texture eigenvalue of the external window; It should be noted that the gray level co-occurrence matrix is ​​a well-known technology and will not be described in detail here. As other implementation methods, implementers can use other methods of the prior art to extract texture features, such as the LBP (Local binary patterns) method, etc. This embodiment does not impose any special restrictions on this.

[0048] Taking the difference in texture feature values ​​between the inner window and the outer window as the texture consistency of any boundary pixel point; In this embodiment, the absolute value of the difference between the texture feature values ​​of the inner window and the outer window is used as the texture consistency of any boundary pixel point.

[0049] It should be noted that the smaller the texture consistency is, the more consistent the textures on both sides of the corresponding boundary pixel are, and the more likely the two sides belong to the same type of object, and the more likely the low brightness of the corresponding image block is caused by occlusion shadows.

[0050] Further, based on the brightness distribution difference and the texture consistency, a second evaluation value is determined, specifically: The sum of the products of the brightness distribution difference and the texture consistency of all boundary pixels in each image block is used as the second evaluation value of each image block; In this embodiment, the calculation formula of the second evaluation value of each image block in each frame of image is: ,in, For the Frame image The second evaluation value of the image block, For the Frame image The first The brightness distribution difference of the boundary pixels is For the Frame image The first The texture consistency of the boundary pixels, For the Frame image The number of all boundary pixels in an image block.

[0051] It should be noted that when the image block is a shadow area, due to the influence of diffuse reflection of light, the boundary between the shadow area and the non-shadow area is in a gradient state, and the brightness distribution difference is smaller; at the same time, since the shadow area caused by occlusion only changes the brightness of the shadow area, but does not change the texture characteristics of the shadow area, the two sides of the boundary between the shadow area and the non-shadow area usually belong to the same type of object, the smaller the obtained texture consistency, and thus the smaller the second evaluation value, the more likely the corresponding image block belongs to the shadow area.

[0052] At this point, the second evaluation value of each image block is obtained.

[0053] Step 4, fusing the first evaluation value and the second evaluation value to obtain the discrimination coefficient of each image block; based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of the image, screening all dark blocks in each frame of the image to obtain each shadow block; removing the shadows of all shadow blocks in each frame of the image and then performing an inverse color space transformation to obtain each frame of the image after the shadows are removed; training the target detection model through all the images in the image set after the shadows are removed, and performing bare soil recognition on the real-time monitoring screen image.

[0054] Based on the first evaluation value and the second evaluation value, a discrimination coefficient is determined, specifically: using a ratio of the first evaluation value to the second evaluation value as a discrimination coefficient of each image block; It should be noted that, the larger the discrimination coefficient is, the greater the possibility that the corresponding image block is a shadow area is.

[0055] Thus, the discrimination coefficient of each bright block and the discrimination coefficient of each dark block can be obtained; Further, based on the discrimination coefficients of the bright blocks and the dark blocks, the dark blocks are screened to obtain the shadow blocks, specifically: The upper quartile of the discrimination coefficient of all bright blocks in each frame of the image is recorded as the judgment threshold; The dark blocks in each frame image whose discrimination coefficient is less than the discrimination threshold are recorded as shadow blocks; The step flow chart of the method for obtaining each shadow block provided in the embodiment of the present application is as follows: Figure 3 shown.

[0056] Secondly, according to the obtained shadow block, remove the shadow in the image, specifically: All shadow blocks in each frame of image are removed by using geodesic shadow removal algorithm, and then HIS inverse transformation is performed to obtain each frame of image after shadow removal; It should be noted that the geodesic shadow removal algorithm and HIS inverse transform are well-known technologies and will not be described in detail here.

[0057] Annotate each frame of the image after shadow removal to form an image set; In this embodiment, the Label Studio image annotation tool is used for image annotation, wherein the Label Studio image annotation tool is a well-known technology and will not be described in detail here.

[0058] The target detection model is trained by using the image set to obtain a trained target detection model; The shadows of the monitoring images obtained by the UAV in real time are removed by the above method, and the bare soil uncovered areas in the real-time images after shadow removal are identified by the trained target detection model, and real-time monitoring is carried out. If there are bare soil uncovered areas in the real-time images, the targets are tracked and early warnings are issued.

[0059] In this embodiment, the YoloV8 target detection algorithm is used for model training, wherein the YoloV8 target detection algorithm is a well-known technology and will not be described in detail here. Secondly, if there is an uncovered area of ​​bare soil in the real-time picture, the target is tracked by the DeepSport algorithm, wherein the DeepSport algorithm is a well-known technology and will not be described in detail here.

[0060] Based on the same inventive concept as the above method, an embodiment of the present application also provides a bare soil automatic identification system based on real-time images of unmanned aerial vehicles, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned methods for automatic bare soil identification based on real-time images of unmanned aerial vehicles are implemented.

[0061] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0062] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the present application. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application without departing from the content of the technical solution of the present application, shall fall within the protection scope of the technical solution of the present application.

Claims

1. A bare soil automatic identification method based on real-time images from unmanned aerial vehicles, characterized in that: The method comprises the following steps: Through the drone inspection process, the monitoring screen images are obtained in real time, and several frames of images containing bare soil uncovered areas are obtained to form an image set; Performing color space transformation on each frame of the image in the image set, extracting brightness information, and segmenting each frame of the image based on the brightness of each frame of the image to obtain multiple image blocks; Based on the average brightness of the pixels in each image block, all image blocks in each frame of the image are divided into dark blocks and bright blocks; Calculate the first evaluation value of each image block by using the energy distribution characteristics of the brightness of the pixel points in each image block in the frequency domain and the change trend of the brightness difference of different pixel points in the spatial domain; Analyze the distribution difference of brightness of pixels located at the boundary in the local area of ​​each image block, and the texture characteristics of the local areas in different image blocks on both sides of the pixels located at the boundary, and calculate the second evaluation value of each image block; fusing the first evaluation value and the second evaluation value to obtain a discrimination coefficient of each image block; Based on the discrimination coefficients of all bright blocks and all dark blocks in each frame of image, all dark blocks in each frame of image are screened to obtain each shadow block; After removing the shadows of all shadow blocks in each frame of the image, an inverse color space transform is performed to obtain each frame of the image after the shadows are removed. The target detection model is trained through all the images after the shadows are removed in the image set, and bare soil recognition is performed on the real-time monitoring screen images.

2. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The color space transformation is performed on each frame of the image in the image set to extract the brightness information, including: performing HIS positive transformation on each frame of the image, converting the image from the RGB space to the HIS space, and extracting the I component.

3. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 2, characterized in that: The step of dividing all image blocks in each frame of image into dark blocks and bright blocks includes: Calculate the mean value of the I component of all pixels in each image block and record it as the average brightness; The segmentation threshold of the average brightness of all image blocks in each frame of image is obtained, and image blocks with average brightness less than or equal to the segmentation threshold are recorded as dark blocks, and vice versa, they are recorded as bright blocks.

4. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 2, characterized in that: The calculating the first evaluation value of each image block comprises: Perform frequency domain analysis on the I components of all pixels in each image block to obtain a spectrum, extract the low-frequency energy in the spectrum, and calculate the proportion of the low-frequency energy in the spectrum, which is recorded as the low-frequency energy ratio; Calculate the distance between the position coordinates of each pixel point and the central pixel point in each image block and the difference in I component, which are recorded as relative distance and brightness difference respectively; form the relative distance and the brightness difference into a two-dimensional array, obtain the slope of all two-dimensional arrays in each image block after linear fitting, and record it as brightness change rate; The first evaluation value is the product of the low-frequency energy ratio and the brightness change rate.

5. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 2, characterized in that: The calculating the second evaluation value of each image block comprises: The pixel points at the boundary of each image block are recorded as boundary pixel points; a local window of a preset size is constructed with any boundary pixel point in each image block as the center; Cluster the I components of all pixels in the local window to obtain two clusters; calculate the difference in the I components between the cluster centers of the two clusters, which is recorded as the brightness distribution difference; Analyze the difference in texture features of local areas in the image blocks on both sides of any boundary pixel point to obtain the texture consistency of any boundary pixel point; The second evaluation value is the sum of the products of the brightness distribution differences of all boundary pixels in each image block and the texture consistency.

6. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 5, characterized in that: The obtaining of the texture consistency of any boundary pixel point comprises: The remaining image blocks adjacent to any boundary pixel point in each image block are recorded as adjacent image blocks; Constructing windows of preset sizes in two image blocks on both sides of any boundary pixel point, respectively, recording the window located on one side of the image block to which any boundary pixel point belongs as an internal window, and recording the window located on one side of the adjacent image block as an external window, wherein both the internal window and the external window contain any boundary pixel point; Graying each frame of the image to obtain a grayscale image; extracting texture features of the internal window and the external window in the grayscale image corresponding to the region to obtain texture feature values ​​of the internal window and the external window respectively; The difference in the texture feature value between the inner window and the outer window is used as the texture consistency of any boundary pixel point.

7. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The discrimination coefficient of each image block is a ratio of the first evaluation value to the second evaluation value.

8. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The further acquisition method of each shadow block is as follows: The upper quartile of the discrimination coefficients of all bright blocks in each frame of image is calculated and recorded as the discrimination threshold; the dark blocks in each frame of image whose discrimination coefficients are less than the discrimination threshold are recorded as shadow blocks.

9. The method for automatically identifying bare soil based on real-time images of unmanned aerial vehicles according to claim 1, characterized in that: The step of obtaining each frame of image after shadow removal includes: removing shadows from all shadow blocks in each frame of image by using a geodesic shadow removal algorithm and then performing an HIS inverse transformation to obtain each frame of image after shadow removal.

10. A bare soil automatic identification system based on real-time images from unmanned aerial vehicles, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method for automatic identification of bare soil based on real-time images of unmanned aerial vehicles as described in any one of claims 1 to 9 are implemented.

Images