SAM-based aerial image ground feature segmentation method, apparatus and device, and medium

Abstract

The invention discloses an SAM-based aerial image ground object segmentation method, device and equipment and a medium, and the method comprises the steps: collecting a to-be-segmented agricultural scene image through an unmanned aerial vehicle, and obtaining an original unmanned aerial vehicle aerial image; preprocessing the original unmanned aerial vehicle aerial image to obtain a target unmanned aerial vehicle aerial image; obtaining segmentation prompt information on the aerial image of the target unmanned aerial vehicle, and inputting the segmentation prompt information and the aerial image of the target unmanned aerial vehicle into an image segmentation model for ground object segmentation to obtain a ground object segmentation result, the segmentation prompt information at least comprises any one of point segmentation prompt information, frame segmentation prompt information and text segmentation prompt information, and the image segmentation model is a model obtained by training and verifying an SAM model by using multi-scene sample data, sample prompt information and agricultural scene data. According to the invention, the segmentation efficiency and the segmentation precision of ground feature segmentation are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular, to a method, device, equipment and medium for ground object segmentation of aerial images based on SAM. Background Art

[0002] With the rapid development of unmanned aerial vehicle (UAV) technology, its aerial images are increasingly widely used in the agricultural field, especially in agricultural land monitoring, non-agriculturalization review and non-grainization review. However, there are obvious deficiencies in the existing technology for processing ground object segmentation of UAV aerial images. Traditional ground object segmentation methods, such as threshold-based segmentation methods and edge detection methods, have low segmentation accuracy in complex agricultural scenarios and far cannot meet the high-precision monitoring requirements; while the existing deep learning-based ground object segmentation methods have made some progress, but the model generalization ability is insufficient, the demand for training data is too high, and the actual application effect is limited; traditional reviews rely too much on manual interpretation, which is not only inefficient but also prone to errors and difficult to cope with the monitoring requirements of today's large-scale agricultural land. Therefore, there is an urgent need for an efficient and accurate ground object segmentation method to improve the level of agricultural resource management. Summary of the Invention

[0003] The embodiments of the present invention provide a method, device, equipment and medium for ground object segmentation of aerial images based on SAM, aiming to solve the problem that the segmentation efficiency and segmentation accuracy of the existing ground object segmentation methods are both low.

[0004] In the first aspect, the embodiments of the present invention provide a method for ground object segmentation of aerial images based on SAM, including: Collecting an agricultural scene image to be segmented by a UAV to obtain an original UAV aerial image; Preprocessing the original UAV aerial image to obtain a target UAV aerial image; Obtaining segmentation prompt information on the target UAV aerial image, and inputting the segmentation prompt information and the target UAV aerial image into an image segmentation model for ground object segmentation to obtain a ground object segmentation result, wherein the segmentation prompt information at least includes any one of point segmentation prompt information, box segmentation prompt information and text segmentation prompt information, and the image segmentation model is a model obtained by training and validating the SAM model using multi-scene sample data, sample prompt information and agricultural scene data.

[0005] In the second aspect, the embodiments of the present invention further provide a device for ground object segmentation of aerial images based on SAM, including: A collection unit, configured to collect an agricultural scene image to be segmented by a UAV to obtain an original UAV aerial image; A preprocessing unit, configured to preprocess the original UAV aerial image to obtain a target UAV aerial image; Obtain a segmentation unit, which is used to obtain segmentation prompt information on the target UAV aerial image, and input the segmentation prompt information and the target UAV aerial image into an image segmentation model for ground object segmentation to obtain a ground object segmentation result. Wherein, the segmentation prompt information at least includes any one of point segmentation prompt information, box segmentation prompt information, and text segmentation prompt information. The image segmentation model is a model obtained by training and validating the SAM model using multi-scene sample data, sample prompt information, and agricultural scene data.

[0006] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program, the above method is implemented.

[0007] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above method can be implemented.

[0008] An embodiment of the present invention provides a method, device, equipment, and medium for ground object segmentation of aerial images based on SAM. Among them, the method includes: collecting an agricultural scene image to be segmented by a UAV to obtain an original UAV aerial image; preprocessing the original UAV aerial image to obtain a target UAV aerial image; obtaining segmentation prompt information on the target UAV aerial image, and inputting the segmentation prompt information and the target UAV aerial image into an image segmentation model for ground object segmentation to obtain a ground object segmentation result. Wherein, the segmentation prompt information at least includes any one of point segmentation prompt information, box segmentation prompt information, and text segmentation prompt information. The image segmentation model is a model obtained by training and validating the SAM model using multi-scene sample data, sample prompt information, and agricultural scene data. The technical solution of the embodiment of the present invention preprocesses the UAV aerial image to obtain a target UAV aerial image before ground object segmentation, improving the accuracy of ground object segmentation; because the image segmentation model is a model obtained by training and validating the SAM model using multi-scene sample data, sample prompt information, and agricultural scene data, its generalization ability and robustness are relatively good, and it will also perform segmentation according to the segmentation prompt information during the ground object segmentation process, further improving the accuracy of ground object segmentation; because there is no need for manual interpretation during the entire ground object segmentation process, the segmentation efficiency of ground object segmentation is greatly improved. Description of the Drawings

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0010] Figure 1 It is a schematic flowchart of a method for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention; Figure 2 It is a schematic sub - flowchart of a method for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention; Figure 3 It is another schematic sub - flowchart of a method for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention; Figure 4 It is still another schematic sub - flowchart of a method for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention; Figure 5 It is another schematic flowchart of a method for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention; Figure 6 It is a schematic block diagram of a device for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention; Figure 7 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0011] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0012] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0013] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in this specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0014] It should also be further understood that the term "and / or" used in the specification and appended claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0015] As used in this specification and the appended claims, the term "if" may be construed, depending on the context, as "when...", "once", "in response to determining", or "in response to detecting". Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, to mean "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0016] Please refer to Figure 1 , Figure 1 is a schematic flowchart of the method for segmenting ground objects in aerial images based on SAM provided by an embodiment of the present invention. The method for segmenting ground objects in aerial images based on SAM will be described in detail below. As Figure 1 shown, the method includes the following steps S110 - S130.

[0017] S110. Obtain the original drone aerial image by collecting the agricultural scene image to be segmented through a drone.

[0018] In this embodiment, a camera is mounted on the drone. In the data collection stage, the drone flies at a height of 100 meters and a speed of 5 meters per second, and the 20 - megapixel camera mounted on it takes pictures at a frequency of 2 frames per second to ensure that the overlapping rate of the collected agricultural scene image to be segmented is more than 80%, generating an original drone aerial image with a resolution of 0.1 meter per pixel.

[0019] In one embodiment, as Figure 2 shown, step S110 may specifically include steps S111 - S115: S111. Obtain the drone aerial image by collecting the agricultural scene image to be segmented through a drone; S112. Perform grayscale conversion and noise reduction processing on the drone aerial image to generate a grayscale noise - reduced image, S113. Perform Laplace transform on the grayscale noise - reduced image to obtain the Laplace value of the image, and calculate the variance of the Laplace value of the image to obtain the image variance; S114. If the image variance is lower than the preset image variance, trigger a reshooting prompt, and return to execute the step of obtaining the drone aerial image by collecting the agricultural scene image to be segmented through a drone; S115. If the variance of the image is not lower than the preset image variance, then use the UAV aerial image as the original UAV aerial image.

[0020] In this embodiment, the UAV aerial image is grayscale-converted to reduce the amount of calculation, and the noise reduction process is specifically performed by Gaussian noise reduction. Understandably, after obtaining the image variance, it is judged whether the image variance is lower than the preset image variance. If it is lower, it is determined that the image is blurred and unclear, then a reshooting prompt is triggered, and the step of obtaining the UAV aerial image by collecting the agricultural scene image to be segmented by the UAV is returned and executed to avoid the influence of low-quality images on the ground object segmentation effect; if it is not lower, it is determined that the image is clear, then the UAV aerial image is used as the original UAV aerial image. It should be noted that in this embodiment, the preset image variance is 50. Understandably, in other embodiments, the preset image variance can also be set to other values, such as 55, which is set according to actual needs and will not be specifically limited here.

[0021] S120. Perform preprocessing on the original UAV aerial image to obtain a target UAV aerial image.

[0022] In this embodiment, after obtaining the original UAV aerial image, perform preprocessing on the original UAV aerial image to obtain a target UAV aerial image. It should be noted that image preprocessing, as a basic link of ground object segmentation, is directly related to the subsequent model performance. In this embodiment, the preprocessing includes three key steps: image correction, denoising processing, and image enhancement. Specifically, as Figure 3 shown, step S120 may specifically include steps S121 - S123: S121. Perform image correction on the original UAV aerial image to obtain a corrected aerial image; S122. Filter the corrected aerial image using the median filter algorithm to obtain a filtered aerial image; S123. Perform image enhancement on the filtered aerial image through the adaptive histogram equalization method to obtain the target UAV aerial image. It should be noted that image correction uses geometric correction technology based on perspective transformation. Understandably, performing image correction on the original UAV aerial image can effectively eliminate the distortion caused by height and angle changes during UAV shooting, ensuring that the matching accuracy is controlled within an error of less than 1 pixel. It should also be noted that the filtering window of the median filter algorithm is set to 5×5, and it can also be dynamically adjusted to 7×7 according to the noise intensity, making full use of its robustness to salt-and-pepper noise to reduce noise interference. More importantly, during the process of image enhancement, the clipping threshold of the adaptive histogram equalization method is set to 0.01, and the grid size is set to 8×8, which can effectively enhance the ground object edge and texture features and improve the detail recognition ability of the image segmentation model.

[0023] Further, step S121 specifically includes: extracting key feature points of the original UAV aerial image by using the SIFT algorithm to obtain aerial key points; calculating a perspective transformation matrix based on the aerial key points, and performing image correction on the UAV aerial image according to the perspective transformation matrix to obtain the corrected aerial image. It should be noted that the SIFT algorithm is a scale-invariant feature transform algorithm, which can solve the problems of scale, rotation, and illumination changes in image matching, extract local feature points with invariance, and is widely used in fields such as image stitching, target recognition, and 3D reconstruction. It should also be noted that the perspective transformation matrix is a homography matrix, and the homography matrix maps the original UAV aerial image from the original perspective to the target perspective, eliminating the perspective distortion caused by the tilt or height change of the UAV.

[0024] S130. Obtain the segmentation prompt information on the target UAV aerial image, and input the segmentation prompt information and the target UAV aerial image into an image segmentation model for ground object segmentation to obtain a ground object segmentation result, where the segmentation prompt information includes at least any one of point segmentation prompt information, box segmentation prompt information, and text segmentation prompt information, and the image segmentation model is a model obtained by training and validating the SAM model by using multi-scene sample data, sample prompt information, and agricultural scene data.

[0025] In this embodiment, obtain the segmentation prompt information on the target UAV aerial image, where the segmentation prompt information includes at least any one of point segmentation prompt information, box segmentation prompt information, and text segmentation prompt information; it should be noted that the point segmentation prompt information is the prompt information generated when a point prompt is given, and a point prompt means that the user clicks on the center of the ground object to generate a corresponding segmentation mask; the box segmentation prompt information is the prompt information generated when a box prompt is given, and a box prompt means that the user is allowed to select a target area to complete precise segmentation; the text segmentation prompt information is the prompt information generated when a text prompt is given, and a text prompt means accepting the semantic description of the user for intelligent segmentation.

[0026] Further, after obtaining the segmentation prompt information, input the segmentation prompt information and the target UAV aerial image into an image segmentation model for ground object segmentation to obtain a ground object segmentation result, where the image segmentation model is a model obtained by training and validating the SAM model using multi-scenario sample data, sample prompt information, and agricultural scenario data. It should be noted that in this embodiment, SAM (Segment Anything Model) is used as the core model. This model is designed based on the Vision Transformer architecture and combined with a mask Transformer, and can efficiently process high-resolution images and generate fine segmentation masks. The prompt-based segmentation technology of SAM realizes flexible and accurate segmentation through three prompt methods: point prompt, box prompt, and text prompt. Moreover, the SAM model shows significant advantages in agricultural scenario applications: its strong generalization ability adapts to the diversity of ground objects, its friendly interaction method, and its efficient deployment characteristics greatly reduce data preparation and model training time.

[0027] Furthermore, in this embodiment, as Figure 4 shown, step S130 may specifically include steps S131 - S134: S131. Divide the target UAV aerial image into multiple sub-block images of a fixed size; S132. Perform model quantization on the image segmentation model to obtain a quantized image segmentation model; S133. For each sub-block image, input the sub-block image into the quantized image segmentation model for ground object segmentation through multi-threading to achieve GPU parallel acceleration, and obtain a sub-ground object segmentation result; S134. Stitch multiple sub-ground object segmentation results to obtain the ground object segmentation result. It should be noted that the target UAV aerial image is a large-size image, for example, 4000×3000 pixels. It is necessary to divide the target UAV aerial image into multiple sub-block images of a fixed size, where the fixed size is 512×512 pixels, and model quantization and GPU parallel acceleration technologies are adopted in the model inference stage to control the inference time of a single sub-block image within 0.5 seconds. It should also be noted that model quantization is achieved by converting high-precision floating-point operations (such as FP32) into low-precision integer operations (such as INT8 / INT4) to realize model lightweight and acceleration; Utilize the GPU multi-core architecture to process data in parallel to improve the computing throughput.

[0028] Furthermore, the image segmentation model is obtained by training and validating the SAM model using multi-scenario sample data, sample prompt information, and agricultural scenario data, including: collecting UAV aerial images in multi-scenarios such as rainy, low-light, cloud-obscured, and sunny as the multi-scenario sample data, and dividing the multi-scenario sample data into a training data set and a validation data set; inputting the training data and the sample prompt information into the SAM model for training, and fine-tuning the parameters of the SAM model in combination with the agricultural scenario data during the training process until a preset number of training times is reached; inputting the validation data set into the trained SAM model for validation to obtain a predicted segmentation result; if the intersection over union (IoU) of the predicted segmentation result and the annotated segmentation result in the validation data set is greater than a preset IoU, then the trained SAM model is used as the image segmentation model. Understandably, if the IoU of the predicted segmentation result and the annotated segmentation result in the validation data set is not greater than the preset IoU, then return to execute the step of dividing the multi-scenario sample data into a training data set and a validation data set until the IoU of the predicted segmentation result and the annotated segmentation result in the validation data set is greater than the preset IoU. It should be noted that in this embodiment, the intersection over union (IoU) is the core index for measuring the overlap degree between the predicted result and the true label in tasks such as image segmentation and object detection. The value range of IoU is [0,1]. The higher the IoU value, the closer the predicted result is to the true label, that is, the closer the predicted segmentation result is to the annotated segmentation result in the validation data set, which also indicates that the image segmentation model is more accurate. It should also be noted that during the training stage of the SAM model, by collecting UAV aerial images in multi-scenarios such as rainy, low-light, cloud-obscured, and sunny as the multi-scenario sample data, the adaptability of the image segmentation model can be enhanced, ensuring that the IoU remains above 0.75 under extreme conditions, not only improving the accuracy of the image segmentation model, but also improving the robustness of the image segmentation model. More importantly, in the method for ground object segmentation of aerial images based on SAM in this embodiment, a fault recovery mechanism is also set up. The fault recovery mechanism is used to automatically switch the image segmentation model to a lightweight MobileNet-based backup model when a downtime occurs, and record detailed logs for subsequent analysis to ensure the continuous and stable operation of ground object segmentation.

[0029] Figure 5 Another process schematic diagram of ground object segmentation of aerial images based on SAM provided by an embodiment of the present invention is shown in Figure 5 As shown, in this embodiment, the method includes steps S110 - S150. That is, in this embodiment, after step S130 of the above embodiment, the method further includes steps S140 and S150.

[0030] S140. Save the ground object segmentation result as a ground object segmentation mask image in PNG format, and generate an Excel report including ground object types, area statistics, and geographical coordinates to obtain a ground object segmentation report; S150. Based on the ground object segmentation mask image and the ground object segmentation report, dock with the GIS system by calling the API interface to achieve spatial analysis and management of agricultural resources.

[0031] In this embodiment, a ground object segmentation mask image and a ground object segmentation report are generated according to the ground object segmentation result, and dock with the GIS system by calling the API interface to achieve spatial analysis and management of agricultural resources. It should be noted that GIS (Geographic Information System) is a computer system used to collect, store, manage, analyze, and display geospatial data. By combining geographical coordinates with attribute data, it supports spatial queries, spatial analysis, and visualization, and is widely used in fields such as urban planning, agriculture, and environmental monitoring.

[0032] For the convenience of understanding, the following two embodiments are used to elaborate on the ground object segmentation of aerial images based on SAM in detail: Embodiment 1: In this embodiment, a high-definition camera device is carried by a drone to collect images of typical agricultural scenes such as farmland, forest land, and water areas; for image preprocessing, geometric correction is used to eliminate distortion, median filtering is used to remove noise, contrast adjustment and color correction are used to improve visual quality; the SAM is selected as the visual neural network for model training, and the hyperparameters are optimized using the labeled agricultural data to ensure the segmentation accuracy; the preprocessed image is input into the trained model to obtain the ground object segmentation result, and a ground object segmentation report and a ground object segmentation mask image are generated according to the ground object segmentation result, realizing the accurate recognition of agricultural ground objects.

[0033] Embodiment 2: This embodiment has two aspects of improvement on the basis of Embodiment 1. The first aspect is enhanced adaptability. The image segmentation model has been comprehensively tested under different lighting conditions (such as sunny days and cloudy days) and weather environments to ensure the adaptability of the image segmentation model in various scenarios. The second aspect is technology integration. The possibility of combining with remote sensing technology is explored, and the segmentation accuracy and application scope are further improved through multi-source data fusion, significantly expanding the application potential of the ground object segmentation technology in agricultural resource monitoring.

[0034] In summary, in this embodiment, before ground object segmentation, the UAV aerial image is preprocessed to obtain the target UAV aerial image. Among them, the preprocessing includes image correction, filtering, and image enhancement, which improves the accuracy of ground object segmentation. Since the image segmentation model is a model obtained by training and validating the SAM model using multi-scene sample data, sample prompt information, and agricultural scene data, the generalization ability and robustness of the model are relatively good, and during the ground object segmentation process, segmentation is also performed according to the segmentation prompt information, further improving the accuracy of ground object segmentation. Since no manual interpretation is required during the entire ground object segmentation process, the segmentation efficiency of ground object segmentation is greatly improved.

[0035] Figure 6 FIG. is a schematic block diagram of a ground object segmentation device 200 for aerial images based on SAM provided by an embodiment of the present invention. As Figure 6 shown, corresponding to the above-mentioned ground object segmentation method for aerial images based on SAM, the present invention also provides a ground object segmentation device 200 for aerial images based on SAM. The ground object segmentation device 200 for aerial images based on SAM includes units for executing the above-mentioned ground object segmentation method for aerial images based on SAM, and this device can be configured in a computer device. Specifically, please refer to Figure 6 FIG., the ground object segmentation device 200 for aerial images based on SAM includes a collection unit 201, a preprocessing unit 202, and an acquisition and segmentation unit 203.

[0036] Among them, the collection unit 201 is used to collect the agricultural scene image to be segmented through a UAV to obtain the original UAV aerial image; the preprocessing unit 202 is used to preprocess the original UAV aerial image to obtain the target UAV aerial image; the acquisition and segmentation unit 203 is used to obtain the segmentation prompt information on the target UAV aerial image, and input the segmentation prompt information and the target UAV aerial image into an image segmentation model for ground object segmentation to obtain a ground object segmentation result. Among them, the segmentation prompt information includes at least any one of point segmentation prompt information, box segmentation prompt information, and text segmentation prompt information, and the image segmentation model is a model obtained by training and validating the SAM model using multi-scene sample data, sample prompt information, and agricultural scene data.

[0037] In some embodiments, such as this embodiment, the collection unit 201 includes a collection subunit, a conversion and noise reduction unit, a transformation calculation unit, a trigger execution unit, and a function unit.

[0038] Among them, the acquisition sub-unit is used to collect the agricultural scene image to be segmented by a drone to obtain an aerial image captured by the drone; the conversion and noise reduction unit is used to perform gray-scale conversion and noise reduction processing on the aerial image captured by the drone to generate a gray-scale and noise-reduced image, and the transformation calculation unit is used to perform Laplace transformation on the gray-scale and noise-reduced image to obtain an image Laplace value, and calculate the variance of the image Laplace value to obtain an image variance; the trigger execution unit is used to trigger a reshooting prompt if the image variance is lower than a preset image variance, and return to execute the step of collecting the agricultural scene image to be segmented by the drone to obtain the aerial image captured by the drone; the acting unit is used to use the aerial image captured by the drone as the original aerial image captured by the drone if the image variance is not lower than the preset image variance.

[0039] In some embodiments, such as this embodiment, the preprocessing unit 202 includes a calibration unit, a filtering unit, and an enhancement unit.

[0040] Among them, the calibration unit is used to perform image calibration on the original aerial image captured by the drone to obtain a calibrated aerial image; the filtering unit is used to filter the calibrated aerial image by using a median filtering algorithm to obtain a filtered aerial image; the enhancement unit is used to perform image enhancement on the filtered aerial image by using an adaptive histogram equalization method to obtain the target aerial image captured by the drone.

[0041] In some embodiments, such as this embodiment, the calibration unit includes an extraction unit and a calibration sub-unit.

[0042] Among them, the extraction unit is used to extract key feature points of the original aerial image captured by the drone by using the SIFT algorithm to obtain aerial key points; the calibration sub-unit is used to calculate a perspective transformation matrix based on the aerial key points, and perform image calibration on the aerial image captured by the drone according to the perspective transformation matrix to obtain the calibrated aerial image.

[0043] In some embodiments, such as this embodiment, the acquisition and segmentation unit 203 includes a division unit, a quantization unit, a ground object segmentation unit, and a splicing unit.

[0044] Among them, the division unit is used to divide the target aerial image captured by the drone into multiple sub-block images of a fixed size; the quantization unit is used to perform model quantization on the image segmentation model to obtain a quantized image segmentation model; the ground object segmentation unit is used to input each sub-block image into the quantized image segmentation model for ground object segmentation through multi-threading to achieve GPU parallel acceleration, and obtain a sub-ground object segmentation result; the splicing unit is used to splice multiple sub-ground object segmentation results to obtain the ground object segmentation result.

[0045] In some embodiments, such as this embodiment, the SAM-based aerial image ground object segmentation device 200 further includes a saving and generating unit and a docking unit.

[0046] Among them, the saving and generating unit is used to save the ground object segmentation result as a ground object segmentation mask image in PNG format, and generate an Excel report including ground object types, area statistics, and geographical coordinates to obtain a ground object segmentation report; the docking unit is used to dock with the GIS system by calling the API interface based on the ground object segmentation mask image and the ground object segmentation report, so as to realize the spatial analysis and management of agricultural resources.

[0047] The above-mentioned SAM-based aerial image ground object segmentation device can be implemented in the form of a computer program, and this computer program can run on a computer device as shown in Figure 7 shown.

[0048] Please refer to Figure 7 , Figure 7 , which is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device 300 is a device with the function of SAM-based aerial image ground object segmentation.

[0049] Referring to Figure 7 , the computer device 300 includes a processor 302, a memory, and a network interface 305 connected through a system bus 301. Among them, the memory may include a non-volatile storage medium 303 and an internal memory 304.

[0050] The non-volatile storage medium 303 can store an operating system 3031 and a computer program 3032. When the computer program 3032 is executed, the processor 302 can be made to execute a SAM-based aerial image ground object segmentation method.

[0051] The processor 302 is used to provide computing and control capabilities to support the operation of the entire computer device 300.

[0052] The internal memory 304 provides an environment for the operation of the computer program 3032 in the non-volatile storage medium 303. When the computer program 3032 is executed by the processor 302, the processor 302 can be made to execute a SAM-based aerial image ground object segmentation method.

[0053] The network interface 305 is used for network communication with other devices. Those skilled in the art can understand, Figure 7The structure shown is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the computer device 300 to which the solution of the present invention is applied. Specifically, the computer device 300 may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0054] Among them, the processor 302 is used to run the computer program 3032 stored in the memory to implement any embodiment of the above-mentioned SAM-based aerial image ground object segmentation method.

[0055] It should be understood that in this embodiment, the processor 302 may be a central processing unit (CPU), and this processor 302 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0056] Those of ordinary skill in the art can understand that all or part of the processes in the methods of implementing the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a storage medium, and the storage medium is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0057] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, the processor is caused to execute any embodiment of the above-mentioned SAM-based aerial image ground object segmentation method.

[0058] The storage medium may be various computer-readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disc that can store program codes.

[0059] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0060] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0061] The steps in the method embodiments of the present invention can be adjusted, combined, and deleted according to actual needs. The units in the device embodiments of the present invention can be combined, divided, and deleted according to actual needs. In addition, the functional units in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0062] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device to execute all or part of the steps of the methods described in each embodiment of the present invention.

[0063] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0064] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

[0065] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for aerial image object segmentation based on SAM, characterized in that: include: The original drone aerial image is obtained by collecting the agricultural scene image to be segmented through the drone; Preprocessing the original UAV aerial image to obtain a target UAV aerial image; The segmentation prompt information on the target UAV aerial image is obtained, and the segmentation prompt information and the target UAV aerial image are input into an image segmentation model for ground object segmentation to obtain a ground object segmentation result, wherein the segmentation prompt information at least includes any one of point segmentation prompt information, frame segmentation prompt information and text segmentation prompt information. The image segmentation model is a model obtained by training and verifying the SAM model using multi-scene sample data, sample prompt information and agricultural scene data.

2. The method according to claim 1, characterized in that The method of acquiring the agricultural scene image to be segmented by using a drone to obtain the original drone aerial image includes: The drone aerial images are obtained by collecting the agricultural scene images to be segmented through the drone; grayscale conversion and noise reduction processing are performed on the drone aerial image to generate a grayscale noise reduction image. Performing Laplace transform on the grayscale denoised image to obtain image Laplace values, and calculating the variance of the image Laplace values ​​to obtain image variance; If the image variance is lower than the preset image variance, a reshoot prompt is triggered, and the process returns to the step of acquiring the agricultural scene image to be segmented by using a drone to obtain the drone aerial image; If the image variance is not less than the preset image variance, the drone aerial image is used as the original drone aerial image.

3. The method according to claim 1, characterized in that The preprocessing of the original UAV aerial image to obtain the target UAV aerial image includes: Performing image correction on the original UAV aerial image to obtain a corrected aerial image; Using a median filter algorithm to filter the corrected aerial image to obtain a filtered aerial image; The filtered aerial image is enhanced by an adaptive histogram equalization method to obtain the target UAV aerial image.

4. The method according to claim 3, characterized in that The performing image correction on the original UAV aerial image to obtain a corrected aerial image includes: The SIFT algorithm is used to extract key feature points of the original UAV aerial image to obtain aerial key points; A perspective transformation matrix is ​​calculated based on the aerial photography key points, and image correction is performed on the drone aerial image according to the perspective transformation matrix to obtain the corrected aerial image.

5. The method according to any one of claims 1 to 4, characterized in that: The image segmentation model is obtained by training and verifying the SAM model using multi-scene sample data, sample prompt information, and agricultural scene data, including: Collect drone aerial images in multiple scenes such as cloudy and rainy days, low light, fog and cloud cover, and sunny days as the multi-scene sample data, and divide the multi-scene sample data into a training data set and a verification data set; Input the training data and the sample prompt information into the SAM model for training, and fine-tune the parameters of the SAM model in combination with the agricultural scene data during the training process until a preset number of training times is reached; Inputting the verification data set into the trained SAM model for verification to obtain a predicted segmentation result; If the intersection-and-union ratio of the predicted segmentation result and the labeled segmentation result in the verification data set is greater than a preset intersection-and-union ratio, the trained SAM model is used as the image segmentation model.

6. The method according to any one of claims 1 to 4, characterized in that: The step of inputting the segmentation prompt information and the target UAV aerial image into an image segmentation model to perform ground object segmentation to obtain a ground object segmentation result includes: Dividing the target UAV aerial image into a plurality of sub-block images of fixed sizes; Performing model quantization on the image segmentation model to obtain a quantized image segmentation model; For each of the sub-block images, the sub-block images are input into the quantized image segmentation model through multi-threading to perform object segmentation so as to achieve GPU parallel acceleration, thereby obtaining a sub-object segmentation result; The plurality of sub-land feature segmentation results are concatenated to obtain the land feature segmentation result.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The object segmentation result is saved as an object segmentation mask image in PNG format, and an Excel report including object type, area statistics and geographic coordinates is generated to obtain an object segmentation report; Based on the object segmentation mask image and the object segmentation report, the spatial analysis and management of agricultural resources are realized by connecting with the GIS system through calling the API interface.

8. A SAM-based aerial image object segmentation device, characterized in that: include: A collection unit, used for collecting agricultural scene images to be segmented by using a drone to obtain original drone aerial images; A preprocessing unit, used for preprocessing the original UAV aerial image to obtain a target UAV aerial image; A segmentation acquisition unit is used to acquire segmentation prompt information on the target UAV aerial image, and input the segmentation prompt information and the target UAV aerial image into an image segmentation model for object segmentation to obtain an object segmentation result, wherein the segmentation prompt information at least includes any one of point segmentation prompt information, frame segmentation prompt information and text segmentation prompt information, and the image segmentation model is a model obtained by training and verifying the SAM model using multi-scene sample data, sample prompt information and agricultural scene data.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 can be implemented.

Images