Intelligent extraction method of greenhouses based on aerial photographs and improved tilt frame detection

By introducing greenhouse angle prediction branches into the aerial film data, building a greenhouse extraction network model, and using YOLOv5 for tilt frame detection, the problem of inaccurate extraction of greenhouses in the existing technology is solved, and the accurate extraction of greenhouses and area is achieved , reduces the impact of background noise.

CN118470526BActive Publication Date: 2024-11-12BEIJING AIERSI TIMES TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410568811.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-09
Publication Date
2024-11-12
Estimated Expiration
2044-05-09

AI Technical Summary

Technical Problem

The existing neural network model cannot take into account the accuracy of greenhouse quantity and area extraction. The segmentation network model has problems of adhesion and incomplete distinction in quantity extraction, while the target detection network model will introduce a large amount of background noise in area extraction.

Method used

Based on the aerial data, the greenhouse angle prediction branch was introduced to build a greenhouse extraction network model, and using YOLOv5 as the basic model, the position, category and inclination angle of the greenhouse were simultaneously predicted through the inclined frame detection, and the optimal inclined frame was output to determine the boundary of the greenhouse to avoid background. The introduction of noise.

Benefits of technology

The accurate extraction of the number and area of ​​greenhouses is achieved, the accuracy of area extraction is improved, the accuracy of quantity and area is taken into account, and the impact of background noise is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118470526B_ABST
    Figure CN118470526B_ABST
Patent Text Reader

Abstract

The present application relates to the field of precision agriculture technology, and provides a greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection. The method utilizes the very high resolution of aerial photographs and the morphological characteristics of greenhouses, annotates greenhouse samples based on aerial photographs, uses YOLOv5 as a basic model, introduces a greenhouse angle prediction branch to simultaneously predict the location, category and tilt angle of the greenhouse, and uses the optimal tilt frame output by the greenhouse extraction network model to accurately locate the boundary of the greenhouse. Based on the object-level extraction idea, the area statistics result of the optimal tilt frame is used as the area of ​​the greenhouse. This method can not only extract the number of greenhouses more accurately, but also, by adding the prediction of the tilt angle, according to the characteristics of the greenhouse as a regular rectangle, the optimal tilt frame is used as the area result of the greenhouse extraction during reasoning, avoiding the introduction of too much background noise, and improving the accuracy of area extraction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of precision agriculture technology, and in particular to a greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection. Background Art

[0002] Agricultural greenhouses are one of the important agricultural facilities. Through identification and extraction of agricultural greenhouses, the area, quantity and spatial distribution of greenhouses can be obtained, which can provide strong support for crop monitoring and yield estimation.

[0003] In recent years, the continuous optimization of spatial observation methods, sensor technology and computing power, as well as the continuous improvement of spatial resolution, spectral resolution and temporal resolution of remote sensing data, have made the extraction of greenhouses through remote sensing methods the mainstream, including greenhouse index extraction methods, machine learning-based methods and deep learning-based methods. Among them, the classic network models based on deep learning include segmentation networks and target detection networks. Segmentation networks specifically include: Unet, SegNet, Deeplab, PSPNet and other models, and target detection networks specifically include: YOLO, SSD, Fast R-CNN and other models. However, accurate greenhouse extraction includes both accurate counting of the number of greenhouses and accurate statistics of the area of ​​greenhouses, while segmentation network models are usually only able to count the area of ​​greenhouses. In the extraction of greenhouse numbers, there are problems such as inaccurate number extraction due to the adhesion of dense greenhouses, inability to distinguish, incomplete distinction or distorted greenhouses. The target detection network model can obtain a relatively accurate number of greenhouses, but a large amount of background noise will be introduced in the acquisition of greenhouse area, resulting in inaccurate statistics of greenhouse area. In other words, the existing neural network model cannot take into account the accuracy of greenhouse number and area extraction.

[0004] Therefore, it is necessary to provide an improved technical solution that can take into account the accuracy of greenhouse quantity and area extraction. Summary of the invention

[0005] The purpose of this application is to provide a greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection to solve or alleviate the problems existing in the above-mentioned prior art.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] The present application provides a greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection, including:

[0008] Obtain aerial photographic data of the target study area;

[0009] Annotate greenhouse sample data based on the aerial photograph data;

[0010] Based on YOLOv5 as the basic model, a greenhouse angle prediction branch is added to construct a greenhouse extraction network model; wherein the greenhouse angle prediction branch is used to receive greenhouse samples carrying tilt angles and predict the tilt angle of the greenhouse in the output results, and the tilt angle of the greenhouse is defined as the angle between the longest side of the greenhouse and the horizontal direction;

[0011] The greenhouse extraction network model is trained using the greenhouse sample data, and the number and area information of the greenhouses are simultaneously extracted based on the trained greenhouse extraction network model, wherein the number of greenhouses is the number of optimal tilted boxes output by the greenhouse extraction network model, and the area of ​​the greenhouse is obtained by counting the areas of the optimal tilted boxes output by the greenhouse extraction network model. The optimal tilted box is obtained by screening multiple candidate boxes corresponding to each greenhouse individual output by the greenhouse extraction network model according to preset rules, and the candidate box is a rectangular bounding box that may contain a greenhouse identified by the greenhouse extraction network model.

[0012] Preferably, labeling greenhouse sample data on the aerial photograph data includes:

[0013] The information of the tilted box is annotated on the aerial photo data using the long side definition method.

[0014] Preferably, the greenhouse angle prediction branch includes a data loading module, a feature extraction module, and an angle processing module;

[0015] The data loading module loads the greenhouse sample data with tilt angles and the aerial photograph data;

[0016] The feature extraction module performs multi-scale downsampling on the aerial photograph data to generate a multi-scale feature map, and predicts multiple candidate frames related to each greenhouse in the aerial photograph data based on the multi-scale feature map;

[0017] The angle processing module uses a circular smooth label method to convert the regression of the tilt angle into an angle classification problem to predict the tilt angle of each candidate frame;

[0018] During the training phase, the tilt angle of each candidate box is used to calculate the tilt angle carried in the greenhouse sample data to generate an angle loss for backward feedback.

[0019] Preferably, the method further comprises:

[0020] In the prediction stage, an improved non-maximum suppression method is used to remove redundancy from the multiple candidate frames to obtain the optimal tilted frame of the greenhouse;

[0021] The improved non-maximum suppression method comprises the following steps:

[0022] The candidate frame with the highest confidence among the multiple candidate frames is taken as the optimal candidate frame, and the rest are pending candidate frames;

[0023] Calculating the distance between the center point of each of the pending candidate boxes and the center point of the optimal candidate box, and if the distance is less than a preset distance threshold, marking the pending candidate box as a redundant box;

[0024] The rotational intersection-and-union ratios of the optimal candidate frame and each redundant frame are calculated, and the redundant frames whose rotational intersection-and-union ratios are greater than a preset intersection-and-union ratio threshold are removed to obtain the optimal tilted frame of the greenhouse.

[0025] Preferably, the preset distance threshold is 100 meters.

[0026] Preferably, the loss function of the greenhouse extraction network model includes classification loss, angle loss, horizontal box loss and confidence loss, the horizontal box loss adopts the complete intersection-over-union (CIoU) function, and the angle loss is a binary cross entropy loss function.

[0027] Preferably, the greenhouse sample data includes: low-density greenhouses, medium-density greenhouses and high-density greenhouses;

[0028] Low-density greenhouses were defined as having less than 0.1 greenhouses per acre in aerial photographs;

[0029] Medium-density greenhouses are defined as 0.1 to 0.3 greenhouses per acre of aerial photographs;

[0030] High-density greenhouses are defined as having more than 0.3 greenhouses per acre in aerial photographs.

[0031] Preferably, the greenhouse sample data includes: gray-white transparent plastic greenhouses, black plastic greenhouses, green plastic greenhouses and blue plastic greenhouses.

[0032] Preferably, the greenhouse sample data includes: wide and long greenhouses, narrow and long greenhouses, wide and short greenhouses, and narrow and short greenhouses;

[0033] A wide and long greenhouse is defined as: a length-to-width ratio of less than 15, and a long side length greater than 120 meters;

[0034] Narrow and long greenhouses are defined as: the length-to-width ratio is greater than 15, and the long side length is greater than 120 meters;

[0035] A wide and short greenhouse is defined as: a length-to-width ratio of less than 15, and a long side length of less than 120 meters;

[0036] A narrow and short greenhouse is defined as one with a length-to-width ratio greater than 15 and a long side length less than 120 meters.

[0037] Preferably, before using the greenhouse sample data to train the greenhouse extraction network model, the method further includes:

[0038] Resampling the aerial photograph data and cropping it to a preset size according to a preset overlap ratio;

[0039] Mosaic data enhancement and geometric data enhancement are performed on the cropped aerial photograph data to expand the training data.

[0040] The technical solution of the embodiment of the present application has the following beneficial effects:

[0041] In the technical solution provided in this embodiment, aerial photographs of the target study area are used as basic data. Utilizing the high-resolution characteristics of aerial photographs (resolution reaches centimeters or even higher), greenhouse samples are first marked out based on the aerial photographs. YOLOv5 is used as the basic model to introduce a greenhouse angle prediction branch to simultaneously predict the location, category and tilt angle of the greenhouse. The optimal tilt frame output by the greenhouse extraction network model is used to accurately locate the boundary of the greenhouse, and the area statistics of the optimal tilt frame are used as the greenhouse area. This method is based on the target detection model YOLOv5. It can not only extract the number of greenhouses more accurately, but also, by adding the prediction of the tilt angle and using the optimal tilted box as the result of greenhouse extraction during inference, that is, the optimal tilted box is output as the entity result of the greenhouse itself. Since most greenhouses are regular rectangles, the oriented bounding box (OBB) can completely fit the boundary of the greenhouse and output it directly as a greenhouse entity object. This can avoid introducing too much background noise, and to a certain extent solves the problem of inaccurate area statistics caused by background noise in the area statistics process, and improves the accuracy of area extraction. Therefore, this method is a new method that can take into account both the number of greenhouses and the accuracy of area extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. Among them:

[0043] Figure 1 A schematic flow chart of a greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection according to some embodiments of the present application.

[0044] Figure 2 Schematic diagram of the inclination angles of a rectangular greenhouse and a trapezoidal greenhouse provided according to some embodiments of the present application.

[0045] Figure 3 A flowchart of an improved non-maximum suppression method provided according to some embodiments of the present application.

[0046] Figure 4A schematic diagram of an improved non-maximum suppression method provided according to some embodiments of the present application.

[0047] Reference numerals:

[0048] 1-optimal candidate box, 2-range of distance threshold, 3-candidate box outside the range of distance threshold, 4-redundant box. DETAILED DESCRIPTION

[0049] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. Each example is provided by way of explanation of the present application and does not limit the present application. In fact, it will be clear to those skilled in the art that modifications and variations may be made in the present application without departing from the scope or spirit of the present application. For example, a feature shown or described as a part of an embodiment may be used in another embodiment to produce yet another embodiment. Therefore, it is desired that the present application includes such modifications and variations within the scope of the appended claims and their equivalents.

[0050] In the following description, the terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0052] The present application embodiment provides a greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection. Figure 1 The process of the method is shown as Figure 1 As shown, the method includes:

[0053] Step S101: Acquire aerial photograph data of the target study area.

[0054] In this embodiment, the target study area refers to the geographical scope or spatial scope selected or defined for specific research in the practical project. The study area of ​​this embodiment can be determined by a variety of methods. For example, the scope of the study area can be determined by understanding existing research results and related information, or the scope and direction of the study can be determined according to the purpose and problem of the study. The geographic information system GIS technology can also be used to perform spatial analysis and geographic data processing on the study area, and the boundaries and scope of the study area can be determined according to the research objectives and data availability. This application does not limit the specific scope and determination method of the study area.

[0055] Preferably, considering the distribution area of ​​greenhouses and the differences in characteristics of greenhouses in different regions, this embodiment selects greenhouse aerial photograph data from 11 regions including Jilin Province, Hebei Province, Tianjin City, Shanxi Province, Jiangsu Province, Zhejiang Province, Ningxia Hui Autonomous Region, Jiangxi Province, Guangdong Province, Yunnan Province and Hainan Province, among which the aerial photograph data of Jiangsu Province, Zhejiang Province and Jiangxi Province account for more than 70% of all aerial photograph data. The above-mentioned research area includes various provinces with relatively high greenhouse areas in my country. In order to meet the normal growth of crops in winter, greenhouses in the north usually have good thermal insulation performance, and are mostly plastic vegetable greenhouses or solar greenhouses. The climate in the south has the characteristics of high temperature and high humidity and frequent monsoons. The greenhouses mainly need to pay attention to waterproofing, and are mostly film multi-span greenhouses and glass intelligent greenhouses. The lighting conditions, greenhouse covering materials and greenhouse distribution in different regions are different, which provides a wealth of greenhouse spectral features, texture features and shape features for greenhouse identification.

[0056] In this embodiment, aerial photograph data, which may also be referred to as aerial images, aerial images, or aerial photographs for short, refers to pictures obtained through aerial photography, which are photos of ground scenes taken from airplanes or other aircraft. Aerial photography is usually performed by installing a camera device on an aircraft, and the camera continuously takes photos of ground scenes as the aircraft flies.

[0057] In recent years, the continuous optimization of space observation methods, sensor technology and computing power, as well as the continuous improvement of spatial resolution, spectral resolution and temporal resolution of remote sensing data, have made the extraction of greenhouses through remote sensing methods the mainstream. Early remote sensing data came from satellite images with low resolution, which was prone to the phenomenon of different spectra for the same object, resulting in misclassification or omission of agricultural greenhouses. At the same time, in medium and low resolution images, the width of a pixel is often greater than the width of a single greenhouse, resulting in unclear greenhouse boundaries and low extraction accuracy of agricultural greenhouses. Considering that high-resolution satellite images can provide more detailed feature information for greenhouse extraction, a single greenhouse can be accurately distinguished. In order to improve accuracy, some technologies attempt to use high-resolution satellite images for greenhouse extraction. However, the timeliness of high-resolution satellite images is relatively limited. Unlike satellite remote sensing data, aerial photograph data has the characteristics of high resolution, high timeliness and low cost. These images can capture subtle features of the surface and pay more attention to the characteristics of each greenhouse, rather than just segmenting and extracting greenhouses as a type of agricultural land.

[0058] Specifically, this embodiment uses a total of 622 aerial photographs. Aerial photographs are obtained using DJI consumer aircraft, such as DJI Phantom 4 series, Mavic II series and other drones with open flight control interfaces. Third-party software can be used to control the ground remote control to send instructions to the aircraft, and control the aircraft to take photos according to a certain degree of aerial survey overlap. The area of ​​one aerial photograph is about 500 acres, and the coordinate system of the aerial photograph data projection is WGS_1984_UTM_Zone_50N.

[0059] Step S102: labeling greenhouse sample data based on aerial photograph data.

[0060] In this embodiment, sample data refers to a data set used to train, verify, and test machine learning models. For the YOLOv5 model, its sample data may include two main parts: image data and annotation information. Among them, image data is an image used to train, verify, and test the target detection model. The data contains examples of various scenes, different lighting conditions, different viewing angles, and various greenhouses. The image data of this embodiment is obtained from aerial photographs and can be stored in a file format (such as JPEG, PNG, etc.), and each file represents an image; the annotation information is the greenhouse location and category information corresponding to the image data. In this embodiment, the boundary of the greenhouse is described by a rectangular bounding box with an inclined angle.

[0061] That is to say, this embodiment changes the label format of the greenhouse samples, and modifies the original label format of the position and category of the YOLOv5 model into a label format of the position, category and tilt angle.

[0062] In a specific implementation, greenhouse samples can be marked in aerial photographs manually, or greenhouse samples can be generated in batches using semi-automatic or automatic methods. For example, image segmentation technology can be used to generate a rough target location, which can then be further refined by manual or automatic algorithms. Of course, greenhouse samples can also be marked in other ways. This embodiment does not limit the specific method of obtaining greenhouse samples.

[0063] Step S103: Taking YOLOv5 as the basic model, add a greenhouse angle prediction branch and construct a greenhouse extraction network model; wherein the greenhouse angle prediction branch is used to receive greenhouse samples carrying inclination angles and predict the inclination angle of the greenhouse in the output results. The inclination angle of the greenhouse is defined as the angle between the longest side of the greenhouse and the horizontal direction.

[0064] It should be noted that YOLOv5 is a single-stage target detection algorithm. Compared with other single-stage target detection algorithms, it can greatly improve the detection speed while achieving similar detection accuracy, and achieve a good balance between accuracy and speed, which can meet the needs of fast and accurate extraction of aerial greenhouses. In addition, YOLOv5 adopts a multi-scale feature fusion strategy, which enables the model to detect targets at different scales, which is suitable for the characteristics of large scale differences of aerial greenhouse targets. This embodiment constructs a greenhouse extraction network model based on the YOLOv5 model, which can make full use of the above advantages of the YOLOv5 model to quickly and accurately obtain the extraction results of the greenhouse.

[0065] The YOLOv5 model itself only provides target position prediction and category prediction, and does not provide tilt angle prediction capability. For this reason, this embodiment adds a greenhouse angle prediction branch based on the YOLOv5 model and integrates it into a specific layer of the network. Through the greenhouse angle prediction branch, the greenhouse extraction network model obtains the ability to predict the greenhouse tilt angle, so as to simultaneously predict the target's position, category and tilt angle.

[0066] Among them, the inclination angle of the greenhouse is defined as the angle between the longest side of the greenhouse and the horizontal direction.

[0067] It should be noted that the above definition of the inclination angle is applicable to regular rectangular greenhouses and irregular greenhouses, such as greenhouses with trapezoidal boundaries. Figure 2 A schematic diagram showing the inclination angles of a regular rectangular greenhouse and an irregular trapezoidal greenhouse is shown in FIG. Figure 2 As shown, in an image, the upper left corner is the origin of the coordinate system, the right and the downward directions are the horizontal and vertical directions respectively, (cx, cy) is the center point of the greenhouse sample, and a straight line is obtained by connecting the origin and the center point of the greenhouse. The angle between the straight line and the horizontal direction is the tilt angle. That is to say, in this embodiment, regardless of whether the shape of the greenhouse is a regular rectangle or a trapezoidal greenhouse, the tilt angle is defined as the angle between the horizontal axis and the longest side of the greenhouse. According to the different rotation directions around the horizontal axis, the tilt angle can be positive or negative. [-90, 0] can be used to indicate that the longest side rotates counterclockwise around the horizontal axis, and [0, 90] can be used to indicate that the longest side rotates clockwise around the horizontal axis.

[0068] Step S104: Use the greenhouse sample data to train the greenhouse extraction network model, and extract the number and area information of the greenhouses based on the trained greenhouse extraction network model, wherein the number of greenhouses is the number of optimal tilted boxes output by the greenhouse extraction network model, and the area of ​​the greenhouse is obtained by counting the area of ​​the optimal tilted box output by the greenhouse extraction network model.

[0069] In this embodiment, the greenhouse extraction network model is trained. For example, the greenhouse sample data can be first divided into a training set, a validation set, and a test set (for example, it can be divided in a ratio of 3:1:1), and then input into the constructed greenhouse extraction network model for training. During the training process, the training set is used to optimize the model parameters, and the validation set is used to verify the trained model, and the performance of the model on unseen data is evaluated. The model is tuned according to the verification results (for example, the value of the hyperparameter can be adjusted) to improve the performance of the model. Finally, the test set is used to test the final trained model to evaluate the generalization ability and prediction effect of the model. The test set is data that has not been used in training and verification. Repeat the above training process, iterate and optimize continuously until satisfactory model performance is obtained, and the model training can be completed, and the trained model can be output and saved.

[0070] It should be noted that the output results of the YOLOv5 model include a series of candidate boxes (bounding boxes). These candidate boxes are rectangular bounding boxes that the model recognizes may contain the target (greenhouse). In each candidate box, the model estimates the category and position of the target. For any greenhouse individual (greenhouse instance), it may be predicted that there are multiple rectangular bounding boxes containing the target greenhouse, that is, there are multiple candidate boxes, which are usually expressed by the center point coordinates (x, y) and width w and height h.

[0071] Due to the added prediction capability of the tilt angle, the result output by the greenhouse extraction network model of this embodiment is different from the traditional YOLOv5 model. In the output result, in addition to the center point coordinates (x, y) and width w and height h of the rectangular box, the tilt angle angle is also added, so that the original horizontal rectangular box is transformed into a tilted box.

[0072] Unlike the training stage, in the model prediction stage, multiple candidate frames corresponding to each individual greenhouse can be screened according to preset rules, and the optimal tilted frame corresponding to each greenhouse can be output. The optimal tilted frame can be output as the greenhouse itself, and then the number of optimal tilted frames can be used as the number of greenhouses, and the area of ​​the optimal tilted frame can be used as the area of ​​the greenhouse, so as to obtain accurate information on the number and area of ​​the greenhouses.

[0073] In summary, the method provided in this embodiment utilizes the shape feature that most of the greenhouse boundaries are regular rectangles, and adds an angle prediction branch based on YOLOv5 according to the morphological characteristics of the greenhouse, so that it supports the inclined target detection task, and finally uses the optimal inclined frame as the extraction result of the greenhouse, and obtains the number and area of ​​the greenhouse at the same time. Since the inclined frame detection has the advantages of reducing the introduction of background noise, avoiding mutual suppression of dense targets, and accurately locating the detection target, combined with the morphological characteristics of the greenhouse as a regular square, the inclined frame can accurately express the square and generate a fit to the real boundary of the greenhouse. Therefore, applying the inclined frame detection to the greenhouse recognition can not only realize the number of greenhouses, but also accurately locate the greenhouse boundary and realize the statistics of the greenhouse area.

[0074] In traditional target detection models, the optimal tilted frame is usually used to describe the attributes of the target, that is, to provide annotation information of the target, such as location and category, which can answer the questions of what targets are in the image and where the targets are. Compared with these target detection models, the optimal tilted frame of this embodiment is not used to provide description information of the greenhouse, but is directly used as the greenhouse instance itself, and the number and area of ​​the greenhouses are directly counted based on the optimal tilted frame.

[0075] In addition, some existing technologies use segmentation network models to achieve pixel-level greenhouse extraction. Affected by factors such as shooting time, weather conditions, geographical location, and light changes, aerial photographs have strong heterogeneity, and greenhouse features are diverse. Pixel-scale segmentation may cause "holes" in greenhouse recognition results, the same greenhouse instance is segmented into multiple instances due to heterogeneity between pixels, and the boundaries between greenhouse instances overlap and stick together. Compared with the pixel-level model, this embodiment is object-oriented extraction, that is, it uses a target detection model, combined with tilted frame detection, and uses tilted frames to express the instance of the greenhouse itself. It focuses on the category and position of the greenhouse instance on the object scale, and realizes object-based greenhouse detection. In addition to obtaining the accurate number and area of ​​greenhouses at the same time, this processing method can also avoid the overlapping and "hole" phenomena of the borders of greenhouse instances, eliminate the problem of close-arranged greenhouse recognition results sticking to each other, and further improve the accuracy of greenhouse extraction.

[0076] Based on the foregoing embodiments, in some optional embodiments, step S102, labeling greenhouse sample data based on aerial photograph data, includes: labeling tilted frame information on the aerial photograph data using the long edge definition method, wherein the tilted frame information includes a tilt angle.

[0077] Among them, the long side definition method is a method used to determine the size and position of the target box (bounding box), especially in the target detection task. In this method, the target box is usually defined by the long side of the target, that is, the long side of the target is used as one side of the target box, and the other side is determined according to a certain length-to-width ratio.

[0078] In this embodiment, the greenhouse sample data can be manually labeled based on the aerial photograph data, and the tilted frame information can be labeled using the long edge definition method. The sample label format is shown in Table 1. Table 1 is as follows:

[0079] Table 1 Sample label format

[0080] class xx cy longside shortside angle

[0081] Among them, class is the sample category code, which starts from 0. In this embodiment, only one category of greenhouse is included. cx and cy are the coordinates of the center point of the greenhouse label, longside and shortside are the longest side and shortest side of the greenhouse label respectively. The above four parameters are divided by the image width and height to achieve normalization processing, and angle is the tilt angle.

[0082] In order to provide rich features for model learning and further improve model performance, in some embodiments, greenhouse sample data includes: low-density greenhouses, medium-density greenhouses and high-density greenhouses; low-density greenhouses are defined as the number of greenhouses per acre of aerial photographs is less than 0.1; medium-density greenhouses are defined as the number of greenhouses per acre of aerial photographs is 0.1-0.3; high-density greenhouses are defined as the number of greenhouses per acre of aerial photographs is greater than 0.3. Among them, when the number of greenhouses per acre of aerial photographs is less than 0.1, the greenhouses are mostly independent greenhouses with clear boundaries and relatively sparse distribution of greenhouses. When the number of greenhouses per acre of aerial photographs is higher than 0.3, the greenhouses are mostly concentrated and connected, and densely distributed in the entire aerial photograph. Therefore, low-density greenhouse aerial photographs are defined as the number of greenhouses per acre of aerial photographs is less than 0.1, high-density greenhouse aerial photographs are defined as the number of greenhouses per acre of aerial photographs is higher than 0.3, and the number of greenhouses per acre of aerial photographs of medium-density greenhouse aerial photographs is between the two. By collecting greenhouse samples with different density distributions, the model can fully learn the spatial distribution characteristics and mutual positional relationships of the greenhouses, thereby improving the accuracy and generalization ability of the model.

[0083] In some other embodiments, the greenhouse sample data includes: gray-white transparent plastic greenhouse, black plastic greenhouse, green plastic greenhouse and blue plastic greenhouse. By providing greenhouse samples with different covering materials, the greenhouse covering materials are diversified and the spectral characteristics are rich, which can further improve the accuracy and performance of the model.

[0084] Among them, greenhouses with different covering materials are classified according to the mean DN value of the pixels contained in the greenhouse samples. The statistical results are shown in Table 2. Table 2 is as follows:

[0085] Table 2 Statistical information of greenhouses with different covering materials

[0086]

[0087]

[0088] By adopting greenhouse samples with a variety of covering materials, rich data support is provided for fully learning the characteristics of greenhouses.

[0089] Taking into account the different shapes of different greenhouses, in some optional embodiments, the greenhouse shape is defined according to the ratio of the longest side to the shortest side of the greenhouse (aspect ratio) and the length of the longest side of the greenhouse. Based on the shapes of different greenhouses, the greenhouse sample data includes: wide and long greenhouses, narrow and long greenhouses, wide and short greenhouses, and narrow and short greenhouses; wide and long greenhouses are defined as: the aspect ratio is less than 15, and the length of the long side is greater than 120 meters; narrow and long greenhouses are defined as: the aspect ratio is greater than 15, and the length of the long side is greater than 120 meters; wide and short greenhouses are defined as: the aspect ratio is less than 15, and the length of the long side is less than 120 meters; narrow and short greenhouses are defined as: the aspect ratio is greater than 15, and the length of the long side is less than 120 meters. In this embodiment, the greenhouses have diverse shapes, which provides rich data support for fully learning the characteristics of greenhouses.

[0090] In order to further improve the generalization performance of the model, in some embodiments, before using the greenhouse sample data to train the greenhouse extraction network model, a data enhancement step is also included, specifically as follows: resampling the aerial photograph data and cropping it to a preset size according to a preset overlap rate; performing mosaic data enhancement and geometric data enhancement on the cropped aerial photograph data to expand the training data.

[0091] In implementation, the aerial photo data can be resampled to a preset resolution, such as 30 cm, based on the specific situation and accuracy requirements of the original aerial photo data. The aerial photo is then cropped to a size of 1024×1024. To reduce the loss of greenhouses in the aerial photo during the cropping process, the overlap rate of the aerial photo data can be set to 0.1. Moasic data enhancement and geometric data enhancement are then performed on the aerial photo data to expand the training data and improve the model robustness and generalization ability.

[0092] Mosaic data enhancement is to obtain a new image by stitching four images after rotation and scaling, and geometric data enhancement can be, for example, rotation, flipping, cropping, etc. It should be noted that since the information of the tilt angle is added to the sample label, the tilt angle should be adjusted accordingly during the data enhancement process.

[0093] After the above processing, the final greenhouse sample data is obtained, and then divided into training set, validation set and test set according to the ratio of 3:1:1. A total of 1626 training data, 542 validation data and 542 test data are generated, which can meet the training requirements of the model.

[0094] In some embodiments, the greenhouse angle prediction branch includes a data loading module, a feature extraction module, and an angle processing module; the data loading module loads greenhouse sample data and aerial photograph data carrying tilt angles; the feature extraction module performs multi-scale downsampling on the aerial photograph data to generate a multi-scale feature map, and based on the multi-scale feature map, predicts multiple candidate frames related to each greenhouse in the aerial photograph data; the angle processing module uses a circular smoothing label method to convert the regression of the tilt angle into an angle classification problem to predict the tilt angle of each candidate frame; in the training stage, the tilt angle of each candidate frame is used to calculate with the tilt angle carried in the greenhouse sample data to generate an angle loss for backward feedback.

[0095] It should be noted that the network architecture of the YOLOv5 model is mainly composed of three parts: Backbone, Neck and Head. The Backbone part mainly receives the input image (usually the cropped aerial photo) of the data loading module, and then realizes the feature extraction of the aerial photo greenhouse; the Neck part performs multi-scale feature fusion on the feature map and passes these features to the prediction layer. The Head part realizes the final aerial photo greenhouse prediction, including the prediction of the greenhouse tilt box category, center point coordinates, longest side, shortest side, confidence and tilt angle.

[0096] Furthermore, the Backbone part mainly includes Focus, C3, SPP and other modules. The Focus module is used to slice the input image (aerial photo data) to improve computing power without losing information; the C3 module is a lightweight convolutional neural network module, which is the main module for learning residual features. Its main function is to increase the depth and receptive field of the network and improve the ability of feature extraction. It is suitable for high-resolution aerial photo greenhouse detection that requires a larger receptive field; the SPP module is spatial pyramid pooling, which converts feature maps of any size into feature vectors of fixed size.

[0097] The above is the network structure of the traditional YOLOv5 model. Considering that the shape of the greenhouse can be considered as a rectangular frame with a certain tilt angle, the prediction of the greenhouse angle is added on the basis of the original YOLOv5 model to obtain the tilt frame information.

[0098] Compared with the YOLOv5 model, the data loading module of this embodiment adds a function of reading the label format carrying the tilt angle, so that the model can load sample data of this type.

[0099] In order to detect the tilt angle, in the feature extraction module, this embodiment adds a layer to the output layer of each candidate frame based on the detection function of the YOLOv5 model to output the tilt angle of each candidate frame.

[0100] In terms of the prediction of the tilt angle, the traditional YOLOv5 model treats the prediction problem of the rotating target as a position regression problem. Such a setting is prone to angle boundary problems (i.e., the phenomenon that the boundary loss value surges due to the existence of angle periodicity). In order to solve this problem, in the angle processing module of this embodiment, the circular smooth label method is used to convert the regression of the tilt angle into an angle classification problem, that is, the tilt angle is converted into 180 categories, and the tilt angles of the prediction box and the target box are smoothed by setting a window function to avoid the surge in boundary loss due to angle periodicity affecting network training. Since the angle of the greenhouse can be quantitatively described, it is feasible to convert the angle into a classification problem.

[0101] The specific conversion formula is as follows:

[0102]

[0103] Where x is the tilt angle of the angle classification, g(x) is the window function, r is the radius of the window function, and θ is the actual rotation angle.

[0104] The window function can be specifically set to a Gaussian function in order to obtain a better smoothing effect.

[0105] By converting the angle regression problem into an angle classification problem, the angle information of the tilted box is decoupled from the border information, and the loss of the tilted box can be decomposed into two parts: horizontal box loss and angle loss. Therefore, in some embodiments, the loss function of the greenhouse extraction network model includes classification loss, angle loss, horizontal box loss and confidence loss. The horizontal box loss uses the complete intersection union CIoU, and the angle loss is a binary cross entropy loss function. The calculation formula is as follows:

[0106] Loss=Loss conf +Loss angle +Loss bbox +Loss cts (2)

[0107] Where Loss is the loss function of the greenhouse extraction network model, that is, the total network loss, and Loss conf is the confidence loss, Loss angleis the angle loss, Loss bbox is the horizontal box loss, Loss cls is the classification loss.

[0108] Among them, the loss functions of confidence loss and classification loss are consistent with the YOLOv5 model.

[0109] Based on comprehensive consideration of the intersection-over-union ratio, center point distance, and aspect ratio of the predicted box and the labeled box, in this embodiment, the horizontal box loss adopts the complete intersection over union (CIoU), and its calculation formula is as follows:

[0110]

[0111]

[0112]

[0113] Loss bbox =1-CIoU (6)

[0114] In the formula, b represents the prediction box, which is a set of bounding boxes that may contain greenhouses generated by the model on the input aerial photo. It is obtained by fine-tuning the anchor box, also known as the candidate box. gt represents the annotation box, that is, the marked greenhouse bounding box, also known as the greenhouse sample bounding box, ρ represents the Euclidean distance between the predicted box and the annotation box, c is the diagonal distance between the predicted box and the minimum circumscribed box of the annotation box, It is a parameter to measure the consistency of the aspect ratio between the predicted box and the labeled box. gt and w are the widths of the annotation box and the prediction box respectively, h gt and h are the heights of the annotation box and the prediction box respectively, Loss bbox is the horizontal box loss, measured by 1-CIoU.

[0115] This embodiment uses a binary cross entropy loss function to calculate the angle classification loss, and its calculation formula is as follows:

[0116]

[0117] Among them, Loss angle represents the angle loss, N represents the number of prediction boxes, θ n represents the tilt angle predicted by the model, θ ngt It represents the tilt angle marked in the greenhouse sample, that is, the true value of the tilt angle.

[0118] This embodiment introduces a circular smooth label method to transform the regression problem of predicting the tilt angle into an angle classification problem, avoiding the angle boundary problem caused by angle periodicity and improving the accuracy of tilt angle prediction. At the same time, by decoupling the prediction of the tilt angle from the prediction of the horizontal box, the model is iteratively trained using angle loss and horizontal box loss, so that the model learns the characteristics of the tilt box.

[0119] In some embodiments, the greenhouse intelligent extraction method based on aerial photographs and improved tilted frame detection also includes: in the prediction stage, using an improved non-maximum suppression method to deduplicate multiple candidate frames to obtain the optimal tilted frame of the greenhouse; the improved non-maximum suppression method includes the following steps: taking the candidate frame with the highest confidence among the multiple candidate frames as the optimal candidate frame, and the rest as pending candidate frames; calculating the distance between the center point of each pending candidate frame and the center point of the optimal candidate frame, if the distance is less than a preset distance threshold, marking the pending candidate frame as a redundant frame; calculating the rotational intersection-and-union ratio of the optimal candidate frame and each redundant frame, and removing the redundant frames whose rotational intersection-and-union ratio is greater than the preset intersection-and-union ratio threshold to obtain the optimal tilted frame of the greenhouse.

[0120] The traditional non-maximum suppression method (NMS) is a common method for eliminating redundant candidate boxes and determining the optimal candidate box in all candidate boxes. The principle is: all candidate boxes are screened according to a certain confidence threshold, and they are arranged in descending order according to confidence, and the candidate box with the highest confidence is selected as the optimal candidate box. The candidate box and its confidence are removed from the candidate box set, and the intersection and union ratios of other candidate boxes (pending candidate boxes) and the optimal candidate box are calculated in turn. If the calculated intersection and union ratio is greater than the set intersection and union ratio threshold, it is considered that the pending candidate box and the optimal candidate box detect the same object, then the pending candidate box and its confidence are removed from the candidate box set until all pending candidate boxes are traversed, and the candidate box with the highest confidence is selected again. Repeat the above steps until the candidate box set is empty. The formula is as follows:

[0121]

[0122] Among them, s i Represents the confidence score of the current candidate box, iou represents intersection-over-union ratio, M i represents the current optimal candidate box, b i Indicates the other current candidate boxes, that is, the current pending candidate boxes. Threshold indicates the intersection-and-union ratio threshold. If the intersection-and-union ratio of the current candidate box and the optimal candidate box is greater than or equal to the threshold, the score of the current candidate box is set to 0, that is, the candidate box is removed.

[0123] The traditional NMS method requires a large number of intersection-and-union ratio calculations, especially in high-density aerial photographs, where the intersection-and-union ratio calculations and time costs brought by densely distributed greenhouses increase exponentially. In order to improve the greenhouse detection speed, this embodiment improves the NMS method, that is, when performing rotational NMS, before traversing and calculating the rotational intersection-and-union ratio of the optimal candidate frame and other candidate frames (pending candidate frames), the center point distance between the optimal candidate frame and other candidate frames is first calculated. If the distance between the center point of other candidate frames and the optimal candidate frame is greater than or equal to a preset distance threshold T, the pending candidate frame is not a redundant frame of the optimal candidate frame, that is, the pending candidate frame will definitely not be suppressed by the optimal candidate frame at this time, so there is no need to calculate the rotational intersection-and-union ratio of the pending candidate frame and the optimal candidate frame. Otherwise, if the distance between the center point of the pending candidate frame and the center point of the optimal candidate frame is less than the preset distance threshold T, the pending candidate frame is marked as a redundant frame, and the rotational intersection-and-union ratio of the pending candidate frame and the optimal candidate frame needs to be calculated, and then non-maximum suppression processing is performed to finally obtain the optimal tilted frame corresponding to each greenhouse instance. The specific process is as follows: Figure 3 By judging the distance between candidate boxes in advance, a large number of calculations of rotation intersection-over-union ratios can be omitted, which greatly improves the prediction speed while ensuring the prediction accuracy.

[0124] The improved non-maximum suppression method can be expressed by the following formula:

[0125]

[0126] Wherein, distance represents the distance between the center point of the candidate box to be determined and the center point of the optimal candidate box, T represents the distance threshold, and the meanings of other parameters in formula (9) are the same as those in formula (8), which will not be repeated here.

[0127] Figure 4 A schematic diagram showing the comparison between the improved non-maximum suppression method and the traditional method is shown in FIG. Figure 4 As shown, the optimal candidate box 1 is represented by black, the green box represents the range where the distance from the center point of the optimal candidate box is less than the distance threshold T, the candidate box 3 outside the distance threshold T is represented by blue, and the candidate box within the distance threshold T is the redundant box 4, which is represented by red. The traditional NMS method needs to calculate the intersection and union ratio of the optimal candidate box 1 and all the pending candidate boxes, while in the improved NMS method, only the red part, that is, the intersection and union ratio of the redundant box 4 and the optimal candidate box 1, needs to be calculated. The blue candidate box is the candidate box 3 outside the distance threshold range. Because it is far away from the center point of the optimal candidate box 1, it will not be suppressed by the optimal candidate box and will be screened. This setting greatly reduces the number of redundant boxes 4 that need to calculate the intersection and union ratio, greatly reduces the time required for the NMS step, and further improves the efficiency of the model.

[0128] Considering that the determination of the distance threshold T needs to achieve a balance between prediction time and prediction accuracy, a too large distance threshold T is difficult to save time, while a too small distance threshold T will lead to a decrease in prediction accuracy. In a preferred example, the preset distance threshold is 100 meters. The experimental results show that when the distance threshold T is selected as 100 meters, the prediction speed is maximized while ensuring the prediction accuracy, and the prediction time is about 0.1 seconds per scene on average. When T is less than 100 meters, the detection speed is fast, but the detection accuracy of the number of greenhouses is poor. Too small a screening distance may cause the suppressed detection frame to not be suppressed, and multiple prediction frames are eventually generated for the same greenhouse target, and the detection accuracy decreases. When T is greater than 100 meters, as T increases, the greenhouse detection accuracy basically does not change, but the detection time is greatly increased. Therefore, choosing T as 100 meters is a good threshold value for balancing the accuracy and speed of greenhouse detection in aerial photos.

[0129] On the premise of achieving similar prediction accuracy, the greenhouse extraction and prediction speed before and after the improvement of NMS are compared, and the results are as follows:

[0130] Table 3 Comparison of model efficiency before and after NMS improvement

[0131]

[0132] As can be seen from the table, after the NMS is improved, the model FPS can be doubled. In other words, when performing NMS, the distance between the optimal candidate box and other candidate boxes is first judged, and the candidate boxes that are far away from the optimal candidate box are screened out, which can greatly reduce the amount of calculation of the intersection and union ratio, and effectively improve the prediction efficiency of the model while achieving the same prediction accuracy. When there are more aerial photos of high-density greenhouses, the improvement effect will be more obvious.

[0133] In an example, the method provided in this embodiment can be performed according to the following steps:

[0134] Step 1: Extract the feature map of the aerial photo image. The feature map is downsampled by 32 times, 16 times, and 8 times to generate three new feature maps of different scales.

[0135] Step 2: Divide the acquired multi-scale image features into grids. The field of view size corresponding to each pixel is the cell size in the grid. The cell containing the actual center point of the greenhouse is selected as the cell responsible for predicting the greenhouse.

[0136] Step 3: At each scale, generate three anchor boxes for each grid based on the set aspect ratio, obtain the offset of the prediction box relative to the anchor box, fine-tune the prediction box based on the anchor box, and output the category, center point coordinates, longest side, shortest side, tilt angle and confidence of the prediction box. The confidence is determined by the possibility that the prediction box contains the target and the intersection and union ratio of the prediction box and the true box.

[0137] Step 4: Use the improved rotational NMS method to filter the generated series of prediction boxes to obtain the final greenhouse prediction results.

[0138] In summary, this embodiment fully considers the morphological characteristics of greenhouses, adds an angle prediction branch on the basis of YOLOv5 to support the detection of inclined targets, uses a circular smooth label method to solve the angle boundary problem, and outputs the optimal inclined frame as the extraction result of the greenhouse, which can take into account the accuracy of the number of greenhouses and the accuracy of the area. By improving the non-maximum suppression method, the candidate frame is screened according to the distance between the optimal candidate frame and other candidate frames, reducing the amount of calculation of the rotation intersection and union ratio, and improving the efficiency of greenhouse detection.

[0139] In order to verify the accuracy of greenhouse extraction by the method provided in this embodiment, after extracting the number and area information of greenhouses based on the trained greenhouse extraction network model, it also includes: selecting the semantic segmentation method Unet, the semantic segmentation method (Unet+PP) and Oriented_R-CNN for post-processing to compare with the method provided in this embodiment, wherein the semantic segmentation method uses the region-boundary multi-classification model to predict greenhouse instances, and the post-processing includes two steps of instance expansion and minimum bounding rectangle recognition, and the number of instance expansion pixels is set to 1. Four indicators, AP (Average Precision), MIoU (Mean Intersection over Union), BIoU (Boundary Intersection over Union) and Frame Per Second (FPS), are selected to evaluate the extraction results of the model. Among them, AP is used to evaluate the accuracy of predicting the number of greenhouse instances, MIoU is used to evaluate the accuracy of predicting the area of ​​greenhouse instances, BIoU is used to evaluate the accuracy of predicting the boundaries of greenhouse instances, and FPS is used to evaluate the model prediction speed. For the greenhouse prediction frames sorted in descending order of confidence, the intersection and union ratio of a single greenhouse sample and each greenhouse instance in the greenhouse sample is calculated in turn, and the greenhouse sample with the largest intersection and union ratio is considered to match it, and the intersection and union ratio threshold is set to 0.5, that is, when the maximum intersection and union ratio is greater than 0.5, it means that the greenhouse prediction frame is a correctly detected prediction frame, and the prediction frame with an intersection and union ratio lower than 0.5 and other prediction frames with an intersection and union ratio greater than 0.5 with the greenhouse sample instance will be considered as incorrect prediction frames. The calculation formula of each indicator can refer to the prior art, and this embodiment will not be repeated.

[0140] According to experimental analysis, the method provided in this embodiment has a prediction accuracy AP of 87.6%, MIoU of 83.6%, BIoU of 67.78%, and a prediction efficiency FPS of 10. Compared with other semantic segmentation models, inclined box target detection models, and horizontal box target detection models, the accuracy and speed are improved (MIoU can be increased by 3.49% to 27.6%, BIoU can be increased by 3.58% to 25.36%, and AP can be increased by 0.16% to 9.66%). It has obvious advantages and can realize the rapid and accurate extraction of the two indicators of the number and area of ​​aerial photos of greenhouses.

[0141] Experimental data show that compared with other models, the area accuracy and boundary accuracy of the prediction results of the method provided in this embodiment are higher. The output greenhouses of each model are visualized and analyzed to find that the method provided in this embodiment predicts the number of greenhouse instances and the area of ​​greenhouses closer to the actual situation, and has good results in greenhouse accurate counting and boundary positioning.

[0142] Among them, the extraction accuracy of Unet and Unet+PP is significantly different from that of the method provided in this embodiment. As a tilted target detection method, Oriented_R-CNN has an accuracy similar to that of the present application, but its model prediction time is three times that of the present embodiment, and it is unable to take into account both prediction accuracy and prediction efficiency.

[0143] Practice has shown that the method provided in this embodiment is suitable for accurately obtaining the distribution of greenhouses in a large-scale spatial range. It uses aerial photograph data with stronger timeliness, higher spatial resolution, and the ability to fully express the characteristics of greenhouses to achieve accurate and rapid extraction of greenhouses, providing an effective method for extracting greenhouses in a large-scale spatial range.

[0144] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A greenhouse intelligent extraction method based on aerial photographs and improved tilt frame detection, characterized in that: include: Obtain aerial photographic data of the target study area; Annotate greenhouse sample data based on the aerial photograph data; Based on YOLOv5 as the basic model, a greenhouse angle prediction branch is added to construct a greenhouse extraction network model; wherein the greenhouse angle prediction branch is used to receive greenhouse samples carrying tilt angles and predict the tilt angle of the greenhouse in the output results, and the tilt angle of the greenhouse is defined as the angle between the longest side of the greenhouse and the horizontal direction; The greenhouse extraction network model is trained using the greenhouse sample data, and the number and area information of the greenhouses are simultaneously extracted based on the trained greenhouse extraction network model, wherein the number of greenhouses is the number of optimal tilted boxes output by the greenhouse extraction network model, and the area of ​​the greenhouse is obtained by counting the areas of the optimal tilted boxes output by the greenhouse extraction network model, and the optimal tilted boxes are obtained by screening multiple candidate boxes corresponding to each greenhouse individual output by the greenhouse extraction network model according to preset rules, and the candidate boxes are rectangular bounding boxes that may contain greenhouses recognized by the greenhouse extraction network model; Also includes: In the prediction stage, an improved non-maximum suppression method is used to remove redundancy from multiple candidate frames to obtain the optimal tilted frame of the greenhouse; The improved non-maximum suppression method comprises the following steps: The candidate frame with the highest confidence among the multiple candidate frames is taken as the optimal candidate frame, and the rest are pending candidate frames; Calculating the distance between the center point of each of the pending candidate boxes and the center point of the optimal candidate box, and if the distance is less than a preset distance threshold, marking the pending candidate box as a redundant box; The rotational intersection-and-union ratios of the optimal candidate frame and each redundant frame are calculated, and the redundant frames whose rotational intersection-and-union ratios are greater than a preset intersection-and-union ratio threshold are removed to obtain the optimal tilted frame of the greenhouse.

2. The method according to claim 1, characterized in that Labeling greenhouse sample data based on the aerial photograph data includes: The information of the tilted box is annotated on the aerial photo data using the long side definition method.

3. The method according to claim 1, characterized in that The greenhouse angle prediction branch includes a data loading module, a feature extraction module, and an angle processing module; The data loading module loads the greenhouse sample data with tilt angles and the aerial photograph data; The feature extraction module performs multi-scale downsampling on the aerial photograph data to generate a multi-scale feature map, and predicts multiple candidate frames related to each greenhouse in the aerial photograph data based on the multi-scale feature map; The angle processing module uses a circular smooth label method to convert the regression of the tilt angle into an angle classification problem to predict the tilt angle of each candidate frame; During the training phase, the tilt angle of each candidate box is used to calculate the tilt angle carried in the greenhouse sample data to generate an angle loss for backward feedback.

4. The method according to claim 1, characterized in that: The preset distance threshold is 100 meters.

5. The method according to claim 1, characterized in that: The loss function of the greenhouse extraction network model includes classification loss, angle loss, horizontal box loss and confidence loss. The horizontal box loss adopts the complete intersection-over-Union (CIoU) function, and the angle loss is a binary cross entropy loss function.

6. The method according to claim 1 or 2, characterized in that: The greenhouse sample data includes: low-density greenhouses, medium-density greenhouses and high-density greenhouses; Low-density greenhouses were defined as having less than 0.1 greenhouses per acre in aerial photographs; Medium-density greenhouses are defined as 0.1 to 0.3 greenhouses per acre of aerial photographs; High-density greenhouses are defined as having more than 0.3 greenhouses per acre in aerial photographs.

7. The method according to claim 6, characterized in that The greenhouse sample data includes: gray-white transparent plastic greenhouses, black plastic greenhouses, green plastic greenhouses and blue plastic greenhouses.

8. The method according to claim 7, characterized in that The greenhouse sample data includes: wide and long greenhouses, narrow and long greenhouses, wide and short greenhouses, and narrow and short greenhouses; A wide and long greenhouse is defined as: a length-to-width ratio of less than 15, and a long side length greater than 120 meters; Narrow and long greenhouses are defined as: the length-to-width ratio is greater than 15, and the long side length is greater than 120 meters; A wide and short greenhouse is defined as: a length-to-width ratio of less than 15, and a long side length of less than 120 meters; A narrow and short greenhouse is defined as one with a length-to-width ratio greater than 15 and a long side length less than 120 meters.

9. The method according to claim 1, characterized in that: Before using the greenhouse sample data to train the greenhouse extraction network model, the method further includes: Resampling the aerial photograph data and cropping it to a preset size according to a preset overlap ratio; Mosaic data enhancement and geometric data enhancement are performed on the cropped aerial photograph data to expand the training data.