Method and device for extracting sheep population in pastoral areas from sub-meter satellite images

Abstract

The present invention discloses a method and device for extracting the number of sheep in pastoral areas from sub-meter satellite images, and belongs to the technical field of high-resolution remote sensing images. The method includes synchronously collecting optical satellite images and unmanned aerial vehicle orthophotos and preprocessing them; constructing a target detection model based on a deep neural network, detecting sheep in the preprocessed optical satellite images, and clipping the images according to the detection frame to obtain satellite image blocks of each sheep flock; classifying the obtained satellite image blocks of each sheep flock to distinguish the sheep flock target and the background; constructing a sheep number estimation model based on the brightness difference between the sheep flock target and the background, and using the preprocessed unmanned aerial vehicle orthophotos to train the sheep number estimation model to obtain the optimal sheep number estimation model. The present invention can effectively utilize the advantages of satellite images, such as wide range, low cost, and high degree of standardization, to improve the efficiency of livestock number accounting in a large range of pastoral areas and reduce application costs.

Description

Technical Field

[0001] The invention belongs to the technical field of high-resolution remote sensing images, and in particular relates to a method and a device for extracting the number of sheep in a pastoral area from sub-meter satellite images. Background Art

[0002] Timely and accurate grasp of the number of livestock in pastoral areas is crucial to assessing the development of local animal husbandry. Traditional ground survey methods include household surveys and filling in reports at each level. Household surveys are time-consuming and labor-intensive and difficult to implement comprehensively; filling in reports at each level makes it difficult to avoid false reporting, concealment, and omissions, and are not easy to verify, making it difficult to ensure data quality.

[0003] Drone imaging is flexible and can quickly acquire images within a certain range. It also has centimeter-level spatial resolution, and the outlines of animals are clearly visible, which is conducive to accurately counting the number of livestock within the range. Therefore, it is often used to estimate the number of livestock in grasslands. However, pastoral areas cover a wide range, and the cost of collecting drone images is high. The range of a single imaging is limited, making it difficult to image on a large scale in a short period of time. The only way is to use the sample strip extrapolation method. At the same time, the amount of data from large-area drone images is huge, and the processing process is time-consuming and labor-intensive.

[0004] With the development of satellite remote sensing technology, the spatial resolution of optical satellites has been continuously improved. At the same time, satellite images have the characteristics of wide coverage of single imaging and easy acquisition, making the use of satellite images to monitor livestock in pastoral areas a new direction. At present, many researchers have built models based on sub-meter optical satellite images to monitor the population of large birds and mammals, but the real data in such studies generally come from ground surveys or optical satellite images, and ground surveys are difficult to implement and have low timeliness; the quality of data obtained by manual visual interpretation of optical satellite images is difficult to guarantee, resulting in the lack of synchronous, accurate, and sufficient ground real data support for the above research. In addition, for smaller research objects (such as sheep), the single area is smaller than one pixel, and they like to gather together. They are generally in the form of mixed pixels in satellite images, making it difficult to capture their outlines, which brings great difficulty to the accurate estimation of their population size. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method and device for extracting the number of sheep in pastoral areas from sub-meter satellite images. In view of the characteristics of sheep being small, preferring to gather together, and having a high contrast with the background, a sheep number estimation model based on sub-meter optical satellite images is constructed, and the model is trained using synchronous drone images as the real data source, thereby achieving efficient and accurate grasp of the number of sheep in pastoral areas, providing a reference for grassland ecological environment protection and optimal allocation of pastoral resources.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method for extracting the number of sheep in a pastoral area from a sub-meter satellite image, the method comprising:

[0008] Step 1: Synchronously collect optical satellite images and drone orthophotos and perform preprocessing;

[0009] Step 2: Build a target detection model based on a deep neural network to detect the sheep in the preprocessed optical satellite image, and crop the image according to the detection frame to obtain the satellite image block of each sheep.

[0010] Step 3: classify the satellite image blocks of each flock of sheep to distinguish the flock target and the background;

[0011] Step 4: Based on the brightness difference between the flock target and the background, a sheep quantity estimation model is constructed, and the sheep quantity estimation model is trained using the preprocessed UAV orthophoto to obtain the optimal sheep quantity estimation model.

[0012] On the other hand, the present invention provides a device for extracting the number of sheep in a pastoral area from a sub-meter satellite image, comprising:

[0013] Image acquisition module, used to synchronously collect optical satellite images and drone orthophotos and perform preprocessing;

[0014] The detection module is used to build a target detection model based on a deep neural network, detect the sheep in the preprocessed optical satellite image, and crop the image according to the detection frame to obtain the satellite image block of each sheep;

[0015] The classification module is used to classify the satellite image blocks of each flock of sheep and distinguish the flock target from the background;

[0016] The estimation module is used to construct a sheep quantity estimation model based on the brightness difference between the sheep flock target and the background, and train the sheep quantity estimation model using the preprocessed UAV orthophoto to obtain the optimal sheep quantity estimation model.

[0017] In a third aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for extracting the number of sheep in pastoral areas from sub-meter satellite images.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned method for extracting the number of sheep in pastoral areas from sub-meter satellite images.

[0019] The beneficial effects of the present invention are:

[0020] Compared with the existing methods of animal detection and quantity accounting using satellite images, the present invention constructs a sheep quantity estimation model based on sub-meter optical satellite images, targeting the characteristics of sheep being small, grazing, and having a high contrast with the background, and uses synchronous drone images as the real data source to train the model, thereby achieving efficient and accurate grasp of the number of sheep in pastoral areas. The advantages of satellite images, such as wide range, low cost, and high degree of standardization, are effectively utilized, the efficiency of livestock quantity accounting in a large range of pastoral areas is improved, and the application cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The present invention is a flow chart of the method for extracting the number of sheep in pastoral areas from sub-meter satellite images. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] The present invention aims to solve the problem that the livestock quantity in pastoral areas cannot be estimated quickly and accurately due to the use of a single remote sensing image data source. To this end, the present invention provides a method for extracting the number of sheep in pastoral areas using sub-meter satellite images, which combines sub-meter high-resolution satellite images and synchronous drone images to monitor the number of sheep in pastoral areas. Specifically, the method comprises the following steps:

[0024] Step 1: Synchronously collect optical satellite images and drone orthophotos and perform preprocessing;

[0025] Step 2: Build a target detection model based on a deep neural network to detect the sheep in the preprocessed optical satellite image, and crop the image according to the detection frame to obtain the satellite image block of each sheep.

[0026] Step 3: classify the satellite image blocks of each flock of sheep to distinguish the flock target and the background;

[0027] Step 4: Based on the brightness difference between the flock target and the background, a sheep quantity estimation model is constructed, and the sheep quantity estimation model is trained using the preprocessed UAV orthophoto to obtain the optimal sheep quantity estimation model.

[0028] Example

[0029] The various steps of the present invention are further described below based on specific embodiments.

[0030] Step A: Synchronously collect optical satellite images and drone orthophotos. Given the small size of sheep, optical satellite images with sub-meter resolution should be selected. The acquisition time should be during the growing season (June-August) when the weather is clear and cloudless. The imaging range should cover the pastoral area where the number of sheep needs to be counted, and drone orthophotos of multiple sheep flocks in the same pastoral area should be collected within one hour before and after the satellite passes. The drone orthophotos are used to obtain the true value of the number of sheep flocks to train and verify the model. As many sheep flocks as possible (more than 20 flocks) should be collected, and the resolution should be sufficient to identify the outline of the sheep (better than 5 cm). In addition to the entire sheep flock, each drone orthophoto should also include houses, sheep pens, canals or roads in the background to facilitate subsequent registration.

[0031] Step B: Preprocessing of optical satellite images. Use the RPC OrthorectificationWorkflow tool in ENVI software to orthorectify the panchromatic and multispectral images respectively, and then use the Seamless Mosaic tool to stitch the panchromatic and multispectral images respectively, and check the histogram matching. Use the Gram-Schmidt PanSharpening tool to fuse the panchromatic and multispectral images based on the stitched images, and finally perform low-pass filtering on the fused images to filter out noise.

[0032] Step C, preprocess the drone orthophotos and match them to the target. Use software such as Pix4Dmapper to stitch the drone orthophotos, and use the ArcGIS platform to generate a spatial layer so that its coordinate system is consistent with the optical satellite image. Use the optical satellite image as the reference image, and use characteristic features to select more than 3 pairs of control points to align the drone orthophotos. Use visual interpretation of the aligned images to obtain the number of sheep in each flock. Overlay the preprocessed optical satellite image and the aligned drone orthophoto in ArcGIS. For each flock of sheep in the aligned drone orthophoto, search for the same flock from near to far on the preprocessed optical satellite image within a range of about 1 km. If multiple flocks appear within this range, the distribution of other nearby flocks and the size of the flocks will be used to determine. Create a shpfile surface element for each corresponding flock based on the optical satellite image layer to record the number of sheep in the corresponding flock obtained based on the visual interpretation of the drone orthophoto. .

[0033] Step D: Clipping of sheep flock range based on optical satellite images.

[0034] A target detection model based on a deep neural network is constructed to detect sheep in optical satellite images and crop the images according to the detection frame. Taking Faster RCNN as an example, a training dataset is constructed based on the preprocessed sub-meter high-resolution optical satellite image set: each optical satellite image is cut into several 600×600 pixel image blocks, the sheep position is marked with a marker frame in each optical satellite image, and the corresponding category, coordinates, width and height information are stored in the label file. At the same time, some easily confused objects such as cattle, haystacks, stone piers, etc. are marked as negative samples with a marker frame. All optical satellite image samples are flipped (upside down, horizontally flipped) and rotated (clockwise 90°, 180°, 270°) to expand the sample size. A dataset including optical satellite images and label files is constructed, the dataset is organized into VOC format, and the dataset is divided into training set, validation set and test set in a ratio of 8:1:1. To ensure the training effect of the model, at least 400 sheep samples are required.

[0035] Training model: ① Initialize hyperparameters such as learning rate, batch size, and number of iterations; ② Input the training set and validation set into the Faster RCNN model for training; ③ Input the test set into the trained model. If it meets the set accuracy requirements, this model is used as the flock detection model for satellite images. Otherwise, adjust the hyperparameters and repeat the model training until the test set accuracy meets the requirements.

[0036] Sheep herd range cropping: Substitute the sub-meter high-resolution optical satellite images of the pastoral area into the trained sheep herd detection model to obtain the boundaries of the pastoral herds. Crop the optical satellite images according to the boundaries to obtain satellite image blocks of each sheep herd.

[0037] Preferably, if a sufficient number of samples cannot be obtained, manual visual interpretation can be used to search for flocks of sheep one by one in the optical satellite image, and each flock of sheep can be cropped into a separate image, and the cropping range is preferably just enough to cover all individuals in the flock.

[0038] Preferably, the deep learning target detection algorithm is not limited to a specific model, and deep learning models with target detection functions such as Faster RCNN, Mask RCNN, RetinaNet, SSD, YOLO series, etc. can be used for flock detection.

[0039] Step E: Target classification based on satellite image blocks: For each obtained satellite image block of sheep flocks, a classification system is constructed according to the ground objects in the satellite image block, and samples are selected in ENVI and classified using the maximum likelihood classification tool to distinguish the sheep flock targets from the background.

[0040] Preferably, the classification is not limited to a specific algorithm, and can be replaced by commonly used supervised classification algorithms such as support vector machine, random forest, decision tree, minimum distance, etc.

[0041] Step F: construct a sheep population estimation model.

[0042] The brightness of each flock target pixel is calculated, where: for the panchromatic single-band image, the DN value of the flock target pixel is used as the brightness; for the RGB three-band image, the flock target pixel brightness is calculated as L = 0.299R + 0.587G + 0.114B; for the other multi-band images, the mean of each band is used as the flock target pixel brightness, that is, the flock target pixel brightness refers to the brightness corresponding to each pixel in the flock area in the image.

[0043] Construct a sheep population estimation model. For each satellite image block, select the maximum brightness value of the pixel in the flock as , the background brightness mean is taken as , the number of sheep N in the flock can be calculated using the following formula:

[0044] (1)

[0045] in, is the area of ​​a single pixel, calculated based on the image resolution; is the brightness of the target pixel of the flock; The theoretical average area occupied by each sheep in the satellite image.

[0046] Step G: Model training and verification.

[0047] About 70% of all the UAV image flock data are randomly selected to train the sheep population estimation model, and the rest are used to verify the model accuracy. For each satellite image block j corresponding to the UAV image flock data used for training the model, the sheep population recorded in step C is used to estimate the sheep population. , according to formula (2), calculate the average vertical projection area of ​​each sheep in the flock :

[0048] = (2)

[0049] according to{ }(j∈[1, ], j∈Z) can derive a more generalizable parameter: theoretically, the average area occupied by each sheep on the satellite image block :

[0050] (3)

[0051] in, is the number of sheep flocks used to train the sheep population estimation model, is the number of sheep in the jth flock in the UAV orthophoto.

[0052] For the trained sheep population estimation model, the remaining 30% of data is used to verify the accuracy. If the estimation error of the sheep population estimation model is within 5%, it is considered to meet the accuracy requirements. Otherwise, the classification algorithm is adjusted according to the error situation, and the sheep population estimation model is retrained until the accuracy meets the requirements.

[0053] Step H: Automatically extract the number of sheep.

[0054] For the satellite image to be analyzed, use step E to process to obtain the satellite image block containing the sheep, use the method provided in step F to calculate the brightness of the target pixel, and estimate the number of sheep in the image based on the trained sheep number estimation model (Formula 1).

[0055] The present invention uses the high contrast between the target and the background in the image to construct a sheep population estimation model, uses the synchronized drone image as the source of truth to train the model, and calculates the abundance of the target in a single pixel, thereby achieving an accurate estimation of the number of sheep in the pastoral area. The method proposed in the present invention is applicable to other satellite images of the same resolution, and there is no need to re-acquire drone images when estimating the number of sheep flocks using other satellite images of the same resolution.

[0056] On the other hand, the present invention provides a device for extracting the number of sheep in a pastoral area from a sub-meter satellite image, comprising:

[0057] Image acquisition module, used to synchronously collect optical satellite images and drone orthophotos and perform preprocessing;

[0058] The detection module is used to build a target detection model based on a deep neural network, detect the sheep in the preprocessed optical satellite image, and crop the image according to the detection frame to obtain the satellite image block of each sheep;

[0059] The classification module is used to classify the satellite image blocks of each flock of sheep and distinguish the flock target from the background;

[0060] The estimation module is used to construct a sheep quantity estimation model based on the brightness difference between the sheep flock target and the background, and train the sheep quantity estimation model using the preprocessed UAV orthophoto to obtain the optimal sheep quantity estimation model.

[0061] In a third aspect, the present invention provides an electronic device comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method for extracting the number of sheep in pastoral areas from sub-meter satellite images.

[0062] In a fourth aspect, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the aforementioned method for extracting the number of sheep in pastoral areas from sub-meter satellite images.

[0063] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for extracting the number of sheep in pastoral areas from sub-meter satellite images, characterized in that: The method comprises: Step 1: Synchronously collect optical satellite images and drone orthophotos and pre-process them, and use visual interpretation to obtain the number of sheep in each flock from the registered drone orthophotos; Step 2: Build a target detection model based on a deep neural network to detect the sheep in the preprocessed optical satellite image, and crop the image according to the detection frame to obtain the satellite image block of each sheep. Step 3: classify the satellite image blocks of each flock of sheep to distinguish the flock target and the background; Step 4: Based on the brightness difference between the flock target and the background, a sheep quantity estimation model is constructed, and the sheep quantity estimation model is trained using the pre-processed UAV orthophoto to obtain the optimal sheep quantity estimation model, including: In the sheep population estimation model, for each satellite image block, the maximum brightness value of the target pixel of the flock is selected as , the background brightness mean is taken as , calculate the number of sheep in the flock N: (1) In the formula, is the area of ​​a single pixel; is the brightness of the target pixel of the flock; is the theoretical average area occupied by each sheep on the satellite image; Among them, for each satellite image block j corresponding to the UAV orthophoto sheep data used as a training model, the number of sheep obtained in step 1 is used , according to formula (2), calculate the average vertical projection area of ​​each sheep in the flock target : = (2) according to{ }, j∈[1, ], j∈Z, and the theoretical average area occupied by each sheep on the satellite image block is obtained : （3） in, is the number of sheep flocks used to train the sheep population estimation model, is the number of sheep in the jth flock in the UAV orthophoto.

2. The method for extracting the number of sheep in pastoral areas from sub-meter satellite images according to claim 1 is characterized in that: In the step 1, the optical satellite image preprocessing includes performing orthorectification on the panchromatic and multispectral images, and then performing stitching, histogram matching, and fusion, and performing low-pass filtering on the fused images; The preprocessing of the drone orthophotos includes stitching the drone orthophotos, generating a spatial layer using the ArcGIS platform, using the optical satellite images as reference images, selecting control points using characteristic features to align the drone orthophotos, and obtaining the number of sheep in each flock by visually interpreting the aligned drone orthophotos.

3. The method for extracting the number of sheep in pastoral areas from sub-meter satellite images according to claim 2 is characterized in that: The preprocessed optical satellite image and the registered UAV orthophoto are superimposed and displayed in ArcGIS. For each flock of sheep in the registered UAV orthophoto, the same flock is searched from near to far on the preprocessed optical satellite image. A shpfile surface element is created for each corresponding flock based on the optical satellite image layer to record the number of sheep in the corresponding flock obtained based on the visual interpretation of the UAV orthophoto.

4. The method for extracting the number of sheep in pastoral areas from sub-meter satellite images according to claim 1 is characterized in that: The step 2 comprises: Build a target detection model based on deep neural network; Construct a training data set based on the preprocessed optical satellite image set: cut each preprocessed optical satellite image into image blocks, mark the location of the flock in each image block with a marker box, and store the corresponding category, coordinates, width and height information in the label file to construct a training data set including optical satellite images and label files; Train the object detection model using the training dataset; The preprocessed optical satellite image is substituted into the trained target detection model to obtain the boundary of the sheep herd in the pastoral area. The optical satellite image is cropped according to the boundary to obtain the satellite image blocks of each sheep herd.

5. The method for extracting the number of sheep in pastoral areas from sub-meter satellite images according to claim 1 is characterized in that: The step 3 includes constructing a classification system for the satellite image blocks of each flock of sheep according to the objects in the satellite image blocks, and classifying them using any method of support vector machine, random forest, decision tree, and minimum distance to distinguish the flock targets and background.

6. A device for extracting the number of sheep in pastoral areas from sub-meter satellite images, characterized in that: include: The image acquisition module is used to synchronously collect optical satellite images and drone orthophotos and perform preprocessing, and obtain the number of sheep in each flock by visual interpretation of the registered drone orthophotos; The detection module is used to build a target detection model based on a deep neural network, detect the sheep in the preprocessed optical satellite image, and crop the image according to the detection frame to obtain the satellite image block of each sheep; The classification module is used to classify the satellite image blocks of each flock of sheep and distinguish the flock target from the background; The estimation module is used to construct a sheep quantity estimation model based on the brightness difference between the sheep flock target and the background, and train the sheep quantity estimation model using the preprocessed UAV orthophoto to obtain the optimal sheep quantity estimation model, including: In the sheep population estimation model, for each satellite image block, the maximum brightness value of the target pixel of the flock is selected as , the background brightness mean is taken as , calculate the number of sheep in the flock N: (1) In the formula, is the area of ​​a single pixel; is the brightness of the target pixel of the flock; is the theoretical average area occupied by each sheep on the satellite image; Among them, for each satellite image block j corresponding to the UAV orthophoto sheep data used as a training model, the number of sheep obtained in the image acquisition module is used , according to formula (2), calculate the average vertical projection area of ​​each sheep in the flock target : = (2) according to{ }, j∈[1, ], j∈Z, and the theoretical average area occupied by each sheep on the satellite image block is obtained. : （3） in, is the number of sheep flocks used to train the sheep population estimation model, is the number of sheep in the jth flock in the UAV orthophoto.

7. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; Wherein, when one or more programs are executed by the one or more processors, the one or more processors implement the method for extracting the number of sheep in pastoral areas from sub-meter satellite images as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, which, when executed by a processor, enable the processor to implement the method for extracting the number of sheep herds in pastoral areas from sub-meter satellite images as described in any one of claims 1-5.

Images