Vegetation identification method, device, equipment and readable storage medium
By combining feature coding networks with three-band and red-edge band data, the accuracy of vegetation recognition in low-resolution images is improved, and the problem of low vegetation recognition accuracy in the regulation of palm planting area in Southeast Asia is solved, providing cost-effective vegetation Plant assessment.
Patent Information
- Application Number
- CN202010879884.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-27
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2040-08-27
AI Technical Summary
In the prior art, when vegetation recognition is performed through low-resolution images, the accuracy rate is not high, making it difficult to effectively evaluate vegetation planting. Especially in the regulation of palm planting area in Southeast Asia, high-definition remote sensing images are costly and low update frequency.
Image data including red, green and blue data and red edge band data are used to identify vegetation through feature encoding networks and decoding networks, and combined with semantic graph change analysis and area proportion calculation, the accuracy of the recognition results is improved.
It improves the accuracy of vegetation recognition in low-resolution images, can effectively evaluate vegetation planting, provide visual changes in area proportion, and supports the formulation of economic policies.
Smart Images

Figure CN112001341B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a vegetation recognition method, device, equipment and readable storage medium. Background Art
[0002] Since vegetation can bring economic value and environmental value, maintaining and regulating the planting area of various vegetation has always been a focus of attention and the direction of efforts. For example, in Southeast Asia, palm tree (oil coconut) planting is an important economic source in the region, and it is also a factor that causes tropical forest destruction. Therefore, it is necessary to weigh the impact of palm trees on tropical forest destruction and the economic effects they bring, and regulate their planting area. At present, vegetation planting area is obtained by vegetation identification based on high-definition remote sensing effects as a basis for economic policy. However, the cost of obtaining high-definition remote sensing images is high, and the update frequency of high-definition remote sensing images is low, which is not conducive to evaluating vegetation planting conditions. Using low-resolution images for identification is also a way, but because the vegetation images in low-resolution images are blurred and the vegetation features are not obvious, it is difficult to obtain vegetation identification results with high accuracy. Summary of the invention
[0003] The main purpose of the present invention is to provide a vegetation recognition method, device, equipment and readable storage medium, aiming to solve the problem that the accuracy of vegetation recognition results currently obtained by vegetation recognition through low-resolution images is not high.
[0004] To achieve the above object, the present invention provides a vegetation identification method, which comprises the following steps:
[0005] Acquire image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range;
[0006] The image data is input into a preset vegetation recognition model for recognition, to obtain a vegetation recognition result corresponding to the image data.
[0007] Optionally, the step of inputting the image data into a preset vegetation recognition model for recognition to obtain a vegetation recognition result corresponding to the image data includes:
[0008] Inputting the three-color band data into the first feature coding network in the preset vegetation recognition model for coding to obtain a first feature coding result;
[0009] Inputting the red edge band data into the second feature encoding network in the preset vegetation recognition model for encoding to obtain a second feature encoding result;
[0010] The first feature coding result and the second feature coding result are combined and input into a decoding network in the preset vegetation recognition model for decoding to obtain a vegetation recognition result corresponding to the image data.
[0011] Optionally, when the vegetation recognition result is a semantic map representing whether each pixel in the image data belongs to a vegetation area,
[0012] After the step of inputting the image data into a preset vegetation recognition model for recognition to obtain a vegetation recognition result corresponding to the image data, the method further includes:
[0013] After identifying the semantic graphs corresponding to the image data of the to-be-identified geographical area at different time points, comparing the semantic graphs to obtain a change area of the semantic graph at a later time point relative to the semantic graph at a previous time point;
[0014] After marking the changes in the semantic graph at the subsequent time point according to the change category of the change region, the semantic graph at each time point is output.
[0015] Optionally, when the vegetation recognition result is a result indicating whether each pixel in the image data belongs to a vegetation area,
[0016] After the step of inputting the image data into a preset vegetation recognition model for recognition to obtain a vegetation recognition result corresponding to the image data, the method further includes:
[0017] Counting the number of pixels belonging to the vegetation area in the image data according to the vegetation recognition result;
[0018] The number of pixels is divided by the total number of pixels of the image data to obtain the percentage of vegetation area in the geographic area to be identified.
[0019] Optionally, after the step of dividing the number of pixels by the total number of pixels of the image data to obtain the percentage of vegetation area in the to-be-identified geographical area, the method further comprises:
[0020] After obtaining the vegetation area ratios at different time points based on the image data of the to-be-identified geographical area at different time points, the vegetation area ratios are visualized and output.
[0021] Optionally, before the step of acquiring image data corresponding to the geographic area to be identified, the method further includes:
[0022] Acquire a high-resolution image corresponding to the same actual area as the low-resolution image to be annotated, and acquire a first annotated area in the high-resolution image, wherein the resolution of the low-resolution image is lower than that of the high-resolution image;
[0023] Mapping the first annotated area to the low-resolution image to obtain a second annotated area in the low-resolution image;
[0024] Annotating the second annotated area in the low-resolution image to obtain image annotated data;
[0025] The low-resolution image and the image annotation data are used as training data, and a plurality of the training data are used to train the vegetation recognition model to be trained, so as to obtain the preset vegetation recognition model.
[0026] Optionally, the bands within the red edge range include at least one or more of Sentinel-2 Band 5, Band 6 and Band 7.
[0027] To achieve the above object, the present invention provides a vegetation identification device, the device comprising:
[0028] An acquisition module, used for acquiring image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range;
[0029] The recognition module is used to input the image data into a preset vegetation recognition model for recognition, so as to obtain a vegetation recognition result corresponding to the image data.
[0030] To achieve the above objectives, the present invention also provides a vegetation identification device, which includes: a memory, a processor, and a vegetation identification program stored in the memory and executable on the processor, wherein the vegetation identification program implements the steps of the vegetation identification method described above when executed by the processor.
[0031] In addition, to achieve the above-mentioned purpose, the present invention further proposes a computer-readable storage medium, on which a vegetation recognition program is stored, and when the vegetation recognition program is executed by a processor, the steps of the vegetation recognition method described above are implemented.
[0032] In the present invention, by obtaining image data including three-color band data and red edge band data corresponding to the geographic area to be identified, and inputting the image data into a preset vegetation recognition model for identification, a vegetation recognition result corresponding to the image data is obtained. In the present invention, in addition to the commonly used three-color band data, the image data also includes red edge band data that can reflect the vegetation status, so that the preset vegetation recognition model can make a recognition result based on the vegetation features contained in the three-color band data and the red edge band data. That is, compared with the existing method of performing vegetation recognition based only on three-color band data, the present invention increases the richness of image content by adding red edge band data, which is more helpful for the model to understand the image and extract information, thereby making the vegetation recognition result more accurate. At the same time, the image features added by the red edge band data can also make up for the lack of blurring of the target in the low-resolution image, thereby improving the accuracy of the result of vegetation recognition through the low-resolution image. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;
[0034] Figure 2 It is a flow chart of the first embodiment of the vegetation identification method of the present invention;
[0035] Figure 3 A semantic graph involved in an embodiment of the present invention;
[0036] Figure 4 A semantic graph involved in an embodiment of the present invention;
[0037] Figure 5 A semantic graph involved in an embodiment of the present invention;
[0038] Figure 6 This is a functional schematic module diagram of a preferred embodiment of the vegetation identification device of the present invention.
[0039] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0040] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0041] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.
[0042] It should be noted that the vegetation identification device in the embodiment of the present invention can be a smart phone, a personal computer, a server and other devices, and no specific limitation is made here.
[0043] like Figure 1 As shown, the vegetation identification device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0044] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the vegetation identification device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.
[0045] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a vegetation recognition program. The operating system is a program that manages and controls the hardware and software resources of the device and supports the operation of the vegetation recognition program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; the processor 1001 can be used to call the vegetation recognition program stored in the memory 1005 and perform the following operations:
[0046] Acquire image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range;
[0047] The image data is input into a preset vegetation recognition model for recognition, to obtain a vegetation recognition result corresponding to the image data.
[0048] Furthermore, the step of inputting the image data into a preset vegetation recognition model for recognition to obtain a vegetation recognition result corresponding to the image data includes:
[0049] Inputting the three-color band data into the first feature coding network in the preset vegetation recognition model for coding to obtain a first feature coding result;
[0050] Inputting the red edge band data into the second feature encoding network in the preset vegetation recognition model for encoding to obtain a second feature encoding result;
[0051] The first feature coding result and the second feature coding result are combined and input into a decoding network in the preset vegetation recognition model for decoding to obtain a vegetation recognition result corresponding to the image data.
[0052] Furthermore, when the vegetation recognition result is a semantic map representing whether each pixel in the image data belongs to a vegetation area,
[0053] After the step of inputting the image data into a preset vegetation recognition model for recognition and obtaining a vegetation recognition result corresponding to the image data, the processor 1001 may also be used to call a vegetation recognition program stored in the memory 1005 and perform the following operations:
[0054] After identifying the semantic graphs corresponding to the image data of the to-be-identified geographical area at different time points, comparing the semantic graphs to obtain a change area of the semantic graph at a later time point relative to the semantic graph at a previous time point;
[0055] After marking the changes in the semantic graph at the subsequent time point according to the change category of the change region, the semantic graph at each time point is output.
[0056] Furthermore, when the vegetation recognition result is a result indicating whether each pixel in the image data belongs to a vegetation area,
[0057] After the step of inputting the image data into a preset vegetation recognition model for recognition and obtaining a vegetation recognition result corresponding to the image data, the processor 1001 may also be used to call a vegetation recognition program stored in the memory 1005 and perform the following operations:
[0058] Counting the number of pixels belonging to the vegetation area in the image data according to the vegetation recognition result;
[0059] The number of pixels is divided by the total number of pixels of the image data to obtain the percentage of vegetation area in the geographic area to be identified.
[0060] Further, after the step of dividing the number of pixels by the total number of pixels of the image data to obtain the percentage of the vegetation area in the to-be-identified geographic area, the processor 1001 may also be used to call a vegetation identification program stored in the memory 1005, and perform the following operations:
[0061] After obtaining the vegetation area ratios at different time points based on the image data of the to-be-identified geographical area at different time points, the vegetation area ratios are visualized and output.
[0062] Furthermore, before the step of acquiring image data corresponding to the geographic area to be identified, the processor 1001 may also be used to call a vegetation identification program stored in the memory 1005 and perform the following operations:
[0063] Acquire a high-resolution image corresponding to the same actual area as the low-resolution image to be annotated, and acquire a first annotated area in the high-resolution image, wherein the resolution of the low-resolution image is lower than that of the high-resolution image;
[0064] Mapping the first annotated area to the low-resolution image to obtain a second annotated area in the low-resolution image;
[0065] Annotating the second annotated area in the low-resolution image to obtain image annotated data;
[0066] The low-resolution image and the image annotation data are used as training data, and a plurality of the training data are used to train the vegetation recognition model to be trained, so as to obtain the preset vegetation recognition model.
[0067] Furthermore, the bands within the red edge range include at least one or more of Sentinel-2 Band 5, Band 6 and Band 7.
[0068] Based on the above structure, various embodiments of the vegetation recognition method are proposed.
[0069] Reference Figure 2 , Figure 2 This is a flowchart of the first embodiment of the vegetation identification method of the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here. The execution subject of the vegetation identification method of the present invention may be a device such as a smart phone, a personal computer, and a server. The execution subject is omitted in the following embodiments. In this embodiment, the vegetation identification method includes:
[0070] Step S10, obtaining image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range;
[0071] In this embodiment, for a geographical area that needs to be identified with vegetation, the geographical area can be used as a geographical area to be identified. It should be noted that, depending on the specific task, the vegetation in this embodiment can refer to one or several vegetations, that is, the target object to be identified is one or several vegetations, or it can refer to the general term for all vegetations, that is, the target object is all vegetations, and no distinction is made between various vegetations. Similarly, depending on the specific task, vegetation identification can specifically refer to identifying the area where the vegetation in the geographical area to be identified is located, or it can refer to identifying the growth of vegetation, and so on.
[0072] After determining the geographic area to be identified, image data corresponding to the geographic area to be identified can be obtained, that is, the image data is image data obtained by photographing the geographic area to be identified. The image data can be image data with a higher resolution or image data with a lower resolution, that is, in this embodiment, the resolution of the image data is not limited. The image data can come from map software or from aerial photography equipment such as remote sensing satellites and drones.
[0073] In this embodiment, the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range. Among them, the red, green and blue bands are the red band, the green band and the blue band; the band within the red edge range is the band within the red edge range, and the red edge range is approximately 680-740nm. In this embodiment, one or more bands within the red edge range can be used, and the specific band value range is not limited. Furthermore, in one embodiment, the image data can be derived from the Sentinel-2 multispectral imaging satellite, and the band within the red edge range can include at least one or more of Sentinel-2 Band5, Band6 and Band7. It should be noted that the band within the red edge range is very effective for monitoring vegetation health information.
[0074] Step S20: input the image data into a preset vegetation recognition model for recognition, and obtain a vegetation recognition result corresponding to the image data.
[0075] The image data of the geographic area to be identified is input into a preset vegetation identification model for identification, and a vegetation identification result corresponding to the image data is obtained. The preset vegetation identification model may be a pre-set model for vegetation identification of image data. The model may adopt different image identification model structures according to different vegetation identification targets. For example, when the target is to identify the area where the vegetation is located in the image data, a commonly used image semantic segmentation model may be adopted. When the target is to identify the growth of the vegetation in the image data, a commonly used image classification model may be adopted. The preset vegetation identification model may be obtained by pre-training with image training data containing three-color band data and red edge band data. Through training with a large amount of image training data, the preset vegetation identification model learns the ability to extract vegetation features based on the three-color band data and the red edge band data in the image data, and then perform vegetation identification based on the vegetation features. Therefore, the preset vegetation identification model may be used to process the image data of the geographic area to be identified, extract the vegetation features contained in the three-color band data and the red edge band data, and then perform identification based on the vegetation features to obtain the vegetation identification result corresponding to the image data.
[0076] It should be noted that the form of vegetation recognition results may be different depending on the target of vegetation recognition. For example, when the target of vegetation recognition is to identify the area where vegetation is located in the image data, the vegetation recognition result may be data indicating whether each pixel in the image data belongs to the vegetation area. When the target of vegetation recognition is to identify the growth condition of vegetation in the image data, the vegetation recognition result may be data indicating the generation condition of vegetation in the image data, wherein different categories of generation conditions may be predefined.
[0077] In this embodiment, by obtaining image data including three-color band data and red edge band data corresponding to the geographic area to be identified, and inputting the image data into a preset vegetation recognition model for identification, a vegetation recognition result corresponding to the image data is obtained. In this embodiment, in addition to the commonly used three-color band data, the image data also includes red edge band data that can reflect the vegetation status, so that the preset vegetation recognition model can make a recognition result based on the vegetation features contained in the three-color band data and the red edge band data. That is, compared with the existing method of performing vegetation recognition based only on three-color band data, in this embodiment, by adding red edge band data, the richness of image content is increased, which is more helpful for the model to understand the image and extract information, thereby making the vegetation recognition result more accurate. At the same time, the image features added by the red edge band data can also make up for the lack of blurring of the target in the low-resolution image, and improve the accuracy of the result of vegetation recognition through the low-resolution image.
[0078] Further, based on the above first embodiment, a second embodiment of the vegetation recognition method of the present invention is proposed. In this embodiment, step S20 includes:
[0079] Step S201, inputting the three-color band data into the first feature coding network in the preset vegetation recognition model for coding to obtain a first feature coding result;
[0080] In this embodiment, the preset vegetation recognition model may include a first feature encoding network, a second feature encoding network and a decoding network, wherein the first feature encoding network and the second feature encoding network may adopt the encoding network in a commonly used image recognition model, and the decoding network may also adopt the decoding network in a commonly used image recognition model. Different from setting a feature encoding network and a decoding network in a commonly used image recognition model, in this embodiment, two parallel feature encoding networks are set, the first feature encoding network uses three-color band data as input data, and the second feature encoding network uses red edge band data as input data, and the outputs of the two feature encoding networks are merged and used as input data of the decoding network.
[0081] After acquiring the image data of the geographic area to be identified, the three-color band data in the image data is input into the first feature encoding network in the preset vegetation recognition model for encoding processing to obtain a first feature encoding result. According to different settings of the output layer of the first feature encoding network, the form of the first feature encoding network result is different. For example, the first feature encoding network result can be a vector of a predetermined length, for example, a vector of 512 elements. The first feature encoding result represents the vegetation features extracted from the three-color band data.
[0082] Step S202, inputting the red edge band data into the second feature encoding network in the preset vegetation recognition model for encoding to obtain a second feature encoding result;
[0083] The red edge band data in the image data is input into the second feature encoding network in the preset vegetation recognition model for encoding processing to obtain a second feature encoding result. According to different settings of the second feature encoding output layer, the second feature encoding result has different forms. For example, the second feature encoding result can be a vector of a predetermined length, for example, a vector of 512 elements. The first feature encoding result represents the vegetation features extracted from the red edge band data.
[0084] Step S203: The first feature coding result and the second feature coding result are combined and input into a decoding network in the preset vegetation recognition model for decoding to obtain a vegetation recognition result corresponding to the image data.
[0085] The first feature encoding result and the second feature encoding result are combined, and the combined result is input into the decoding network in the preset vegetation recognition model for decoding to obtain the vegetation recognition result corresponding to the image data. There are many ways to combine the first feature encoding result and the second feature encoding result. For example, when the two results are in vector form, vector splicing can be performed, and when the two results are in matrix form, matrix splicing can be performed. For example, when the first feature encoding result and the second feature encoding result are both vectors of 512 elements, the two vectors can be spliced to obtain a vector of 1024 elements. The splicing method can be to shuffle the elements in the two vectors so that the vegetation features extracted based on the three-color band data and the vegetation features extracted based on the red edge band data are integrated, so that the decoding network can make vegetation recognition results based on the two types of vegetation features more fully.
[0086] The existing method of mixing various band data and inputting them into the recognition model for recognition will make the features in the various band data unclear. In this embodiment, the three-color band data in the image data is input into the first feature coding network for feature coding, and the red edge band data in the image data is input into the second feature coding network for feature coding, so that the preset vegetation recognition model can extract the separate vegetation features in the three-color band data extraction, and can extract the separate vegetation features in the red edge band data, that is, to clearly and fully extract the vegetation features in the two types of band data, and then merge the two types of vegetation features and input them into the decoding network to obtain the vegetation recognition result, so that the decoding network can make a more accurate vegetation recognition result based on the two types of feature coding results.
[0087] Further, based on the above-mentioned first and / or second embodiments, a third embodiment of the vegetation identification method of the present invention is proposed. In this embodiment, the method further includes:
[0088] Step S30, after identifying the semantic graphs corresponding to the image data of the to-be-identified geographical area at different time points, comparing the semantic graphs to obtain a change area of the semantic graph at a later time point relative to the semantic graph at a previous time point;
[0089] In this embodiment, the vegetation recognition result can be a semantic map that characterizes whether each pixel in the image data belongs to the vegetation area. The semantic map is a map of the same size as the image data, that is, each pixel in the semantic map corresponds to each pixel in the image data one by one. There are two types of pixels in the semantic map, one is the pixel belonging to the vegetation area, and the other is the pixel not belonging to the vegetation area. The two types of pixels have different colors in the semantic map to indicate their different categories.
[0090] For the geographic area to be identified, image data of the geographic area to be identified at different time points can be collected, and according to the vegetation identification method in the above embodiment, a semantic map corresponding to each image data is obtained, that is, a semantic map corresponding to each time point is obtained. For example, image data of the geographic area to be identified can be collected once every period of time, such as once a year, a month, or a day.
[0091] For the semantic graphs corresponding to each time point, each semantic graph is compared to obtain the change area of the semantic graph at the later time point relative to the semantic graph at the previous time point. That is, the semantic graphs are compared in pairs, one of the two semantic graphs is at the previous time point, and the other is at the later time point. By comparing the pixels of different categories in the two semantic graphs, it can be obtained that the pixels of the semantic graph at the later time point have changed in category relative to the semantic graph at the previous time point, and the area formed by these pixels is the change area.
[0092] For example, image data of the geographic area to be identified at three time points in 2016, 2017 and 2018 are collected, and the semantic maps corresponding to the three image data are identified; by comparing the semantic maps of 2016 and 2017, it is found that there are two changed areas 1 and 2 in 2017 relative to 2016, 1 is the newly added vegetation area, and 2 is the vegetation area that has been cut off in the original vegetation area; by comparing the semantic maps of 2017 and 2018, it is found that there are two changed areas 3 and 4 in 2018 relative to 2017, 3 is the newly added vegetation area, and 4 is the vegetation area that has been cut off in the original vegetation area; if the change in 2018 relative to 2016 is to be analyzed, the semantic maps of 2016 and 2018 can be compared, and two changed areas 5 and 6 in 2018 relative to 2016 can be obtained, 5 is the newly added vegetation area, and 6 is the vegetation area that has been cut off in the original vegetation area. It can be understood that 5 and 3 may overlap, and 6 and 4 may overlap.
[0093] Step S40 , after marking the changes in the semantic graph at the subsequent time point according to the change category of the change region, output the semantic graph at each time point.
[0094] According to the change category of the change area, the change mark is made in the semantic map at the later time point. Among them, the change category can include two categories: newly added vegetation and cut vegetation. First, the change area is marked so that the change area is distinguished from the original semantic map. Secondly, the change category is different, and the change mark can be different. For example, different colors are used to distinguish the change category, or different picture filling textures are used to distinguish the change category, etc. It should be noted that when the semantic map at the later time point corresponds to multiple semantic maps at the previous time point, the change area of the semantic map at the later time point relative to the semantic map of the multiple previous time points can be merged and then marked. For example, in the above example, 5 and 3 have overlapping areas, then the overlapping area is used as the newly added vegetation area in 2017 in the 2018 semantic map, and the non-overlapping area in 5 is used as the newly added vegetation area in 2018 in the 2018 semantic map, and different change marks are made for these two newly added vegetation areas; similarly, 6 and 4 can also be processed similarly.
[0095] After marking the changes in the semantic graph at the later time point, the semantic graph at each time point can be output. Specifically, it can be output to a display screen for users to intuitively understand the changes in the vegetation planting area of the geographic area to be identified at different time points, thereby facilitating users to identify the changes in the vegetation planting area based on the changes in the vegetation planting area.
[0096] For example, the image data collected at the first time point, the second time point, and the third time point are identified to obtain semantic graphs corresponding to the three time points, and the change area of the semantic graph at the later time point relative to the semantic graph at the previous time point is marked, and the following is obtained: Figure 3 , Figure 4 and Figure 5 The semantic diagram shown in Figure 1 is shown in Figure 1. The time sequence is: the first time point comes first, the second time point comes second, and the third time point comes last. Figure 3 As shown in , it is the semantic map corresponding to the first time point, where area 1 represents the vegetation area and area 2 represents the non-vegetation area; Figure 4 The semantic map corresponding to the second time point is shown in FIG. 3 , where area 3 represents the newly added vegetation area at the second time point relative to the first time point; Figure 5 The figure shows the semantic map corresponding to the third time point, where area 3 represents the newly added vegetation area at the second time point relative to the first time point, area 4 represents the newly added vegetation area at the third time point relative to the second time point, and area 5 represents the vegetation area that has been cut down at the third time point relative to the second time point.
[0097] Furthermore, the method further comprises:
[0098] Step S50, counting the number of pixels belonging to the vegetation area in the image data according to the vegetation recognition result;
[0099] Step S60: Divide the number of pixels by the total number of pixels of the image data to obtain the percentage of vegetation area in the to-be-identified geographic area.
[0100] In one embodiment, the vegetation recognition result may be a result that characterizes whether each pixel in the image data belongs to the vegetation area, such as a semantic map or a confidence matrix. Based on the vegetation recognition result, the number of pixels in the image data that belong to the vegetation area may be counted. After the number of pixels belonging to the vegetation area is obtained, the number of pixels may be divided by the total number of pixels in the image data, and the calculation result may be used as the area ratio of the vegetation area in the geographic area to be identified, that is, the area ratio of the vegetation area in the geographic area to be identified to the entire geographic area to be identified is obtained.
[0101] Furthermore, after obtaining the vegetation area ratio, the area ratio can be output, specifically, it can be output in the form of a numerical value to be displayed on the display screen. Figure 1 The output is used to help users intuitively understand the planting situation of vegetation through semantic maps and area proportions, and then plan and adjust the planting area based on the planting situation to adapt to economic policies.
[0102] Furthermore, after step S60, the method further includes:
[0103] Step S70, after obtaining the vegetation area ratios at different time points based on the image data of the to-be-identified geographical area at different time points, the vegetation area ratios are visualized and output.
[0104] For the geographic area to be identified, image data of the geographic area to be identified at different time points can be collected, and according to the vegetation identification method in the above embodiment, the vegetation identification results corresponding to each image data can be obtained, and the vegetation area ratio can be calculated based on the vegetation identification results, so as to obtain the vegetation area ratio corresponding to each time point. The area ratio of each vegetation area is output visually. Specifically, the vegetation area ratio at each time point can be output visually in the form of a line graph with the horizontal axis as time and the vertical axis as area ratio, so that the user can intuitively understand the changing trend of the vegetation planting area in the geographic area to be identified over time, and then use the changing trend as a strong basis for economic policy.
[0105] Further, based on the above-mentioned first, second and / or third embodiments, a fourth embodiment of the vegetation identification method of the present invention is proposed. In this embodiment, the method further includes:
[0106] Step A10, obtaining a high-resolution image corresponding to the same actual area as the low-resolution image to be annotated, and obtaining a first annotated area in the high-resolution image, wherein the resolution of the low-resolution image is lower than that of the high-resolution image;
[0107] In this embodiment, a large number of low-resolution images can be collected in advance for training the vegetation recognition model to be trained. Since vegetation in low-resolution images is often only presented in the form of 1 or 2 pixels, the human eye cannot distinguish them at all, so it is impossible to annotate the vegetation data on the low-resolution images. It is obviously difficult to train a vegetation recognition model with good recognition effect without annotated data or with only a small amount of annotated data.
[0108] In this embodiment, data annotation is performed on low-resolution images with the help of high-resolution images. High-resolution images and low-resolution images are relative, that is, as long as the resolution of the high-resolution image is higher than that of the low-resolution image, there is no specific limit on the resolution of the high-resolution image and the low-resolution image. For example, when the resolution of the low-resolution image is 10m (indicating that one pixel represents a range of 10m*10m in the actual geographical area), the resolution of the high-resolution image can be greater than 10m. It should be noted that the larger the resolution is in numerical value, the lower the resolution is. For example, the resolution of 10m is lower than the resolution of 0.3m.
[0109] For the low-resolution image to be annotated, a high-resolution image of the same actual area as the low-resolution image may be obtained. The high-resolution image may be obtained from a remote sensing satellite or from a map software, which is not limited in this embodiment. A first annotated area in the high-resolution image is obtained. The first annotated area may be a vegetation area or a non-vegetation area in the high-resolution image.
[0110] The first annotated area can be defined by a coordinate interval in a relative coordinate system such as a pixel interval in a high-resolution image, or by a coordinate interval in an absolute coordinate system such as a longitude and latitude interval, and is not specifically limited in this embodiment. It can be understood that the relative coordinate system is a coordinate system constructed based on a single image, and the coordinate points therein are irrelevant to other images. The absolute coordinate system is a coordinate system constructed based on an actual geographical area, and any point in an image has a corresponding coordinate point in the absolute coordinate system. In addition to a coordinate system constructed with pixels in an image as units, the relative coordinate system can also be a coordinate system constructed with length units such as centimeters or millimeters, and the origin of the relative coordinate system can be any point in the high-resolution image. In addition to the longitude and latitude system, the absolute coordinate system can also be a coordinate system commonly used in other maps, or a custom map coordinate system.
[0111] Step A20, mapping the first annotated area to the low-resolution image to obtain a second annotated area in the low-resolution image;
[0112] The first annotated area is mapped to the low-resolution image to be annotated to obtain a second annotated area in the low-resolution image. There are multiple mapping methods. When the first annotated area is defined by a relative coordinate system, the mapping is performed according to the conversion relationship between the relative coordinate systems of the high-resolution and low-resolution images. When the first annotated area is defined by an absolute coordinate system, the mapping is performed according to the absolute coordinate interval of the first annotated area. In this embodiment, there is no limitation on the mapping method.
[0113] It should be noted that when the first labeled area is a vegetation area, the second labeled area refers to the vegetation area in the low-resolution image; when the first labeled area is a non-vegetation area, the second labeled area refers to the non-vegetation area in the low-resolution image.
[0114] Step A30, annotating the second annotated area in the low-resolution image to obtain image annotated data;
[0115] The second annotated area in the low-resolution image is annotated to obtain image annotation data. Specifically, the image annotation data may be data indicating whether each pixel in the low-resolution image belongs to a vegetation area. Then, the second annotated area is annotated to indicate whether each pixel in the second annotated area in the low-resolution image belongs to a vegetation area. Specifically, when the second annotated area is a vegetation area, all pixels in the second annotated area are annotated as belonging to the vegetation area, for example, annotated as 1. Further, all pixels outside the second annotated area in the low-resolution image are annotated as belonging to a non-vegetation area, for example, annotated as 0. When the second annotated area is a non-vegetation area, all pixels in the second annotated area are annotated as belonging to a non-vegetation area, for example, annotated as 0. Further, all pixels outside the second annotated area in the low-resolution image are annotated as belonging to a vegetation area, for example, annotated as 1.
[0116] Step A40: Use the low-resolution image and the image annotation data as training data, and use a plurality of pieces of the training data to train the vegetation recognition model to be trained, so as to obtain the preset vegetation recognition model.
[0117] Furthermore, after obtaining image annotation data corresponding to multiple low-resolution images using the above-mentioned data annotation method, a training data set can be constructed based on the multiple low-resolution images and the corresponding image annotation data, and the training data set can be used to train the vegetation recognition model to be trained. The trained model is used as a preset vegetation recognition model, and then the vegetation in the low-resolution image of the geographical area to be identified can be identified based on the preset vegetation recognition model.
[0118] In this embodiment, a high-resolution image corresponding to the same actual area as the low-resolution image to be annotated is obtained, and a first annotated area in the high-resolution image is obtained, the first annotated area is mapped to the low-resolution image to obtain a second annotated area, and the second annotated area in the low-resolution image is annotated to obtain image annotation data, and the data annotation of the low-resolution image is completed, and then the vegetation recognition model is trained using the annotated low-resolution image to obtain a preset vegetation recognition model. In this embodiment, the low-resolution image is annotated with the help of the high-resolution image. Since the vegetation in the high-resolution image is clearly visible, it can be accurately mapped to the low-resolution image, and the accurate annotation of the vegetation in the low-resolution image is achieved. That is, compared with the existing method of manually annotating low-resolution image data, the data annotation method in this embodiment will not have the problem of unclear vegetation in the low-resolution image and thus unable to be annotated, and while improving the accuracy of data annotation, it also realizes an automated image data annotation method, and in the scenario where massive image training data is required, the construction efficiency of the training data set is greatly improved, and the labor cost and time cost are reduced.
[0119] Further, in one implementation, the first marked area is mapped to the second marked area in the following manner:
[0120] An absolute coordinate system may be pre-set, and the absolute coordinate system may be a latitude and longitude coordinate system, or other commonly used map coordinate systems, or a custom map coordinate system. It is understandable that, whether it is a low-resolution image or a high-resolution image, each pixel in the image obeys the absolute coordinate system, that is, each pixel in the image has a corresponding coordinate point in the absolute coordinate system.
[0121] Get the absolute coordinate interval corresponding to the first marked area, that is, the first marked area corresponds to an absolute coordinate interval in the absolute coordinate system, and the absolute coordinate interval can be an interval defined by multiple coordinate points. For example, when the first marked area is a rectangle, the absolute coordinate interval is an interval defined by the coordinate points corresponding to the four vertices of the matrix.
[0122] Since each pixel point in the low-resolution image has a corresponding coordinate point in the absolute coordinate system, conversely, the coordinate points corresponding to each point in the actual geographical area corresponding to the low-resolution image in the absolute coordinate system can all correspond to the pixel points in the low-resolution image. Therefore, the area defined by the absolute coordinate interval can be found in the low-resolution image and used as the second annotation area.
[0123] In addition, the present invention also provides a vegetation recognition device. Figure 6 , the device comprises:
[0124] An acquisition module 10 is used to acquire image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range;
[0125] The recognition module 20 is used to input the image data into a preset vegetation recognition model for recognition, and obtain a vegetation recognition result corresponding to the image data.
[0126] Furthermore, the identification module 20 includes:
[0127] A first input unit, used for inputting the three-color band data into a first feature coding network in the preset vegetation recognition model for coding to obtain a first feature coding result;
[0128] A second input unit is used to input the red edge band data into a second feature encoding network in the preset vegetation recognition model for encoding to obtain a second feature encoding result;
[0129] The third input unit is used to combine the first feature encoding result and the second feature encoding result and input them into the decoding network in the preset vegetation recognition model for decoding to obtain the vegetation recognition result corresponding to the image data.
[0130] Furthermore, when the vegetation recognition result is a semantic map representing whether each pixel in the image data belongs to a vegetation area, the device further includes:
[0131] A comparison module is used to compare the semantic maps corresponding to the image data of the geographic area to be identified at different time points after identifying them, so as to obtain a change area of the semantic map at a later time point relative to the semantic map at a previous time point;
[0132] The marking module is used to output the semantic graph of each time point after marking the change in the semantic graph at the subsequent time point according to the change category of the change area.
[0133] Furthermore, when the vegetation recognition result is a result indicating whether each pixel in the image data belongs to a vegetation area, the device further includes:
[0134] A statistics module, used for counting the number of pixels belonging to the vegetation area in the image data according to the vegetation recognition result;
[0135] The calculation module is used to divide the number of pixels by the total number of pixels of the image data to obtain the area ratio of the vegetation area in the geographic area to be identified.
[0136] Furthermore, the device also includes:
[0137] The output module is used to visualize and output the vegetation area ratios at different time points after obtaining the vegetation area ratios at different time points based on the image data of the geographic area to be identified at different time points.
[0138] Furthermore, the acquisition module 10 is further used to acquire a high-resolution image corresponding to the same actual area as the low-resolution image to be annotated, and acquire a first annotated area in the high-resolution image, wherein the resolution of the low-resolution image is lower than that of the high-resolution image;
[0139] The device also includes:
[0140] A mapping module, used for mapping the first annotated area to the low-resolution image to obtain a second annotated area in the low-resolution image;
[0141] A labeling module, used for labeling the second labeling area in the low-resolution image to obtain image labeling data;
[0142] The training module is used to use the low-resolution image and the image annotation data as training data, and use multiple pieces of the training data to train the vegetation recognition model to be trained to obtain the preset vegetation recognition model.
[0143] Furthermore, the bands within the red edge range include at least one or more of Sentinel-2 Band 5, Band 6 and Band 7.
[0144] The expanded contents of the specific implementation manner of the vegetation identification device of the present invention are basically the same as the various embodiments of the above-mentioned vegetation identification method, and will not be elaborated here.
[0145] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a vegetation recognition program is stored. When the vegetation recognition program is executed by a processor, the steps of the vegetation recognition method described below are implemented.
[0146] The various embodiments of the vegetation identification device and the computer-readable storage medium of the present invention may refer to the various embodiments of the vegetation identification method of the present invention, and will not be described in detail here.
[0147] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0148] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0149] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the embodiments of the present invention.
[0150] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A vegetation recognition method, characterized in that: The method comprises the following steps: Acquire image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range; Inputting the image data into a preset vegetation recognition model for recognition, and obtaining a vegetation recognition result corresponding to the image data; When the vegetation recognition result is a semantic map characterizing whether each pixel in the image data belongs to a vegetation area, after identifying the semantic maps corresponding to the image data of the to-be-identified geographical area at different time points, the semantic maps are compared to obtain a change area of the semantic map at a later time point relative to the semantic map at a previous time point, wherein the semantic map is a map of the same size as the image data, each pixel in the semantic map corresponds one-to-one to each pixel in the image data, and there are two types of pixel points in the semantic map, one type is pixel points belonging to the vegetation area, and the other type is pixel points not belonging to the vegetation area, and the two types of pixel points have different colors in the semantic map to indicate that they are different categories; After marking the changes in the semantic graph at the subsequent time point according to the change category of the change region, the semantic graph at each time point is output.
2. The vegetation identification method according to claim 1, characterized in that: The step of inputting the image data into a preset vegetation recognition model for recognition to obtain a vegetation recognition result corresponding to the image data comprises: Inputting the three-color band data into the first feature coding network in the preset vegetation recognition model for coding to obtain a first feature coding result; Inputting the red edge band data into the second feature encoding network in the preset vegetation recognition model for encoding to obtain a second feature encoding result; The first feature coding result and the second feature coding result are combined and input into a decoding network in the preset vegetation recognition model for decoding to obtain a vegetation recognition result corresponding to the image data.
3. The vegetation identification method according to claim 1, characterized in that: When the vegetation recognition result is a result indicating whether each pixel in the image data belongs to a vegetation area, After the step of inputting the image data into a preset vegetation recognition model for recognition to obtain a vegetation recognition result corresponding to the image data, the method further includes: Counting the number of pixels belonging to the vegetation area in the image data according to the vegetation recognition result; The number of pixels is divided by the total number of pixels of the image data to obtain the percentage of vegetation area in the geographic area to be identified.
4. The vegetation identification method according to claim 3, characterized in that: After the step of dividing the number of pixels by the total number of pixels of the image data to obtain the area ratio of the vegetation area in the to-be-identified geographical area, the method further includes: After obtaining the vegetation area ratios at different time points based on the image data of the to-be-identified geographical area at different time points, the vegetation area ratios are visualized and output.
5. The vegetation identification method according to claim 1, characterized in that: Before the step of obtaining the image data corresponding to the geographic area to be identified, the method further includes: Acquire a high-resolution image corresponding to the same actual area as the low-resolution image to be annotated, and acquire a first annotated area in the high-resolution image, wherein the resolution of the low-resolution image is lower than that of the high-resolution image; Mapping the first annotated area to the low-resolution image to obtain a second annotated area in the low-resolution image; Annotating the second annotated area in the low-resolution image to obtain image annotated data; The low-resolution image and the image annotation data are used as training data, and a plurality of the training data are used to train the vegetation recognition model to be trained, so as to obtain the preset vegetation recognition model.
6. The vegetation recognition method according to any one of claims 1 to 5, characterized in that: The bands within the red edge range include at least one or more of Sentinel 2 Band 5, Band 6 and Band 7.
7. A vegetation recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring image data corresponding to the geographic area to be identified, wherein the image data includes three-color band data corresponding to the red, green and blue bands and red edge band data corresponding to the band within the red edge range; A recognition module, used for inputting the image data into a preset vegetation recognition model for recognition, and obtaining a vegetation recognition result corresponding to the image data; The recognition module is also used for: when the vegetation recognition result is a semantic map characterizing whether each pixel in the image data belongs to a vegetation area, after identifying the semantic maps corresponding to the image data of the geographical area to be identified at different time points, comparing the semantic maps to obtain a change area of the semantic map at the later time point relative to the semantic map at the previous time point, wherein the semantic map is a map of the same size as the image data, each pixel in the semantic map corresponds one-to-one to each pixel in the image data, there are two types of pixel points in the semantic map, one type is pixel points belonging to the vegetation area, and the other type is pixel points not belonging to the vegetation area, and the two types of pixel points have different colors in the semantic map to indicate different categories; after marking the changes in the semantic map at the later time point according to the change category of the change area, the semantic map of each time point is output.
8. A vegetation recognition device, characterized in that: The vegetation identification device comprises: a memory, a processor, and a vegetation identification program stored in the memory and executable on the processor. When the vegetation identification program is executed by the processor, the steps of the vegetation identification method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a vegetation recognition program, and when the vegetation recognition program is executed by the processor, the steps of the vegetation recognition method according to any one of claims 1 to 6 are implemented.
