A method and device for improving the accuracy of crop classification results in semantic segmentation networks

Abstract

The present invention discloses a method and device for improving the accuracy of crop classification results of a semantic segmentation network within a year, and belongs to the field of image processing technology. The method includes: obtaining multiple first images to be tested of a target area in multiple phenological periods; constructing a first probability matrix of multiple first images to be tested according to the probability that the pixel of each first image to be tested is the category of the target crop; determining the first pixel of the cloud area in the multiple first images to be tested; based on the multiple first images to be tested, mapping the first pixel to the first probability matrix, and obtaining the classification result of the target crop in the target area. The obtained crop classification result is more consistent with the actual crop planting area, and the crop planting area can be effectively detected.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and device for improving the accuracy of crop classification results of a semantic segmentation network. Background Art

[0002] The crop planting area and spatial distribution are the basis for estimating crop yields and monitoring their growth, and are also the main basis for optimizing the planting structure.

[0003] At the current stage, based on satellite remote sensing technology, current large-scale crop planting area monitoring mainly uses deep learning neural network models to classify images and output classification results.

[0004] However, there may be certain differences in the classification results of shadow maps in different phenological periods within a year. There are cases where the identified classification results are inconsistent with the actual crop planting area, resulting in the inability to effectively detect the crop planting area. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a method and device for improving the accuracy of crop classification results within the year of a semantic segmentation network, so that the obtained crop classification results are more consistent with the actual crop planting area, thereby achieving effective detection of the crop planting area.

[0006] In a first aspect, the present invention provides a method for improving the accuracy of crop classification results of a semantic segmentation network within a year, the method comprising:

[0007] Acquire a plurality of first images to be measured of the target area in a plurality of phenological periods;

[0008] constructing a first probability matrix of the plurality of first images to be tested according to the probability that a pixel of each of the first images to be tested is a category of a target crop;

[0009] Determine a first pixel in a cloud region in the plurality of first images to be measured;

[0010] Based on the multiple first images to be tested, the first pixels are mapped to the first probability matrix to obtain a classification result of the target crop in the target area.

[0011] According to the method for improving the accuracy of crop classification results of the semantic segmentation network of the present invention, a first probability matrix is ​​constructed through the classification probability of pixels in the first test image collected at different phenological periods, and the characteristic differences presented by phenological changes and the influence of the first pixels blocked by clouds are taken into account to obtain the classification results of the target crops in the target area. The obtained crop classification results can cover the characteristics of different phenological periods, are more in line with the actual crop planting area, can effectively improve the accuracy of classification results and large-scale mapping, and realize effective detection of crop planting area.

[0012] According to an embodiment of the present invention, mapping the first pixel to the first probability matrix based on the multiple first images to be tested to obtain a classification result of the target crop in the target area includes:

[0013] According to the position information of the first pixel in each of the first images to be tested, the elements of the corresponding category probabilities in the first probability matrix are set to be empty to obtain a second probability matrix;

[0014] Based on the second probability matrix, calculating the classification confidence of the first image to be tested to obtain a first prediction matrix;

[0015] The classification result is obtained according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested.

[0016] According to an embodiment of the present invention, obtaining the classification result according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested includes:

[0017] Determining a second image to be measured from among the plurality of first images to be measured according to the phenological period;

[0018] constructing a second prediction matrix according to the normalized vegetation index of the pixels of the second image to be tested;

[0019] According to the first prediction matrix and the second prediction matrix, the spatial distribution area of ​​the target crop in the target area is used as the classification result.

[0020] According to an embodiment of the present invention, determining the second image to be measured from the plurality of first images to be measured according to the phenological period includes:

[0021] Determining a time sequence of the plurality of first images to be measured according to a phenological period corresponding to each of the first images to be measured;

[0022] According to the time sequence, the second image to be tested is screened out from the plurality of first images to be tested.

[0023] According to an embodiment of the present invention, the calculating the classification confidence of the first image to be tested based on the second probability matrix to obtain a first prediction matrix includes:

[0024] Calculating the classification confidence of each of the first images to be tested based on the category probability of each pixel of the first images to be tested in the second probability matrix;

[0025] Determine the target image to be tested with the maximum classification confidence;

[0026] The first prediction matrix is ​​determined according to the category probabilities of the pixels of the target image to be tested in the second probability matrix.

[0027] According to one embodiment of the present invention, determining a first pixel in a cloud region in the plurality of first images to be measured includes:

[0028] A first pixel in the first image to be tested is determined according to the digital quantization value of the pixel of the first image to be tested.

[0029] According to an embodiment of the present invention, constructing a first probability matrix of the plurality of first images to be tested according to the probability that the pixel of each of the first images to be tested is a category of a target crop includes:

[0030] Classify each of the first images to be tested by using a semantic segmentation network to determine the category probability of each pixel of the first images to be tested;

[0031] The first probability matrix is ​​constructed based on the position information of the pixels corresponding to the category probabilities.

[0032] In a second aspect, the present invention provides a device for improving the accuracy of crop classification results of a semantic segmentation network within a year, the device comprising:

[0033] An acquisition module, used for acquiring a plurality of first images to be measured of a target area in a plurality of phenological periods;

[0034] A first processing module, configured to construct a first probability matrix of the plurality of first images to be tested according to a probability that a pixel of each of the first images to be tested is a category of a target crop;

[0035] A second processing module, used for determining a first pixel in a cloud region in the plurality of first images to be measured;

[0036] The third processing module is used to map the first pixel to the first probability matrix based on the multiple first images to be tested, so as to obtain a classification result of the target crop in the target area.

[0037] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for improving the accuracy of crop classification results of the semantic segmentation network within the year as described in the first aspect above is implemented.

[0038] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for improving the accuracy of crop classification results within the year of a semantic segmentation network as described in the first aspect above.

[0039] In a fifth aspect, the present invention provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method for improving the accuracy of crop classification results of the semantic segmentation network within the year as described in the first aspect.

[0040] In a sixth aspect, the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the method for improving the accuracy of crop classification results within the year of the semantic segmentation network as described in the first aspect above.

[0041] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0043] Figure 1 It is a flow chart of a method for improving the accuracy of crop classification results of a semantic segmentation network within a year provided by an embodiment of the present invention;

[0044] Figure 2 is a structural schematic diagram of a device for improving the accuracy of crop classification results of a semantic segmentation network within a year provided by an embodiment of the present invention;

[0045] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0047] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0048] In conjunction with the accompanying drawings, the method for improving the accuracy of crop classification results of a semantic segmentation network within the year, the device for improving the accuracy of crop classification results of a semantic segmentation network within the year, the electronic device and the readable storage medium provided by the embodiments of the present invention are described in detail through specific embodiments and their application scenarios.

[0049] Among them, the method for improving the accuracy of crop classification results of the semantic segmentation network can be applied to the terminal, and can be specifically executed by hardware or software in the terminal.

[0050] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0051] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0052] The embodiment of the present invention provides a method for improving the accuracy of crop classification results of a semantic segmentation network within the year. The execution subject of the method can be an electronic device or a functional module or functional entity in the electronic device that can implement the method for improving the accuracy of crop classification results of a semantic segmentation network within the year. The electronic devices mentioned in the embodiment of the present invention include but are not limited to mobile phones, tablet computers, computers, cameras and wearable devices. The following takes the electronic device as an example of the execution subject to illustrate the method for improving the accuracy of crop classification results of a semantic segmentation network within the year provided by the embodiment of the present invention.

[0053] like Figure 1 As shown, the method for improving the accuracy of crop classification results of the semantic segmentation network includes: steps 110 to 140.

[0054] Step 110: Acquire a plurality of first images to be measured of the target area in a plurality of phenological periods.

[0055] Among them, target crops are planted in the target area, and the target crops show different growth characteristics in different phenological periods. In the subsequent embodiments, rice is used as an example to illustrate the classification of target crops, which is not regarded as limiting the scope of protection of the present invention.

[0056] For example, the phenological stages of rice may include the germination stage, seedling stage, tillering stage, jointing stage, heading stage and maturity stage.

[0057] It can be understood that the first image to be tested is the satellite remote sensing data obtained by collecting images of the target area at different phenological periods during the life cycle of the target crop. There can be m first images to be tested (m>2), and the size of each first image to be tested can be the same, including w×h pixels.

[0058] For example, the first image to be tested may be Landsat 8OLI L1 satellite data used for crop classification.

[0059] Step 120: construct a first probability matrix of the plurality of first images to be tested according to the probability that the pixel of each of the first images to be tested is a category of the target crop.

[0060] The category probability may be obtained by performing image classification on each pixel in each first image to be tested through a deep learning semantic segmentation network.

[0061] In actual execution, each first test image is classified through a deep learning semantic segmentation network to obtain the category probability pls of each pixel in the m first test images being the target crop, and a three-dimensional matrix is ​​constructed to obtain a first probability matrix Apls1 with a size of [w, h, m].

[0062] It can be understood that, in the first probability matrix Apls1, each element corresponds to the probability that a pixel is a category of the target crop.

[0063] For example, the class probability pls of a pixel being rice is 0.7, and the probability of a pixel being non-rice is 0.3.

[0064] Step 130: determine a first pixel in a cloud layer area in the plurality of first images to be measured.

[0065] Among them, the first pixel is a pixel belonging to the cloud area.

[0066] It should be noted that since the first image to be tested is satellite remote sensing data collected from the target area during the phenological period, there may be cloud obstruction. It is necessary to remove the first pixel obstructed by clouds in each first image to be tested to reduce the impact of clouds on image classification and improve recognition accuracy.

[0067] In actual implementation, clouds are usually brighter or whiter than vegetation, and the corresponding grayscale value is higher. Each first image to be tested can be converted into a grayscale image to obtain the grayscale value of each pixel, and the first pixel covered by clouds can be screened out according to the size of the grayscale value.

[0068] Alternatively, clouds show characteristics different from those of vegetation in the infrared band, and the first pixel can be determined by the spectral index of each pixel in the first image to be tested.

[0069] Step 140: Based on the plurality of first images to be tested, map the first pixels to the first probability matrix to obtain a classification result of the target crop in the target area.

[0070] The classification result is whether the position corresponding to the pixel in the target area is the target crop.

[0071] In actual implementation, the element corresponding to the first pixel in the first probability matrix can be emptied according to the position coordinates corresponding to the first pixel in the first image to be tested, and the classification probabilities represented by the remaining elements in the first probability matrix can be integrated to obtain the classification results of the target crops in the target area.

[0072] For example, if the category probabilities of the m pixels corresponding to any coordinate position are greater than or equal to 0.5, the classification result of the coordinate position is rice. If among the category probabilities of the m pixels corresponding to any coordinate position, there is a category probability less than 0.5, then the classification result of the coordinate position is non-rice.

[0073] Among them, the classification results can be used for mapping target crops in the target area.

[0074] In the related technology, for large-scale crop mapping, it is almost impossible to obtain a sufficient number of training samples so that the image features of different phenological periods in the test image can be fully covered, resulting in a certain deviation between the characteristic distribution of training samples under different phenological periods and the characteristic distribution of test samples within a year, resulting in large differences in the mapping results of different phenological images of the same crop within the year.

[0075] The final annual mapping results obtained from the mapping results of multiple images of different phenological periods in a year are:

[0076]

[0077] Among them, Resultpre is the final rice mapping result of the year, m represents the number of images within the rice phenological period of the year, i refers to the image number, P i is the class possibility output by the semantic segmentation network, crop is a specific type of crop, and others are non-crops. The above formula shows that in the output of the semantic segmentation network, if there is a mapping result of a certain crop in any period, then the crop mapping result of that year is rice of that type of crop, otherwise it is other crops.

[0078] For example, a certain area in a plot is identified as a certain crop in some periods, and as a non-crop in other periods. The relevant technical methods cannot fully utilize the confidence information of the classification results output by the network. Due to phenological changes, the same crop presents different characteristics in different phenological periods of the year, and the training samples are often difficult to fully cover the characteristics of different phenological periods. There is a situation where the identified classification results are inconsistent with the actual crop planting area, resulting in the inability to effectively detect the crop planting area.

[0079] According to the method for improving the accuracy of crop classification results of the semantic segmentation network provided by an embodiment of the present invention, a first probability matrix is ​​constructed through the classification probability of pixels in the first test image collected at different phenological periods, and the characteristic differences presented by phenological changes and the influence of the first pixels blocked by clouds are taken into account to obtain the classification results of the target crops in the target area. The obtained crop classification results can cover the characteristics of different phenological periods, are more in line with the actual crop planting area, can effectively improve the accuracy of classification results and large-scale mapping, and realize effective detection of crop planting area.

[0080] In some embodiments, step 120, constructing a first probability matrix of the plurality of first images to be tested according to the probability that each pixel of the first images to be tested is a category of a target crop, comprises:

[0081] Classify each of the first images to be tested by using a semantic segmentation network to determine the category probability of each pixel of the first images to be tested;

[0082] The first probability matrix is ​​constructed based on the position information of the pixels corresponding to the category probabilities.

[0083] The position information of the pixel is used to represent the position coordinates of the pixel in the first image to be measured.

[0084] It should be noted that crop classification based on deep learning can be achieved through recurrent neural networks (RNNs) and semantic segmentation networks. RNNs require input of time-series continuous images, but the cloud-free images mentioned above are difficult to obtain and are not applicable in actual crop mapping. The semantic segmentation network can carry out mapping based on only one remote sensing image, which has higher mapping efficiency and better suitability.

[0085] In actual execution, m first test images of size w×h are input into the semantic segmentation network in sequence, and the category probability pls of each pixel in each first test image being rice is obtained, which is output by the semantic segmentation network. The category probability pls of each pixel is arranged according to the position coordinates of the pixel in the first test image, and the category probabilities pls corresponding to the m first test images are constructed into a three-dimensional matrix to obtain the first probability matrix Apls1 with a size of [w, h, m].

[0086] In this embodiment, a semantic segmentation network is selected to classify the first image to be tested. The semantic segmentation network performs well in large-scale crop classification, improves the accuracy of category probability, and provides a basis for improving mapping accuracy.

[0087] In some embodiments, step 130, determining a first pixel in a cloud region in the plurality of first images to be measured, comprises:

[0088] A first pixel in the first image to be tested is determined according to the digital quantization value of the pixel of the first image to be tested.

[0089] It is understandable that the first pixel within the ND value range may be determined by determining whether the digital quantization value (Digital Number, ND) of each pixel is within the ND value range of the cloud layer.

[0090] In actual implementation, for m first images to be tested, the ND of each pixel in each first image to be tested is calculated, and the first pixels are screened according to the ND.

[0091] The first pixel is determined as follows:

[0092]

[0093] Among them, when the DN of the pixel is any one of 2720, 2724, 2728 and 2732, the pixel is not covered by clouds; when the pixel is other values ​​(else), the pixel is determined to be the first pixel covered by clouds.

[0094] In this embodiment, by screening out the first pixel covered by clouds, the influence of cloud cover on classification accuracy can be reduced, thereby improving mapping accuracy.

[0095] In some embodiments, step 140, mapping the first pixel to the first probability matrix based on the plurality of first images to be tested to obtain a classification result of the target crop in the target area, includes:

[0096] According to the position information of the first pixel in each of the first images to be tested, the elements of the corresponding category probabilities in the first probability matrix are set to be empty to obtain a second probability matrix;

[0097] Based on the second probability matrix, calculating the classification confidence of the first image to be tested to obtain a first prediction matrix;

[0098] The classification result is obtained according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested.

[0099] The position information of the first pixel is used to represent the position coordinates of the first pixel in the first image to be tested, and the corresponding element can be found in the first probability matrix. The Normalized Difference Vegetation Index (NDVI)

[0100] In actual execution, in the first probability matrix Apls1, each element corresponds to the probability that a pixel is a category of the target crop, and the element corresponding to the first pixel in the first probability matrix is ​​emptied to obtain m second probability matrices Apls2.

[0101] The second probability matrix Apls2 may be a two-dimensional matrix, corresponding one-to-one to the first image to be tested.

[0102] In some embodiments, the calculating the classification confidence of the first image to be tested based on the second probability matrix to obtain a first prediction matrix includes:

[0103] Calculating the classification confidence of each of the first images to be tested based on the category probability of each pixel of the first images to be tested in the second probability matrix;

[0104] Determine the target image to be tested with the maximum classification confidence;

[0105] The first prediction matrix is ​​determined according to the category probabilities of the pixels of the target image to be tested in the second probability matrix.

[0106] Among them, the classification confidence can be represented by the category deviation Ap.

[0107] In actual execution, for the second probability matrix Apls2 corresponding to each first image to be tested, the category deviation Ap corresponding to each first image to be tested is calculated with 0.5 as the threshold, which represents the classification confidence of the classification result. The calculation formula is as follows:

[0108] Ap=|Apls2-0.5|

[0109] According to the class deviation Ap corresponding to the m second probability matrices Apls2, the image with the largest class deviation Ap can be used as the target image to be tested. The acquisition time corresponding to the target image to be tested is t, which is calculated as follows:

[0110] t=Arg(max(Ap)),t∈[1,...,m]

[0111] For the category probability corresponding to each pixel in the target image to be tested, the first classification result is obtained:

[0112]

[0113] Among them, P t is the category probability pls of the pixel of the target image to be tested with the acquisition time t being the target crop, t is the acquisition time of the target image to be tested. For example, the acquisition time may be the phenological period corresponding to the target image to be tested, Result is the classification result corresponding to the pixel, when Result is 0, the corresponding position of the pixel in the target area is not rice, when Result is 1, the corresponding position of the pixel in the target area is rice.

[0114] According to the classification results corresponding to each pixel in the target image to be tested, the first prediction matrix Result is constructed p .

[0115] In this embodiment, the accuracy of the classification result can be improved by predicting the target image to be tested with the maximum classification confidence.

[0116] In some embodiments, obtaining the classification result according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested includes:

[0117] Determining a second image to be measured from among the plurality of first images to be measured according to the phenological period;

[0118] constructing a second prediction matrix according to the normalized vegetation index of the pixels of the second image to be tested;

[0119] According to the first prediction matrix and the second prediction matrix, the spatial distribution area of ​​the target crop in the target area is used as the classification result.

[0120] The first prediction matrix may be constructed according to the acquisition time of the first image to be tested, and the second prediction matrix may be constructed according to the acquisition time of the second image to be tested.

[0121] In some embodiments, determining the second image to be measured from the plurality of first images to be measured according to the phenological period includes:

[0122] Determining a time sequence of the plurality of first images to be measured according to a phenological period corresponding to each of the first images to be measured;

[0123] According to the time sequence, the second image to be tested is screened out from the plurality of first images to be tested.

[0124] It should be noted that, when crops are exterminated due to floods and other factors during the phenological period, or other landforms are displayed on the image, and the final surface type is bare land, water body, etc., the recognition results of the two parts of the network are inconsistent, and the possibility of the early rice result classification category is greater than the possibility of the later non-rice category, the final annual crop classification result will be mistakenly identified as a rice defect.

[0125] Since only the maximum classification confidence of the category probability is considered, it is not effective for scenarios where crop planting changes, for example, crops are extinct due to floods, human land use and other factors, or the image is temporarily not a crop. In order to obtain accurate recognition results within the year, the present invention introduces NDVI judgment in the second half of the phenological period, that is, the m / 2 to m second test images, which can effectively deal with the scenario where crops change to non-crops during the phenological period.

[0126] It can be understood that, according to the phenological period corresponding to the first image to be tested, the m first images to be tested are arranged in the order of acquisition time, and the second half of the first images to be tested from the m / 2th to the mth are taken as the second images to be tested.

[0127] For the second image to be tested, calculate the NDVI for each pixel:

[0128]

[0129] Among them, ρ nir and ρ red are the reflectances of the near-infrared and red bands, respectively.

[0130] The NDVIs corresponding to the m / 2 pixels of the second image to be tested are used to construct a three-dimensional matrix NDVIpls, and the size of the three-dimensional matrix NDVIpls is [w, h, m / 2].

[0131] By performing a numerical judgment based on the NDVI of each element in the three-dimensional matrix NDVIpls, it is possible to obtain a classification of whether the pixel of the second image to be tested is rice.

[0132] Determine the NDVI of the pixel corresponding to each element in the three-dimensional matrix NDVIpls:

[0133]

[0134] Among them, when Remove is 0, the position corresponding to the pixel in the target area is not rice, and when Remove is 1, the position corresponding to the pixel in the target area is rice.

[0135] Set a matrix of size [w, h], with the initial value set to 1, indicating that all are rice. Update the elements in the matrix with the value of Remove. If there is any NDVI in the m / 2 pixels corresponding to the same position in the target area, and NDVI ≥ 0.3 is not satisfied, the classification result corresponding to the pixel is judged as non-rice, and the value of the corresponding element in Remove is modified to 0, and the second prediction matrix Result is obtained. NDVI .

[0136] In this embodiment, by screening the second test image with representative phenological period, the influence of the extinction of target crops on the classification result can be effectively avoided, thereby improving the crop classification accuracy.

[0137] The first prediction matrix Result p and the second prediction matrix Result NDVI The elements of are taken as the intersection, that is, the classification results of the elements whose values ​​are all 1 are rice, and the classification results of other elements are non-rice.

[0138] In this embodiment, by combining the first prediction matrix with the maximum classification confidence and the second prediction matrix classified by the normalized vegetation index, the accuracy of the classification result can be further improved, which is conducive to accurate mapping of the target area.

[0139] An embodiment is described below.

[0140] In related technologies, the results of different images of the same crop within a year are simply superimposed to obtain the final crop mapping results of the year from the image mapping results of different phenological periods within the year. How to generate the best annual mapping results from multiple crop mapping results of different phenological periods within the year is still a difficulty in large-scale crop mapping.

[0141] Since there are large differences in the accuracy of image mapping results in different phenological periods, directly superimposing image mapping results from different periods does not eliminate the errors. The accuracy of the final annual crop mapping results is still greatly affected, which is not conducive to ensuring high-precision monitoring of crop planting area.

[0142] In actual implementation, satellite remote sensing data collected from a target area during multiple phenological periods is obtained as the first image to be measured.

[0143] Input m first test images of size w×h into the semantic segmentation network in sequence, and obtain the category probability pls of rice for each pixel in each first test image output by the semantic segmentation network. Arrange the category probability pls of each pixel according to the position coordinates of the pixel in the first test image, and construct a three-dimensional matrix of the category probabilities pls corresponding to the m first test images to obtain the first probability matrix Apls1 of size [w, h, m].

[0144] For m first images to be tested, the ND of each pixel in each first image to be tested is calculated, and the first pixels are screened according to the ND.

[0145] The first pixel is determined as follows:

[0146]

[0147] Among them, when the DN of the pixel is any one of 2720, 2724, 2728 and 2732, the pixel is not covered by clouds; when the pixel is other values ​​(else), the pixel is determined to be the first pixel covered by clouds.

[0148] In the first probability matrix Apls1, each element corresponds to the probability that a pixel is a category of the target crop. The element corresponding to the first pixel in the first probability matrix is ​​emptied to obtain m second probability matrices Apls2.

[0149] On the one hand, for the second probability matrix Apls2 corresponding to each first image to be tested, the category deviation Ap corresponding to each first image to be tested is calculated with 0.5 as the threshold, which represents the classification confidence of the classification result. The calculation formula is as follows:

[0150] Ap=|Apls2-0.5|

[0151] According to the class deviation Ap corresponding to the m second probability matrices Apls2, the image with the largest class deviation Ap can be used as the target image to be tested. The acquisition time corresponding to the target image to be tested is t, which is calculated as follows:

[0152] t=Arg(max(Ap)),t∈[1,...,m]

[0153] For the category probability corresponding to each pixel in the target image to be tested, the first classification result is obtained:

[0154]

[0155] Among them, P tis the category probability pls of the pixel of the target image to be tested with the acquisition time t being the target crop, t is the acquisition time of the target image to be tested. For example, the acquisition time may be the phenological period corresponding to the target image to be tested, Result is the classification result corresponding to the pixel, when Result is 0, the corresponding position of the pixel in the target area is not rice, when Result is 1, the corresponding position of the pixel in the target area is rice.

[0156] According to the classification results corresponding to each pixel in the target image to be tested, the first prediction matrix Result is constructed p .

[0157] On the other hand, since only the maximum classification confidence of the category probability is considered, it is not effective for scenarios where crop planting changes occur, for example, crops are extinct due to floods, human land use and other factors, or the image is temporarily not a crop. In order to obtain accurate recognition results within the year, the present invention introduces NDVI judgment in the second half of the phenological period, that is, the m / 2 to m second test images, which can effectively deal with the scenario where crops change to non-crops during the phenological period.

[0158] According to the phenological period corresponding to the first image to be tested, the m first images to be tested are arranged in the order of acquisition time, and the second half of the first images to be tested from the m / 2th to the mth are taken as the second images to be tested.

[0159] For the second image to be tested, calculate the NDVI for each pixel:

[0160]

[0161] Among them, ρ nir and ρ red are the reflectances of the near-infrared and red bands, respectively.

[0162] The NDVIs corresponding to the m / 2 pixels of the second image to be tested are used to construct a three-dimensional matrix NDVIpls, and the size of the three-dimensional matrix NDVIpls is [w, h, m / 2].

[0163] By performing a numerical judgment based on the NDVI of each element in the three-dimensional matrix NDVIpls, it is possible to obtain a classification of whether the pixel of the second image to be tested is rice.

[0164] Determine the NDVI of the pixel corresponding to each element in the three-dimensional matrix NDVIpls:

[0165]

[0166] Among them, when Remove is 0, the position corresponding to the pixel in the target area is not rice, and when Remove is 1, the position corresponding to the pixel in the target area is rice.

[0167] Set a matrix of size [w, h], with the initial value set to 1, indicating that all are rice. Update the elements in the matrix with the value of Remove. If there is any NDVI in the m / 2 pixels corresponding to the same position in the target area, and NDVI ≥ 0.3 is not satisfied, the classification result corresponding to the pixel is judged as non-rice, and the value of the corresponding element in Remove is modified to 0, and the second prediction matrix Result is obtained. NDVI .

[0168] The first prediction matrix Result p and the second prediction matrix Result NDVI The elements of are taken as the intersection, that is, the classification results of the elements whose values ​​are all 1 are rice, and the classification results of other elements are non-rice.

[0169] Based on the classification results, the distribution of target crops in the target area is mapped.

[0170] In this embodiment, the category possibilities of all available images within all phenological periods of the year are considered, and the intersection of the maximum classification confidence and the normalized difference vegetation index classification is used as the final result of the year. This can more effectively solve the problem of large differences in multi-period mapping image results and low mapping accuracy due to differences in characteristics of different phenological periods in large-scale crop mapping using semantic segmentation networks, reduce the impact of differences on annual mapping results, and obtain more accurate annual crop mapping results.

[0171] Based on the category possibility results of mapping different phenological periods within the year output by the model, the mapping results corresponding to the maximum possibility are screened, and the NDVI index that represents crop growth information is combined to overcome the current problem of reduced accuracy caused by directly superimposing the mapping results of multiple phenological periods within the year, so as to obtain high-precision crop mapping results within the year and ensure high-precision monitoring of crop planting area.

[0172] The method for improving the accuracy of crop classification results of a semantic segmentation network within a year provided by the embodiment of the present invention can be performed by a device for improving the accuracy of crop classification results of a semantic segmentation network within a year. In the embodiment of the present invention, the device for improving the accuracy of crop classification results of a semantic segmentation network within a year is taken as an example to illustrate the method for improving the accuracy of crop classification results of a semantic segmentation network within a year provided by the embodiment of the present invention.

[0173] An embodiment of the present invention also provides a device for improving the accuracy of crop classification results of a semantic segmentation network.

[0174] like Figure 2 As shown, the device for improving the accuracy of crop classification results of the semantic segmentation network includes: an acquisition module 210, a first processing module 220, a second processing module 230 and a third processing module 240.

[0175] An acquisition module 210 is used to acquire a plurality of first images to be measured of a target area in a plurality of phenological periods;

[0176] A first processing module 220, configured to construct a first probability matrix of the plurality of first images to be tested according to the probability that each pixel of the first images to be tested is a category of a target crop;

[0177] A second processing module 230, configured to determine a first pixel in a cloud region in the plurality of first images to be measured;

[0178] The third processing module 240 is used to map the first pixel to the first probability matrix based on the multiple first images to be tested, so as to obtain a classification result of the target crop in the target area.

[0179] According to the device for improving the accuracy of crop classification results of the semantic segmentation network provided by an embodiment of the present invention, a first probability matrix is ​​constructed through the classification probability of pixels in the first test image collected at different phenological periods, and the characteristic differences presented by the phenological changes and the influence of the first pixels blocked by clouds are taken into account to obtain the classification results of the target crops in the target area. The obtained crop classification results can cover the characteristics of different phenological periods, are more in line with the actual crop planting area, can effectively improve the accuracy of the classification results and large-scale mapping, and realize the effective detection of the crop planting area.

[0180] In some embodiments, the third processing module 240 is further configured to:

[0181] According to the position information of the first pixel in each of the first images to be tested, the elements of the corresponding category probabilities in the first probability matrix are set to be empty to obtain a second probability matrix;

[0182] Based on the second probability matrix, calculating the classification confidence of the first image to be tested to obtain a first prediction matrix;

[0183] The classification result is obtained according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested.

[0184] In some embodiments, the third processing module 240 is further configured to:

[0185] Determining a second image to be measured from among the plurality of first images to be measured according to the phenological period;

[0186] constructing a second prediction matrix according to the normalized vegetation index of the pixels of the second image to be tested;

[0187] According to the first prediction matrix and the second prediction matrix, the spatial distribution area of ​​the target crop in the target area is used as the classification result.

[0188] In some embodiments, the third processing module 240 is further configured to:

[0189] Determining a time sequence of the plurality of first images to be measured according to a phenological period corresponding to each of the first images to be measured;

[0190] According to the time sequence, the second image to be tested is screened out from the plurality of first images to be tested.

[0191] In some embodiments, the third processing module 240 is further configured to:

[0192] Calculating the classification confidence of each of the first images to be tested based on the category probability of each pixel of the first images to be tested in the second probability matrix;

[0193] Determine the target image to be tested with the maximum classification confidence;

[0194] The first prediction matrix is ​​determined according to the category probabilities of the pixels of the target image to be tested in the second probability matrix.

[0195] In some embodiments, the second processing module 230 is further configured to:

[0196] A first pixel in the first image to be tested is determined according to the digital quantization value of the pixel of the first image to be tested.

[0197] In some embodiments, the first processing module 220 is further configured to:

[0198] Classify each of the first images to be tested by using a semantic segmentation network to determine the category probability of each pixel of the first images to be tested;

[0199] The first probability matrix is ​​constructed based on the position information of the pixels corresponding to the category probabilities.

[0200] The semantic segmentation network annual crop classification result precision improvement device in the embodiment of the present invention can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (AugmentedReality, AR) / virtual reality (Virtual Reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (PersonalDigital Assistant, PDA), etc. It can also be a server, a network attached storage (Network AttachedStorage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present invention.

[0201] The device for improving the accuracy of crop classification results of semantic segmentation network in the embodiment of the present invention may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present invention.

[0202] The device for improving the accuracy of crop classification results of semantic segmentation network provided by the embodiment of the present invention can achieve Figure 1 To avoid repetition, the various processes implemented by the embodiment of the method for improving the accuracy of crop classification results of the semantic segmentation network within the year are not repeated here.

[0203] In some embodiments, Figure 3 As shown, an embodiment of the present invention further provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, each process of the embodiment of the method for improving the accuracy of crop classification results within the year of the semantic segmentation network is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0204] It should be noted that the electronic devices in the embodiments of the present invention include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0205] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the embodiment of the method for improving the accuracy of crop classification results within the year of the semantic segmentation network is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0206] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0207] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for improving the accuracy of crop classification results of the semantic segmentation network within the year.

[0208] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0209] An embodiment of the present invention further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the embodiment of the method for improving the accuracy of crop classification results of the semantic segmentation network mentioned above, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0210] It should be understood that the chip mentioned in the embodiment of the present invention may also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0211] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0212] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the semantic segmentation network annual crop classification result accuracy improvement method of each embodiment of the present invention.

[0213] In the description of the present invention, "first feature" or "second feature" may include one or more of the features.

[0214] In the description of the present invention, "plurality" means two or more.

[0215] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0216] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0217] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for improving the accuracy of crop classification results in a semantic segmentation network, characterized in that: include: Acquire a plurality of first images to be measured of the target area in a plurality of phenological periods; constructing a first probability matrix of the plurality of first images to be tested according to the probability that a pixel of each of the first images to be tested is a category of a target crop; Determine a first pixel in a cloud region in the plurality of first images to be measured; Based on the multiple first images to be tested, mapping the first pixel to the first probability matrix to obtain a classification result of the target crop in the target area; The mapping of the first pixel to the first probability matrix based on the plurality of first images to be tested to obtain a classification result of the target crop in the target area includes: According to the position information of the first pixel in each of the first images to be tested, the elements of the corresponding category probabilities in the first probability matrix are set to be empty to obtain a second probability matrix; Based on the second probability matrix, calculating the classification confidence of the first image to be tested to obtain a first prediction matrix; Obtaining the classification result according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested; The obtaining the classification result according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested includes: According to the phenological period, determining a second image to be tested from the plurality of first images to be tested, wherein the second image to be tested is an image of the second half of the first images to be tested after being sorted in chronological order; constructing a second prediction matrix according to the normalized vegetation index of the pixels of the second image to be tested; According to the first prediction matrix and the second prediction matrix, a spatial distribution area of ​​the target crop in the target area is determined as the classification result.

2. The method for improving the accuracy of crop classification results of semantic segmentation network in the year according to claim 1 is characterized in that: The step of determining the second image to be measured from the plurality of first images to be measured according to the phenological period includes: Determining a time sequence of the plurality of first images to be measured according to a phenological period corresponding to each of the first images to be measured; According to the time sequence, the second image to be tested is screened out from the plurality of first images to be tested.

3. The method for improving the accuracy of crop classification results of semantic segmentation network in the year according to claim 1 is characterized in that: The step of calculating the classification confidence of the first image to be tested based on the second probability matrix to obtain a first prediction matrix includes: Calculating the classification confidence of each of the first images to be tested based on the category probability of each pixel of the first images to be tested in the second probability matrix; Determine the target image to be tested with the maximum classification confidence; The first prediction matrix is ​​determined according to the category probabilities of the pixels of the target image to be tested in the second probability matrix.

4. The method for improving the accuracy of crop classification results of a semantic segmentation network in a year according to any one of claims 1 to 3, characterized in that: Determining a first pixel in a cloud region in the plurality of first images to be measured includes: A first pixel in the first image to be tested is determined according to the digital quantization value of the pixel of the first image to be tested.

5. The method for improving the accuracy of crop classification results of a semantic segmentation network in a year according to any one of claims 1 to 3, characterized in that: The step of constructing a first probability matrix of the plurality of first images to be tested according to the probability that the pixel of each of the first images to be tested is a category of the target crop comprises: Classify each of the first images to be tested by using a semantic segmentation network to determine the category probability of each pixel of the first images to be tested; The first probability matrix is ​​constructed based on the position information of the pixels corresponding to the category probabilities.

6. A device for improving the accuracy of crop classification results in a semantic segmentation network, characterized in that: include: An acquisition module, used for acquiring a plurality of first images to be measured of a target area in a plurality of phenological periods; A first processing module, configured to construct a first probability matrix of the plurality of first images to be tested according to a probability that a pixel of each of the first images to be tested is a category of a target crop; A second processing module, used for determining a first pixel in a cloud region in the plurality of first images to be measured; A third processing module, configured to map the first pixel to the first probability matrix based on the plurality of first images to be tested, to obtain a classification result of the target crop in the target area; The third processing module is further used for: According to the position information of the first pixel in each of the first images to be tested, the elements of the corresponding category probabilities in the first probability matrix are set to be empty to obtain a second probability matrix; Based on the second probability matrix, calculating the classification confidence of the first image to be tested to obtain a first prediction matrix; Obtaining the classification result according to the first prediction matrix and the normalized vegetation index of the pixels of the first image to be tested; The third processing module is further used for: According to the phenological period, determining a second image to be tested from the plurality of first images to be tested, wherein the second image to be tested is an image of the second half of the first images to be tested after being sorted in chronological order; constructing a second prediction matrix according to the normalized vegetation index of the pixels of the second image to be tested; According to the first prediction matrix and the second prediction matrix, a spatial distribution area of ​​the target crop in the target area is determined as the classification result.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for improving the accuracy of crop classification results of the semantic segmentation network within the year as described in any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for improving the accuracy of crop classification results of a semantic segmentation network within a year as described in any one of claims 1 to 5 is implemented.

Images