Label generation method, device, equipment, storage medium and program product

Abstract

The present invention discloses a label generation method, device, equipment, storage medium and program product, the method comprising: obtaining first classification label values ​​respectively corresponding to each first image unit in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area in a first period; obtaining a second remote sensing image acquired from a target geographical area in a second period; converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein the difference between the approximate label value and the first classification label value is within a preset interval; using the approximate label value as the classification label value corresponding to the second image unit, wherein the second image unit is an image unit in the second remote sensing image corresponding to the same geographical area as the first image unit. The present invention realizes an automatic label generation scheme that can ensure the effect of the classification model and reduces the workload of manual annotation.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image technology, and in particular to a label generation method, device, equipment, storage medium and program product. Background Art

[0002] With the development and maturity of artificial intelligence technology and remote sensing imaging technology, image classification based on remote sensing images has been widely used in various scenarios. For example, ecological environment monitoring can be achieved by classifying objects in remote sensing images. Compared with manual visual interpretation, automated remote sensing analysis technology can improve processing efficiency and cost. However, when training classification models based on remote sensing images, remote sensing images are easy to obtain, but training annotations require manual annotations. In particular, due to differences in seasons, lighting, atmospheric conditions, sensors, etc., the tones of remote sensing images in different periods vary greatly. The model trained for remote sensing images of one period has poor classification effects on remote sensing images of other periods. Therefore, remote sensing images of each period need to participate in model training to improve the generalization ability of the model. Then, remote sensing images of different periods in the same area must also be manually annotated with training labels, which greatly increases the annotation workload. Summary of the invention

[0003] The main purpose of the present invention is to provide a label generation method, device, equipment, storage medium and program product, aiming to solve the current technical problem of large workload of label annotation in classification model training based on remote sensing images.

[0004] To achieve the above object, the present invention provides a label generation method, which is applied to a label generation device and comprises the following steps:

[0005] Obtaining first classification label values ​​corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area during a first period;

[0006] Acquire a second remote sensing image acquired over the target geographic area during a second period;

[0007] Converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein a difference between the approximate label value and the first classification label value is within a preset interval;

[0008] The approximate label value is used as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image corresponding to the same geographical area as the first image unit.

[0009] Optionally, the first classification label value includes category label values ​​corresponding to various classification categories, and the step of converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value includes:

[0010] When the category label value is 1, the result obtained by subtracting the preset difference from 1 is used as the approximate label value corresponding to the category label value, wherein the preset difference is the upper limit value of the preset interval;

[0011] When the category label value is 0, the result obtained by dividing the preset difference by the remaining number of categories is used as the approximate label value corresponding to the category label value, wherein the remaining number of categories is the total number of classification categories minus 1.

[0012] Optionally, before the step of subtracting a preset difference from 1 as the approximate label value corresponding to the category label value when the category label value is 1, the step further includes:

[0013] Calculating a feature difference between the first remote sensing image and the second remote sensing image in a preset image feature;

[0014] According to a preset mapping relationship between the difference degree and the difference value, the difference value corresponding to the feature difference degree is used as the preset difference value.

[0015] Optionally, the step of obtaining first classification label values ​​respectively corresponding to each first image unit in the first remote sensing image includes:

[0016] Acquire manually annotated data of a first portion of an image area in the first remote sensing image;

[0017] Extracting a classification label value of each image unit in the first partial image area from the manually annotated data;

[0018] A first image classification model is obtained by training using the first partial image area and the classification label value of each image unit in the first partial image area;

[0019] The first image classification model is used to classify a second partial image area in the first remote sensing image except the first partial image area, so as to obtain a classification label value of each image unit in the second partial image area.

[0020] Optionally, after the step of using the approximate label value as the classification label value corresponding to the second image unit, the method further includes:

[0021] A second image classification model is obtained by training using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image.

[0022] Optionally, the step of training the second image classification model using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image includes:

[0023] Performing a round of training on a preset image classification model to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, and detecting whether the classification accuracy of the image classification model to be trained after the training is increased relative to the classification accuracy before the training;

[0024] If it increases, updating the classification label value of the second image unit according to a preset reduction scale to reduce the difference between the classification label value of the second image unit and the corresponding first classification label value;

[0025] Based on the trained classification model of the image to be trained and the updated classification label value of the second image unit, return to execute the step of performing a round of training on the preset classification model of the image to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, until it is detected that the preset iteration stop condition is met, and the trained classification model of the image to be trained is used as the second image classification model.

[0026] Optionally, after the step of training the second image classification model using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, the step further includes:

[0027] When a remote sensing image to be classified is obtained, a shooting period of the remote sensing image to be classified is obtained;

[0028] When the shooting period is the first period, the first image classification model is used to classify the remote sensing image to be classified to obtain a classification result of each image unit in the remote sensing image to be classified;

[0029] When the shooting period is the second period, the second image classification model is used to classify the remote sensing image to be classified to obtain a classification result of each image unit in the remote sensing image to be classified.

[0030] To achieve the above object, the present invention further provides a label generation device, the label generation device comprising:

[0031] A first acquisition module is used to acquire first classification label values ​​respectively corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area in a first period;

[0032] A second acquisition module is used to acquire a second remote sensing image acquired from the target geographical area in a second period;

[0033] a conversion module, configured to convert the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein a difference between the approximate label value and the first classification label value is within a preset interval;

[0034] A determination module is configured to use the approximate label value as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image that corresponds to the same geographical area as the first image unit.

[0035] To achieve the above-mentioned purpose, the present invention also provides a label generation device, which includes: a memory, a processor, and a label generation program stored in the memory and executable on the processor, and the label generation program implements the steps of the label generation method described above when executed by the processor.

[0036] In addition, to achieve the above-mentioned purpose, the present invention further proposes a computer-readable storage medium, on which a label generation program is stored, and when the label generation program is executed by a processor, the steps of the label generation method described above are implemented.

[0037] In addition, to achieve the above object, the present invention also proposes a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the label generation method described above are implemented.

[0038] In the present invention, the first classification labels corresponding to the first image units in the first remote sensing image are obtained by collecting the first remote sensing image of the target geographical area in the first period; the second remote sensing image collected in the second period of the target geographical area is obtained; the first classification label value of the first image unit is converted into an approximate label value close to the first classification label value, and the approximate label value of the first image unit is used as the classification label value of the second image unit in the second remote sensing image corresponding to the same geographical area as the first image unit. The present invention realizes a label generation scheme that can ensure the effect of the classification model. Based on the label generation scheme, the classification labels of remote sensing images in one period can be used to automatically generate classification labels for remote sensing images in other periods, reducing the workload of manual labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the structure of the hardware operating environment involved in the embodiment of the present invention;

[0040] Figure 2 This is a flow chart of a first embodiment of a label generation method of the present invention;

[0041] Figure 3 Schematic diagram of functional modules of a preferred embodiment of the label generating device of the present invention.

[0042] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0044] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0045] It should be noted that the label generating device in the embodiment of the present invention can be a smart phone, a personal computer, a server and other devices, and no specific limitation is made here.

[0046] like Figure 1 As shown, the label generating device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0047] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the label generating device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0048] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a label generation program. The operating system is a program that manages and controls the hardware and software resources of the device and supports the operation of the label generation program and other software or programs. Figure 1In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the label generation program stored in the memory 1005 and perform the following operations:

[0049] Obtaining first classification label values ​​corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area during a first period;

[0050] Acquire a second remote sensing image acquired over the target geographic area during a second period;

[0051] Converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein a difference between the approximate label value and the first classification label value is within a preset interval;

[0052] The approximate label value is used as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image corresponding to the same geographical area as the first image unit.

[0053] Furthermore, the first classification label value includes category label values ​​corresponding to various classification categories, and converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value includes:

[0054] When the category label value is 1, the result obtained by subtracting the preset difference from 1 is used as the approximate label value corresponding to the category label value, wherein the preset difference is the upper limit value of the preset interval;

[0055] When the category label value is 0, the result obtained by dividing the preset difference by the remaining number of categories is used as the approximate label value corresponding to the category label value, wherein the remaining number of categories is the total number of classification categories minus 1.

[0056] Further, when the category label value is 1, before taking the result obtained by subtracting the preset difference from 1 as the approximate label value corresponding to the category label value, the processor 1001 may also be used to call the label generation program stored in the memory 1005 to perform the following operations:

[0057] Calculating a feature difference between the first remote sensing image and the second remote sensing image in a preset image feature;

[0058] According to a preset mapping relationship between the difference degree and the difference value, the difference value corresponding to the feature difference degree is used as the preset difference value.

[0059] Furthermore, obtaining the first classification label value corresponding to each first image unit in the first remote sensing image includes:

[0060] Acquire manually annotated data of a first portion of an image area in the first remote sensing image;

[0061] Extracting a classification label value of each image unit in the first partial image area from the manually annotated data;

[0062] A first image classification model is obtained by training using the first partial image area and the classification label value of each image unit in the first partial image area;

[0063] The first image classification model is used to classify a second partial image area in the first remote sensing image except the first partial image area, so as to obtain a classification label value of each image unit in the second partial image area.

[0064] Further, after taking the approximate label value as the classification label value corresponding to the second image unit, the processor 1001 may also be used to call the label generation program stored in the memory 1005 to perform the following operations:

[0065] A second image classification model is obtained by training using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image.

[0066] Furthermore, the training of the second image classification model using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image comprises:

[0067] Performing a round of training on a preset image classification model to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, and detecting whether the classification accuracy of the image classification model to be trained after the training is increased relative to the classification accuracy before the training;

[0068] If it increases, updating the classification label value of the second image unit according to a preset reduction scale to reduce the difference between the classification label value of the second image unit and the corresponding first classification label value;

[0069] Based on the trained classification model of the image to be trained and the updated classification label value of the second image unit, return to execute the step of performing a round of training on the preset classification model of the image to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, until it is detected that the preset iteration stop condition is met, and the trained classification model of the image to be trained is used as the second image classification model.

[0070] Further, after the second image classification model is obtained by training the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, the processor 1001 can also be used to call the label generation program stored in the memory 1005 to perform the following operations:

[0071] When a remote sensing image to be classified is obtained, a shooting period of the remote sensing image to be classified is obtained;

[0072] When the shooting period is the first period, the first image classification model is used to classify the remote sensing image to be classified to obtain a classification result of each image unit in the remote sensing image to be classified;

[0073] When the shooting period is the second period, the second image classification model is used to classify the remote sensing image to be classified to obtain a classification result of each image unit in the remote sensing image to be classified.

[0074] Based on the above structure, various embodiments of the label generation method are proposed.

[0075] Reference Figure 2 , Figure 2 FIG. 1 is a flow chart of a first embodiment of a label generation method according to the present invention.

[0076] The embodiment of the present invention provides an embodiment of the label generation method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here. In this embodiment, the label generation method is applied to a label generation device, which may be a smart phone, a personal computer, a server or other device. For ease of description, the execution subject is omitted below to explain each embodiment. In this embodiment, the label generation method includes:

[0077] Step S10, obtaining first classification label values ​​respectively corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area in a first period;

[0078] In this embodiment, in order to solve the problem that remote sensing images of each period need to be manually annotated with training labels, which results in a large amount of labeling workload, a solution is proposed by borrowing remote sensing image labels of one period to generate remote sensing image labels of another period, thereby greatly reducing the workload of manual labeling.

[0079] Specifically, the remote sensing images collected for the target geographical area in the first period are taken as the first remote sensing images. The first period is set as required, and can be a time point or a time period. The time span of the time period should not be too large to ensure that the remote sensing images collected by the remote sensing satellite in the first period do not differ too much in image features. The image features may refer to the image's color tone, detail texture features, etc.

[0080] Obtain the classification label values ​​corresponding to each image unit in the first remote sensing image. To distinguish, the image unit in the first remote sensing image is referred to as the first image unit, and the classification label value corresponding to the first image unit is referred to as the first classification label value. It can be understood that the first remote sensing image can be divided into multiple first image units, and each image unit corresponds to a first classification label value. It should be noted that a first classification label value does not mean that there is only one value, but can include one or more category label values ​​according to the number of categories to be classified; for example, when there is only one (or one) classification category, a first classification label value includes a category label value, which is used to indicate whether the image unit belongs to the classification category; when there are multiple classification categories, a first classification label value includes multiple category label values, each category label value corresponds to a classification category, which is used to indicate whether the image unit belongs to the classification category. In some embodiments, the category label value is set to 1 or 0 to indicate "yes" or "no", but in this embodiment, there is no specific restriction on how to set the category label value. The classification category is also not limited in this embodiment, for example, it can be classified into vegetation, buildings and other categories.

[0081] The image unit can be obtained by dividing the image according to certain rules according to specific needs. For example, in one embodiment, the pixels of the image can be used as image units, that is, one pixel is one image unit. This embodiment does not limit the division rules of the image units.

[0082] The first classification label value may be a manually labeled label value, or a label value obtained by labeling using an existing automatic labeling method, which is not limited in this embodiment.

[0083] Step S20, obtaining a second remote sensing image acquired for the target geographical area during a second period;

[0084] Acquire a remote sensing image acquired from the target geographical area in the second period, hereinafter referred to as the second remote sensing image. The second period can also be set as needed, and can be a time point or a time period. The time span of the time period should not be too large to ensure that the remote sensing images acquired by the remote sensing satellite in the second period do not differ too much in image features. The acquisition device for acquiring the second remote sensing image can be the same as or different from the acquisition device for acquiring the first remote sensing image; if the acquisition devices are the same, the first period and the second period can be set differently; if the acquisition devices are different, the first period and the second period can be set the same. That is, the features of remote sensing images acquired in different periods are different, so the label of one period can be borrowed to generate the label of another period, and the remote sensing images acquired by different acquisition devices are different, so the label of the remote sensing image acquired by one acquisition device can be borrowed to generate the label of the remote sensing image acquired by another acquisition device. The second remote sensing image and the first remote sensing image are both remote sensing images acquired from the target geographical area, and the target geographical area can be selected according to specific needs, and there is no restriction here.

[0085] Step S30, converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein the difference between the approximate label value and the first classification label value is within a preset interval;

[0086] For each first image unit in the first remote sensing image, the first classification label value corresponding to the first image unit is converted into an approximate label value that is close in size to the first classification label value. Among them, the purpose of close size can be achieved by constraining the difference between the approximate label value and the first classification label value. Specifically, a preset interval can be set, and the difference between the converted approximate label value and the first classification label value can be within the preset interval. The preset interval can be set as needed, for example, set to (0, 0.1]; that is, the degree of closeness between the approximate label value and the first classification label value is not limited in this embodiment. In a specific implementation, there are many ways to convert the first classification label value into an approximate label value that is close in size, which is not limited in this embodiment. For example, a value can be added or subtracted from the first classification label value, and the value is within the preset interval, so as to ensure that the difference between the converted approximate label value and the first classification label value is within the preset interval.

[0087] It should be noted that the preset interval does not include 0, because if the difference between the approximate label value and the first classification label value is 0, the purpose of converting to an approximate value of similar size cannot be achieved. The preset interval can be an interval that only includes positive numbers, so the absolute value of the difference between the approximate label value and the first classification label value can be in the preset interval. When the first classification label value includes multiple category label values, each category label value is converted into an approximate label value separately, and the difference between each category label value and the approximate label value obtained by its conversion can be in the preset interval.

[0088] Step S40: Using the approximate label value as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image that corresponds to the same geographical area as the first image unit.

[0089] After converting the classification label value of the first image unit to obtain an approximate label value, the approximate label value is used as the classification label value corresponding to the second image unit, and the second image unit is an image unit in the second remote sensing image corresponding to the same geographical area as the first image unit. Since the first remote sensing image and the second remote sensing image are both images collected from the target geographical area, the image units in the first remote sensing image correspond one to one with the image units in the second remote sensing image. For each second image unit in the second remote sensing image, the approximate label value of the corresponding first image unit can be used as the classification label value of the second image unit, so the classification label value of each second image unit in the second remote sensing image can be generated.

[0090] In this embodiment, the principle of using the approximate label value obtained by converting the classification label value of the first image unit as the classification label value of the second image unit is: we guess that if the classification label value of the first image unit is directly used as the classification label value of the second image unit to train the model, the model will have a poor classification effect on the remote sensing images collected in the second period, and experiments have found that this is indeed the case; through analysis, it is found that the reason is that there is a difference between the real classification label value of the second image unit and the classification label value of the first image unit. If the classification label value of the first image unit is directly used for training, the model will believe that the classification label value of the first image unit is the classification label value of the second image unit during the training process, thereby causing the training direction to deviate; through analysis, it is found that the first remote sensing image and the second remote sensing image are images collected from the same geographical area, and the difference in classification label values ​​will not be too large. If using When the classification label value of the first image unit is used for training, the model can be less sure that the classification label value of the first image unit is the classification label value of the second image unit during the training process, which can offset the small difference in the classification label values ​​between the first remote sensing image and the second remote sensing image, correct the training direction, and improve the classification effect; and by converting the classification label value of the first image unit into an approximate label value of similar size as the classification label value of the second image unit, although the correct classification label value of the second image unit is not necessarily obtained, it can just simulate the state that the model is not so sure about the classification label value, thereby ensuring the classification effect of the trained classification model on the remote sensing images collected in the second period, that is, the classification label value of the second remote sensing image generated in this way is not necessarily true, but it can simulate the real label training effect, that is, it has a certain degree of usability.

[0091] This embodiment obtains the first classification labels corresponding to each first image unit in the first remote sensing image acquired from the target geographical area in the first period; obtains the second remote sensing image acquired from the target geographical area in the second period; converts the first classification label value of the first image unit into an approximate label value close in size to the first classification label value, and uses the approximate label value of the first image unit as the classification label value of the second image unit in the second remote sensing image corresponding to the same geographical area as the first image unit. This embodiment implements a label generation scheme that can ensure the effect of the classification model. Based on the label generation scheme, the classification labels of remote sensing images in one period can be used to automatically generate classification labels for remote sensing images in other periods, reducing the workload of manual labeling.

[0092] Furthermore, in one embodiment, after obtaining the classification label value of each second image unit in the second remote sensing image, the second remote sensing image and the classification label value can be used to train a classification model. Specifically, only the classification label value corresponding to the second remote sensing image can be used to train a classification model for the remote sensing image of the second period, or the second remote sensing image and the first remote sensing image and the corresponding classification label value can be used to train a classification model for the remote sensing images of the first period and the second period, or the second remote sensing image and remote sensing images of other periods and the corresponding classification label value can be used to train a classification model for the second period and remote sensing images of other periods. The classification model can adopt a commonly used image semantic segmentation model, or other image classification models, which are not limited here.

[0093] Further, based on the above first embodiment, a second embodiment of the label generation method of the present invention is proposed. In this embodiment, step S30 includes:

[0094] Step S301, when the category label value is 1, a result obtained by subtracting a preset difference from 1 is used as an approximate label value corresponding to the category label value, wherein the preset difference is an upper limit value of the preset interval;

[0095] In this embodiment, a method for converting a first classification label value into an approximate label value is proposed. Specifically, a first classification label value corresponding to a first image unit includes category label values ​​corresponding to various classification categories. Among multiple category label values ​​in a first classification label value, one is 1 and the rest are 0, indicating that the category of the first image unit is the classification category corresponding to 1. Alternatively, among multiple label values ​​in a first classification label value, one is 0 and the rest are 1, indicating that the category of the first image unit is the classification category corresponding to 0.

[0096] For each category label value in a first classification label value, when the category label value is 1, the result obtained by subtracting the preset difference from 1 is used as the approximate label value corresponding to the category label value. The preset difference is the upper limit of the preset interval. The preset difference is represented by ε, so the approximate label value converted from the category label value of 1 is 1-ε. For example, if the preset difference is set to 0.1, the approximate label value converted from the category label value of 1 is 0.9.

[0097] Step S302: When the category label value is 0, a result obtained by dividing the preset difference by the number of remaining categories is used as an approximate label value corresponding to the category label value, wherein the number of remaining categories is the total number of classification categories minus 1.

[0098] Subtract 1 from the total number of classification categories, and the result is called the remaining number of categories for distinction. When the category label value is 0, the result obtained by dividing the preset difference by the remaining number of categories is used as the approximate label value corresponding to the category label value. The total number of classification categories is represented by k, so the approximate label value converted from the category label value of 0 is 1 / (1-k). It should be noted that if the division is not exact, an approximate value can be taken. For example, when the total number of classification categories is 4, that is, there are 4 categories that need to be classified, the first classification label value of a first image unit is (0, 0, 1, 0), indicating that the first image unit belongs to the third category, and the approximate label value obtained after conversion is (0.03, 0.03, 0.9, 0.03).

[0099] Furthermore, in one embodiment, the method further comprises:

[0100] Step S50, calculating the feature difference between the first remote sensing image and the second remote sensing image in the preset image features;

[0101] Step S60: according to a preset mapping relationship between the difference degree and the difference value, the difference value corresponding to the feature difference degree is used as the preset difference value.

[0102] The preset difference can be determined based on the difference between the first remote sensing image and the second remote sensing image. Specifically, the feature difference between the first remote sensing image and the second remote sensing image on the preset image features can be calculated. Among them, the preset image features may include features such as hue, texture features and / or color, which are not limited in the present embodiment. The feature values ​​of the two images can be extracted by a corresponding feature extraction method, and then the difference between the two feature values ​​is calculated. When multiple image features are included, the difference between the two remote sensing images on each image feature can be calculated separately, and then the differences are averaged or weighted averaged to obtain a feature difference. Since the geographical area covered by the first remote sensing image and the second remote sensing image may be large, the difference calculation can be performed only on part of the area of ​​the first remote sensing image and the second remote sensing image, and the part of the area can be determined by sampling.

[0103] After calculating the characteristic difference between the first remote sensing image and the second remote sensing image, the difference corresponding to the characteristic difference can be used as the preset difference according to the mapping relationship between the preset difference and the difference. The mapping relationship between the difference and the difference can be set in advance, in which the difference can increase as the difference increases, that is, when the characteristic difference between the first remote sensing image and the second remote sensing image is larger, the preset difference is larger, so that the uncertainty of the model regarding the classification label value of the first remote sensing image as the classification label value of the second remote sensing image is consistent with the actual difference between the two images, thereby improving the classification accuracy of the classification model trained based on the classification label value of the generated second remote sensing image, that is, improving the availability of the classification label value of the generated second remote sensing image.

[0104] Further, based on the above-mentioned first and / or second embodiments, a third embodiment of the label generation method of the present invention is proposed. In this embodiment, step S10 includes:

[0105] Step S101, obtaining manually labeled data of a first portion of an image area in the first remote sensing image;

[0106] In this embodiment, when the scope of the target geographical area is large, a partial image area in the first remote sensing image and the manually annotated data of this partial image area can be used to train a classification model, and then the trained classification model is used to classify the remaining image area to obtain the classification label value of the remaining image area, thereby further reducing the workload of manual annotation.

[0107] Specifically, the manual annotation data of a partial image area in the first remote sensing image (hereinafter referred to as the first partial image area) can be obtained. The manual annotation data can be annotation data indicating the category to which each area in the first partial image area belongs. For example, a manual annotation tool can be used to circle a part in the image and mark the category 1 for the part, and circle another part and mark the category 2 for the part.

[0108] Step S102, extracting the classification label value of each image unit in the first partial image area from the manually annotated data;

[0109] After obtaining the manually annotated data, the classification label value of each image unit in the first part of the image area is extracted from the manually annotated data. Specifically, the classification label value of the image unit in the area can be set to the classification label value corresponding to the category of the area according to the category to which each area in the manually annotated data belongs. For example, there are 4 categories in total. According to the manually annotated data, if a certain area is determined to belong to the first category, the classification label value of each image unit in the area is set to (1, 0, 0, 0). If another area belongs to the second category, the classification label value of each image unit in the area is set to (0, 2, 0, 0), and so on.

[0110] Step S103, training to obtain a first image classification model using the first partial image area and the classification label values ​​of each image unit in the first partial image area;

[0111] An image classification model (hereinafter referred to as the first image classification model) is obtained by training the classification label values ​​of the first image region and each image unit in the first image region. The first image classification model can adopt a commonly used image semantic segmentation model, and the model parameters in the model are determined through training. The specific training method can refer to the existing model training method, which will not be described in detail here.

[0112] Step S104: using the first image classification model to classify a second partial image area in the first remote sensing image except the first partial image area, to obtain a classification label value of each image unit in the second partial image area.

[0113] After the first image classification model is trained, the second image area except the first image area in the first remote sensing image is classified using the first image classification model to obtain the classification label value of each image unit in the second image area. At this point, the classification label value of each first image unit in the first remote sensing image is obtained. It can be understood that the classification label values ​​of some first image units are extracted from manually annotated data, and the classification label values ​​of another part of the first image units are obtained by classification using the first image classification model. Therefore, when the classification label value of the first remote sensing image is used to generate the classification label value of the second remote sensing image, only part of the image area in the first remote sensing image needs to be manually annotated, which further reduces the workload of manual annotation.

[0114] Furthermore, in one embodiment, the method further comprises:

[0115] Step S70: training a second image classification model using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image.

[0116] After obtaining the classification label value of each second image unit in the second remote sensing image, the second remote sensing image and the classification label value of each second image unit in the second remote sensing image can be used to train an image classification model (hereinafter referred to as the second image classification model). The second image classification model can adopt a commonly used image semantic segmentation model. The model parameters in the model are determined through training. The specific training method can refer to the existing model training method, which will not be described in detail here.

[0117] Further, in one embodiment, step S70 includes:

[0118] Step S701, performing a round of training on a preset image classification model to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, and detecting whether the classification accuracy of the image classification model to be trained after the training is increased relative to the classification accuracy before the training;

[0119] In order to further optimize the training effect of the second image classification model trained with the approximate label values, the approximate label values ​​may be adjusted during the training process to optimize the degree of fit of the approximate label values ​​to the state of the simulated true label values.

[0120] Specifically, the second remote sensing image and the classification label values ​​of each second image unit in the second remote sensing image can be used to perform a round of training on the preset image classification model to be trained. Among them, during one round of training, the model parameters in the image classification model to be trained can be updated once or multiple times. The specific training process can refer to the existing model training method, which is not described in detail in this embodiment. The image classification model to be trained is an image classification model that needs to determine the model parameters through training. Specifically, an image semantic segmentation model or other image classification models can be used.

[0121] After a round of training, it is detected whether the classification accuracy of the trained image classification model is increased relative to the classification accuracy of the trained image classification model before the current round of training. Specifically, the classification accuracy after training and the classification accuracy before training can be calculated, and the classification accuracy after training minus the classification accuracy before training can be used to determine whether it has increased. Among them, the classification accuracy can adopt a commonly used model classification accuracy calculation method, which will not be described in detail here.

[0122] Step S702: if increased, updating the classification label value of the second image unit according to a preset reduction scale to reduce the difference between the classification label value of the second image unit and the corresponding first classification label value;

[0123] If the classification accuracy after training increases relative to the classification accuracy before training, it means that the classification label value of the first remote sensing image is actually less different from the classification label value of the second remote sensing image. By converting the classification label value of the first remote sensing image into an approximate label value as the classification label value of the second remote sensing image, the training effect of the model can be positively improved. Therefore, the difference between the approximate label value (that is, the classification label value of the current second image unit) and the corresponding first classification label value can be reduced to further increase the positive effect of the approximate label value on the training effect, thereby improving the classification accuracy of the model. Specifically, the classification label value of the second image unit can be updated according to the preset reduction scale to reduce the difference between the classification label value of the second image unit and the corresponding first classification label value. Among them, the preset reduction scale can be set as needed, for example, set to reduce by 10%. For example, the original classification label value of the second image unit is 0.9, the corresponding first classification label value is 1, and the difference between the two is 0.1. Reducing by 10% means reducing the difference between the two to 0.09, then the updated classification label value of the second image unit is 0.91.

[0124] In one embodiment, if the classification accuracy after training decreases relative to the classification accuracy before training, the classification label value of the second image unit can be updated according to a certain increase scale to increase the difference between the classification label value of the second image unit and the corresponding first classification label value.

[0125] Step S703, based on the trained classification model of the image to be trained and the updated classification label value of the second image unit, returns to the step of performing a round of training on the preset classification model of the image to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, until it is detected that the preset iteration stop condition is met, and the trained classification model of the image to be trained is used as the second image classification model.

[0126] After the classification label value of the second image unit is updated, the step S701 can be returned to execution based on the image classification model to be trained after the current round of training and the classification label value of the updated second image unit, that is, the next round of training can be performed again until it is detected that the preset iteration stop condition is met, and the trained image classification model to be trained is used as the second image classification model. The preset iteration stop condition can be set as needed, for example, it can be set to stop when the number of training rounds reaches a certain number of rounds, or when the training time reaches a certain time, or when the classification accuracy of the trained image classification model to be trained is greater than a certain value, or when the loss function of the image classification model to be trained converges.

[0127] It should be noted that after the first round of training, the classification label value of the second image unit is updated based on the initially generated classification label value, and after each subsequent round of training, the classification label value of the second image unit is updated based on the previously updated classification label value.

[0128] Furthermore, in one embodiment, after step S70, the following steps are further included:

[0129] Step A10, when a remote sensing image to be classified is obtained, obtaining a shooting period of the remote sensing image to be classified;

[0130] After the first image classification model and the second image classification model are trained, the two models can be used to classify the remote sensing images collected in the first period and the second period. Specifically, when a remote sensing image to be classified (hereinafter referred to as a remote sensing image to be classified) is obtained, the shooting period of the remote sensing image to be classified can be obtained, and the image classification model of the corresponding period is selected according to the shooting period to classify the remote sensing image to be classified.

[0131] Step A20, when the shooting period is the first period, classifying the remote sensing image to be classified using the first image classification model to obtain a classification result of each image unit in the remote sensing image to be classified;

[0132] When the shooting period is the first period, the first image classification model is used to classify the remote sensing image to be classified, and the classification results of each image unit in the remote sensing image to be classified are obtained. Specifically, the classified remote sensing image can be input into the first image classification model, and the classification result is obtained after being processed and output by the first image classification model. According to the classification result, the category to which each image unit in the remote sensing image to be classified belongs can be known. In a specific application scenario, the classification result can be output or the classification result can be processed and then output for the user to apply the classification result. For example, when the classification categories include vegetation and non-vegetation, the area of ​​the vegetation area in the remote sensing image to be classified can be calculated according to the classification result, and then the area of ​​the vegetation area in the geographical area corresponding to the remote sensing image to be classified can be calculated, and the area is output, so that the user can analyze the change of the vegetation area in the geographical area.

[0133] Step A30: when the shooting period is the second period, the second image classification model is used to classify the remote sensing image to be classified, so as to obtain classification results of each image unit in the remote sensing image to be classified.

[0134] When the shooting period is the second period, the remote sensing image to be classified is classified using the second image classification model to obtain the classification results of each image unit in the remote sensing image to be classified. Specifically, the remote sensing image to be classified can be input into the second image classification model, and the classification result is obtained after being processed and output by the second image classification model.

[0135] In this embodiment, the first remote sensing image is used to train the first image classification model, and the second remote sensing image is used to train the second image classification model. The first image classification model and the second image classification model are trained separately, thereby improving the classification effects of the two models respectively. Therefore, when the first image classification model is used to classify the remote sensing images taken in the first period, and the second image classification model is used to classify the remote sensing images taken in the second period, the generalization ability of the overall model can be improved.

[0136] In addition, the present invention also provides a label generation device, referring to Figure 3 , the device comprises:

[0137] A first acquisition module 10 is used to acquire first classification label values ​​corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area in a first period;

[0138] A second acquisition module 20 is used to acquire a second remote sensing image acquired from the target geographical area in a second period;

[0139] A conversion module 30, configured to convert the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein a difference between the approximate label value and the first classification label value is within a preset interval;

[0140] The determination module 40 is configured to use the approximate label value as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image that corresponds to the same geographical area as the first image unit.

[0141] Furthermore, the first classification label value includes category label values ​​corresponding to various classification categories, and the conversion module 30 includes:

[0142] A first conversion unit, configured to, when the category label value is 1, use a result obtained by subtracting a preset difference from 1 as an approximate label value corresponding to the category label value, wherein the preset difference is an upper limit value of the preset interval;

[0143] The second conversion unit is used to use the result obtained by dividing the preset difference by the remaining number of categories as the approximate label value corresponding to the category label value when the category label value is 0, wherein the remaining number of categories is the total number of classification categories minus 1.

[0144] Furthermore, the device also includes:

[0145] A calculation module, used for calculating the feature difference between the first remote sensing image and the second remote sensing image in terms of preset image features;

[0146] The determining module 40 is further configured to use the difference value corresponding to the feature difference degree as the preset difference value according to a mapping relationship between the preset difference degree and the difference value.

[0147] Furthermore, the first acquisition module 10 includes:

[0148] An acquisition unit, used for acquiring manually annotated data of a first portion of an image area in the first remote sensing image;

[0149] An extraction unit, used for extracting a classification label value of each image unit in the first partial image area from the manually annotated data;

[0150] A first training unit, configured to obtain a first image classification model by training the first partial image area and the classification label values ​​of each image unit in the first partial image area;

[0151] A classification unit is used to classify a second partial image area in the first remote sensing image except the first partial image area by using the first image classification model to obtain a classification label value of each image unit in the second partial image area.

[0152] Furthermore, the device also includes:

[0153] A training module is used to train a second image classification model using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image.

[0154] Furthermore, the training module includes:

[0155] a second training unit, configured to perform a round of training on a preset image classification model to be trained using the second remote sensing image and the classification label values ​​of each second image unit in the second remote sensing image, and detect whether the classification accuracy of the image classification model to be trained after the training is increased relative to the classification accuracy before the training;

[0156] an updating unit, configured to update the classification label value of the second image unit according to a preset reduction scale if increased, so as to reduce the difference between the classification label value of the second image unit and the corresponding first classification label value;

[0157] A loop unit is used to return to the step of performing a round of training on the preset image classification model to be trained using the second remote sensing image and the classification label values ​​of each of the second image units in the second remote sensing image based on the trained image classification model to be trained and the updated classification label values ​​of the second image units, until it is detected that a preset iteration stop condition is met, and the trained image classification model to be trained is used as the second image classification model.

[0158] Furthermore, the device also includes:

[0159] A third acquisition module is used to acquire the shooting period of the remote sensing image to be classified when the remote sensing image to be classified is acquired;

[0160] a first classification module, configured to classify the remote sensing image to be classified by using the first image classification model when the shooting period is the first period, so as to obtain a classification result of each image unit in the remote sensing image to be classified;

[0161] The second classification module is used to classify the remote sensing image to be classified by using the second image classification model when the shooting period is the second period, so as to obtain the classification result of each image unit in the remote sensing image to be classified.

[0162] The expanded contents of the specific implementation manner of the label generation device of the present invention are basically the same as the above-mentioned embodiments of the label generation method, and will not be elaborated here.

[0163] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a label generation program is stored. When the label generation program is executed by a processor, the steps of the label generation method described below are implemented.

[0164] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the label generation method described above are implemented.

[0165] The various embodiments of the label generating device, computer-readable storage medium, and computer program product of the present invention may all refer to the various embodiments of the label generating method of the present invention, and will not be described in detail here.

[0166] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0167] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0169] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A label generation method, characterized in that: The method is applied to a label generating device, and the method comprises the following steps: Obtaining first classification label values ​​corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area during a first period; Acquire a second remote sensing image acquired over the target geographic area during a second period; Converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein a difference between the approximate label value and the first classification label value is within a preset interval, and the preset interval does not include 0; The approximate label value is used as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image corresponding to the same geographical area as the first image unit.

2. The label generation method according to claim 1, characterized in that: The first classification label value includes category label values ​​corresponding to various classification categories, and the step of converting the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value includes: When the category label value is 1, the result obtained by subtracting the preset difference from 1 is used as the approximate label value corresponding to the category label value, wherein the preset difference is the upper limit value of the preset interval; When the category label value is 0, the result obtained by dividing the preset difference by the remaining number of categories is used as the approximate label value corresponding to the category label value, wherein the remaining number of categories is the total number of classification categories minus 1.

3. The label generation method according to claim 2, characterized in that: Before the step of subtracting a preset difference from 1 as the approximate label value corresponding to the category label value when the category label value is 1, the step further includes: Calculating a feature difference between the first remote sensing image and the second remote sensing image in a preset image feature; According to a preset mapping relationship between the difference degree and the difference value, the difference value corresponding to the feature difference degree is used as the preset difference value.

4. The label generation method according to any one of claims 1 to 3, characterized in that: The step of obtaining the first classification label value corresponding to each first image unit in the first remote sensing image comprises: Acquire manually annotated data of a first portion of an image area in the first remote sensing image; Extracting a classification label value of each image unit in the first partial image area from the manually annotated data; A first image classification model is obtained by training using the first partial image area and the classification label value of each image unit in the first partial image area; The first image classification model is used to classify a second partial image area in the first remote sensing image except the first partial image area, so as to obtain a classification label value of each image unit in the second partial image area.

5. The label generation method according to claim 4, characterized in that: After the step of using the approximate label value as the classification label value corresponding to the second image unit, the method further includes: A second image classification model is obtained by training using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image.

6. The label generation method according to claim 5, characterized in that: The step of training the second image classification model using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image comprises: Performing a round of training on a preset image classification model to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, and detecting whether the classification accuracy of the image classification model to be trained after the training is increased relative to the classification accuracy before the training; If it increases, updating the classification label value of the second image unit according to a preset reduction scale to reduce the difference between the classification label value of the second image unit and the corresponding first classification label value; Based on the trained classification model of the image to be trained and the updated classification label value of the second image unit, return to execute the step of performing a round of training on the preset classification model of the image to be trained using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image, until it is detected that the preset iteration stop condition is met, and the trained classification model of the image to be trained is used as the second image classification model.

7. The label generation method according to claim 5, characterized in that: After the step of using the second remote sensing image and the classification label value of each second image unit in the second remote sensing image to train and obtain a second image classification model, the method further includes: When a remote sensing image to be classified is obtained, a shooting period of the remote sensing image to be classified is obtained; When the shooting period is the first period, the first image classification model is used to classify the remote sensing image to be classified to obtain a classification result of each image unit in the remote sensing image to be classified; When the shooting period is the second period, the second image classification model is used to classify the remote sensing image to be classified to obtain a classification result of each image unit in the remote sensing image to be classified.

8. A label generating device, characterized in that: The label generating device comprises: A first acquisition module is used to acquire first classification label values ​​respectively corresponding to first image units in a first remote sensing image, wherein the first remote sensing image is a remote sensing image acquired from a target geographical area in a first period; A second acquisition module is used to acquire a second remote sensing image acquired from the target geographical area in a second period; a conversion module, configured to convert the first classification label value corresponding to the first image unit into an approximate label value close in size to the first classification label value, wherein a difference between the approximate label value and the first classification label value is within a preset interval, and the preset interval does not include 0; A determination module is configured to use the approximate label value as a classification label value corresponding to a second image unit, wherein the second image unit is an image unit in the second remote sensing image that corresponds to the same geographical area as the first image unit.

9. A label generating device, characterized in that: The label generating device comprises: a memory, a processor, and a label generating program stored in the memory and executable on the processor, wherein the label generating program implements the steps of the label generating method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a label generation program, and when the label generation program is executed by the processor, the steps of the label generation method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the label generation method according to any one of claims 1 to 7 are implemented.

Images