Training method and device for extracting mine land feature model

Abstract

The present invention provides a training method and device for a mine land feature extraction model, which relates to the field of remote sensing data processing. The training method of the feature extraction model includes: obtaining an original training sample set; performing a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set; performing a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement process methods; respectively inputting the first training sample set and the second training sample set into a model to be trained, performing iterative training, and stopping the iterative training until the similarity loss of the first feature and the second feature meets a preset condition, thereby obtaining a feature extraction model. The present invention solves the problem in the related art that the extracted mine land features are not suitable for multiple tasks.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing data processing, and in particular to a training method and device for a mine land feature extraction model. Background Art

[0002] In the study of mine land occupation, feature extraction of remote sensing data is the basis for realizing remote sensing scene classification, semantic segmentation, target detection and other tasks. At present, neural network models are mainly used to extract mine land occupation features in remote sensing data. In related technologies, a large amount of remote sensing data and corresponding label data are usually used to train the model to obtain a feature extraction model. However, since the label data is the correct output or target information of the remote sensing data under a certain task, the trained feature extraction model mainly captures the mine land occupation features related to the task corresponding to the label, resulting in insufficient or inapplicable feature performance in other tasks.

[0003] Therefore, there is a problem in the related art that the extracted mine area features are not suitable for multiple tasks. Summary of the invention

[0004] The problem solved by the present invention is that the mine area characteristics extracted in the related art are not well applicable to multiple tasks.

[0005] In order to solve the above problems, the present invention provides a training method and device for a mine land feature extraction model.

[0006] In a first aspect, the present invention provides a method for training a mine area feature extraction model, comprising:

[0007] Acquire an original training sample set, wherein the original training sample set includes a plurality of remote sensing data;

[0008] Performing a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set;

[0009] Performing a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods;

[0010] The first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition, and the iterative training is stopped to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process.

[0011] Optionally, the first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition and the iteration is stopped to obtain a feature extraction model, including: inputting the first training sample set into the first network branch in the model to be trained to obtain the first feature; inputting the second training sample set into the second network branch in the model to be trained to obtain the second feature; determining the similarity between the first feature and the second feature to obtain the similarity loss; adjusting the parameters of the model to be trained according to the similarity loss, and using the adjusted model to be trained for the next training.

[0012] Optionally, adjusting the parameters of the model to be trained according to the similarity loss includes: adjusting the parameters of the first network branch according to the similarity loss to obtain first parameters; sharing the first parameters to the second network branch to adjust the parameters of the second network branch to the first parameters.

[0013] Optionally, the method also includes: obtaining a third training sample set, wherein the third training sample set is a subset of the original training sample set; inputting the third training sample set and the data label set corresponding to the third training sample set into the third network branch in the model to be trained to obtain a prediction label set corresponding to the third training sample set; determining the loss of the data label set and the prediction label set to obtain a supervised loss; adjusting the parameters of the model to be trained according to the supervised loss and the similarity loss, and using the adjusted model to be trained for the next training.

[0014] Optionally, after obtaining the feature extraction model, the method further includes: inputting the target remote sensing data into a multi-task processing network, performing feature extraction on the target remote sensing data through a target extraction module in the multi-task processing network to obtain target features, wherein the target extraction module is obtained based on the feature extraction model; determining instance segmentation results, semantic segmentation results, and super-resolution reconstruction results corresponding to the target remote sensing data based on the target features.

[0015] Optionally, determining the instance segmentation result, semantic segmentation result, and super-resolution reconstruction result corresponding to the target remote sensing data according to the target features includes: decoding the target features through a super-resolution reconstruction task decoder to obtain a super-resolution reconstruction result, wherein a resolution of the super-resolution reconstruction result is greater than a resolution of the target remote sensing data; performing feature extraction on the super-resolution reconstruction result to obtain a third feature; and determining the instance segmentation result and the semantic segmentation result according to the third feature and the target feature.

[0016] Optionally, determining the instance segmentation result and the semantic segmentation result based on the third feature and the target feature includes: fusing the third feature and the target feature to obtain a fourth feature; decoding the fourth feature by an instance segmentation task decoder to obtain a fifth feature; decoding the fourth feature by a semantic segmentation task decoder to obtain a sixth feature; and determining the instance segmentation result and the semantic segmentation result based on the fifth feature and the sixth feature.

[0017] In a second aspect, the present invention provides a training device for a mine land feature extraction model, comprising:

[0018] An acquisition unit, used for acquiring an original training sample set, wherein the original training sample set includes a plurality of remote sensing data;

[0019] A first enhancement unit, configured to perform a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set;

[0020] A second enhancement unit is used to perform a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods;

[0021] An iterative unit is used to input the first training sample set and the second training sample set into the model to be trained respectively, perform iterative training, and stop the iterative training when the similarity loss of the first feature and the second feature meets a preset condition, so as to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process.

[0022] In a third aspect, the present invention provides an electronic device, including a memory and a processor;

[0023] The memory is used to store computer programs;

[0024] The processor is used to implement the training method of the mine area feature extraction model as described in the first aspect when executing the computer program.

[0025] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the training method for the mine area feature extraction model as described in the first aspect is implemented.

[0026] The beneficial effects of the training method and device of the mine land feature extraction model of the present invention are:

[0027] By performing different enhancement processing on the original training sample set, a first training sample set and a second training sample set are obtained. The model to be trained is obtained by training the first training sample set and the second training sample set. When the first feature and the second feature corresponding to the output first training sample set and the second training sample set respectively meet the preset conditions, the feature extraction model is determined. In the process of training the model, there is no need to add label data to the original training sample set, which shortens the time for producing training data. In addition, through comparative learning of the features output by the first training sample set and the second training sample set, the feature extraction model obtained can effectively learn the feature representation of the input data itself, so that the feature extraction model learns more abstract and more general feature representations that are not related to a single task. These features can be used in multiple tasks, which solves the problem that the extracted mine area features in the related technology are not well applicable to multiple tasks, thereby improving the applicability and generalization ability of the feature extraction model in multiple different tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic flow chart of a method for training a mine land feature extraction model according to an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the architecture of a model to be trained according to an embodiment of the present invention;

[0030] Figure 3 A schematic diagram of the architecture of a model to be trained according to another embodiment of the present invention;

[0031] Figure 4 A schematic diagram of a network architecture of a multi-task processing network according to an embodiment of the present invention;

[0032] Figure 5 A schematic diagram of network training of a multi-task processing network according to an embodiment of the present invention;

[0033] Figure 6 It is a structural schematic diagram of a training device for a mine land feature extraction model according to an embodiment of the present invention;

[0034] Figure 7 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0036] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0037] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0038] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0039] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0040] In the related art, a large amount of remote sensing data and corresponding label data are usually used to train the model to obtain a feature extraction model, wherein the label data corresponds to the task. For example, for scene classification tasks, the label data usually indicates the category label to which each remote sensing data belongs, such as "mountain", "desert", "city", etc. For target detection tasks, the label data usually includes the location information of the target (such as a bounding box) and category information. For semantic segmentation tasks, the label data is to mark the category to which each pixel belongs. In the related art, remote sensing data and label data of remote sensing data under a certain task are used to train a feature extraction model. The features captured by the trained model and related to the task corresponding to the label can only be used to perform the task, but the features extracted by the trained model cannot be used for other tasks. Therefore, there is a problem in the related art that the mine area features extracted by the trained feature extraction model are not suitable for multiple tasks.

[0041] In addition, there are problems such as huge workload, data quality limitations, and high marking difficulty when labeling remote sensing data.

[0042] In view of the problems existing in the above-mentioned related technologies, this embodiment provides a training method and device for a mine land feature extraction model.

[0043] like Figure 1 As shown, a method for training a mine land feature extraction model provided by an embodiment of the present invention includes:

[0044] S100, obtaining an original training sample set, wherein the original training sample set includes a plurality of remote sensing data;

[0045] Specifically, the original training sample set contains a large amount of remote sensing data, that is, multiple remote sensing data, wherein the multiple remote sensing data in the original training sample set can be multi-source remote sensing data, and multi-source remote sensing data refers to remote sensing data from different sensors or platforms, which can specifically include but are not limited to the following remote sensing data types: optical remote sensing data, radar remote sensing data, thermal infrared remote sensing data, and meteorological satellite data. The original training sample set is used as a training sample for the training feature extraction module. In this embodiment, when the original training sample set is used for iterative training, there is no need to add label data to the remote sensing data therein. It should be noted that the above remote sensing data is data collected in the mining area.

[0046] S200, performing a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set;

[0047] S300, performing a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods;

[0048] Specifically, two different data enhancement processing methods are used to enhance the original training sample set respectively, and the above two different data enhancement processing methods are the first enhancement processing and the second enhancement processing. Data enhancement processing includes but is not limited to flipping and rotation, cropping and scaling, color adjustment (brightness, contrast, saturation, etc.) and noise addition. The first enhancement processing can be a data enhancement processing in the above examples or a combination of multiple data enhancement processing. The first enhancement processing can be a data enhancement processing in the above examples or a combination of multiple data enhancement processing. The first enhancement processing and the second enhancement processing can also use the same enhancement processing, but the parameters in the enhancement processing process are different. For example, the first enhancement processing and the second enhancement processing can both use data enhancement processing with noise addition, but the first enhancement processing and the second enhancement processing have different noise quantities and positions added in the original training sample set, that is, it is only necessary to ensure that the first enhancement processing and the second enhancement processing are not exactly the same.

[0049] S400, respectively inputting the first training sample set and the second training sample set into the model to be trained, performing iterative training until the similarity loss between the first feature and the second feature meets a preset condition, and stopping the iterative training to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process.

[0050] Specifically, the first training sample set and the second training sample set obtained after different enhancement processing are respectively input into the model to be trained, and iterative training is performed. In any training process in the iterative training, the first training sample set is input into the model to be trained to obtain the corresponding first feature, and the second training sample set is input into the model to be trained to obtain the corresponding second feature. The input is a sample set after different data enhancement, so the extracted first feature and second feature should be different, but the original sample set used is the same. Therefore, the model is used to input the characteristics of the mining data itself. The higher the similarity between the first feature and the second feature, the stronger the feature extraction ability of the model, that is, after different enhancement processing is performed on the remote sensing data in the same original sample set, the features extracted by the model are still similar or the same. When the similarity between the first feature and the second feature output by the model to be trained meets the preset conditions, the model corresponding to the current model to be trained is determined to be the feature extraction model.

[0051] In this embodiment, by performing different enhancement processing on the original training sample set, a first training sample set and a second training sample set are obtained, and the model to be trained is obtained by training the first training sample set and the second training sample set. When the first feature and the second feature corresponding to the output first training sample set and the second training sample set respectively meet the preset conditions, the feature extraction model is determined. In the process of training the model, there is no need to add label data to the original training sample set, which shortens the time for producing training data. In addition, through comparative learning of the features output by the first training sample set and the second training sample set, the feature extraction model obtained can effectively learn the feature representation of the input data itself, so that the feature extraction model learns more abstract and more general feature representations that are not related to a single task. These features can be used in multiple tasks, which solves the problem that the extracted mine area features in the related art are not suitable for multiple tasks, thereby improving the applicability and generalization ability of the feature extraction model in multiple different tasks.

[0052] Optionally, the first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition and the iteration is stopped to obtain a feature extraction model, including: inputting the first training sample set into the first network branch in the model to be trained to obtain the first feature; inputting the second training sample set into the second network branch in the model to be trained to obtain the second feature; determining the similarity between the first feature and the second feature to obtain the similarity loss; adjusting the parameters of the model to be trained according to the similarity loss, and using the adjusted model to be trained for the next training.

[0053] Specifically, Figure 2 As shown, the architecture of the model to be trained includes a first network branch and a second network branch, the first training sample set and the second training sample set are respectively input into the first network branch and the second network branch, and the first feature and the second feature are respectively outputted, and the similarity loss is determined by comparing the similarity between the first feature and the second feature. When the similarity loss does not meet the preset conditions, the parameters of the model to be trained in the current training are adjusted according to the similarity loss calculated in the current training, and the adjusted model to be trained is trained for the next time, and the iterative training is stopped until the similarity loss meets the preset conditions, and the model to be trained in the last training is determined as the feature extraction module.

[0054] Preferably, if Figure 2 As shown, the first network branch includes an encoder (corresponding to Figure 2 The first encoder in) and the projector (corresponding to Figure 2 The first projector in the second network branch includes an encoder (corresponding to Figure 2The second encoder in Figure 2 The first encoder of the first network branch encodes the data in the first training sample set, extracts the seventh feature, inputs the seventh feature to the first projector, and obtains the eighth feature; the first encoder of the second network branch encodes the data in the first training sample set, extracts the ninth feature, inputs the ninth feature to the first projector, and obtains the second feature; in addition, since the data input to the first network branch and the second network branch are obtained through different enhancement processes, the two features output by the first projector and the second projector will be different due to the different enhancement processes. By adding a predictor after the first projector, the predictor is used to predict the deviation of feature extraction caused by the difference between the first enhancement process and the second enhancement process in the eighth feature, and eliminates the predicted deviation in the eighth feature, the above-mentioned first feature is obtained, so that the first feature and the second feature are as similar as possible, so as to improve the performance and training accuracy of the feature extraction model.

[0055] Optionally, adjusting the parameters of the model to be trained according to the similarity loss includes: adjusting the parameters of the first network branch according to the similarity loss to obtain first parameters; sharing the first parameters to the second network branch to adjust the parameters of the second network branch to the first parameters.

[0056] Specifically, in the model to be trained, the parameters of the first network branch and the second network branch need to be the same in order to better train and obtain the feature extraction model. Therefore, when adjusting the parameters of the model to be trained in the iterative training process, the parameters in the first network branch are first adjusted according to the similarity loss, and then the parameters of the second network branch are adjusted according to the adjusted parameters of the first network branch, so that the parameters of the first network branch and the parameters of the second network branch are consistent. Figure 2 For the column, when adjusting the parameters of the model to be trained, the parameters of the first encoder and the first projector in the first network branch are adjusted, and then the adjusted parameters of the first encoder and the first projector are shared to the second encoder and the second projector in the second network branch to adjust the parameters of the second encoder and the second projector, wherein after the adjustment, the parameters of the first encoder and the second encoder are the same, and the parameters of the first projector and the second projector are the same.

[0057] In this embodiment, the parameters in the first network branch are adjusted first, and then the parameters of the first network branch are shared with the parameters of the second network branch to ensure that the parameters of the first network branch and the second network branch are consistent.

[0058] Optionally, the method also includes: obtaining a third training sample set, wherein the third training sample set is a subset of the original training sample set; inputting the third training sample set and the data label set corresponding to the third training sample set into the third network branch in the model to be trained to obtain a prediction label set corresponding to the third training sample set; determining the loss of the data label set and the prediction label set to obtain a supervised loss; adjusting the parameters of the model to be trained according to the supervised loss and the similarity loss, and using the adjusted model to be trained for the next training.

[0059] Specifically, in order to further improve the feature extraction capability of the feature extraction model, such as Figure 3 As shown, on the basis of the model to be trained including the first network branch and the second network branch, a third network branch is added, and the input of the third network branch is the third training sample set, wherein the third training sample set is obtained by selecting certain data from the original training sample set, preferably, all optical remote sensing data contained in the multi-source remote sensing data in the original training sample set are selected, and label data are added to the selected optical remote sensing data to obtain a label data set corresponding to the third training sample set.

[0060] In the process of iterative training of the model to be trained, in addition to the above-mentioned inputting of the first training sample set into the first network branch to obtain the first feature, inputting the second training sample set into the second network branch to obtain the second feature, and calculating the similarity between the first feature and the second feature, the third training sample set and the corresponding label data set are input into the third network branch. The third network branch includes a third encoder and a classifier, which are used to extract the features of the third training sample set and predict the labels of the third training sample set to obtain a predicted label set. The supervision loss is determined according to the predicted label set and the input label set. The supervision loss can represent the task processing capability of the third network branch. The total loss value of the model to be trained is obtained by adding the supervision loss and the similarity loss. Directional propagation is performed according to the total loss value, that is, the model to be trained is adjusted, including adjusting the parameters of the first network branch, the second network branch and the third network branch.

[0061] Specifically, after obtaining the feature extraction model, the method also includes: inputting the target remote sensing data into a multi-task processing network, performing feature extraction on the target remote sensing data through a target extraction module in the multi-task processing network to obtain target features, wherein the target extraction module is obtained based on the feature extraction model; determining instance segmentation results, semantic segmentation results, and super-resolution reconstruction results corresponding to the target remote sensing data based on the target features.

[0062] Specifically, after the feature extraction model is obtained through the above training, the target extraction module can be used to extract the features of the remote sensing data, and multiple downstream tasks can be performed based on the extracted features. The multi-task processing network is used to perform instance segmentation, semantic segmentation, and super-resolution reconstruction on the input remote sensing data. Figure 4 As shown, the network architecture of the multi-task processing network includes a target extraction module, a super-resolution reconstruction task decoder, an instance segmentation task decoder and a semantic segmentation task decoder. The target remote sensing data is input into the multi-task processing network, and the feature extraction module in the multi-task processing network extracts features of the target remote sensing data to obtain target features. The super-resolution reconstruction task decoder, the instance segmentation task decoder and the semantic segmentation task decoder perform decoding operations according to the target features to obtain instance segmentation results, semantic segmentation results and super-resolution reconstruction results.

[0063] In this embodiment, the feature extraction module obtained through the above training extracts the target features of the target remote sensing data, and the target features can be used for the execution of multiple tasks (instance segmentation, semantic segmentation, super-resolution reconstruction), thereby improving the execution efficiency and accuracy of multiple tasks such as instance segmentation, semantic segmentation, and super-resolution reconstruction.

[0064] Before using the multitasking network to perform the above multiple tasks, such as Figure 5 As shown, firstly, an initial processing network is constructed, that is, the feature extraction module obtained by the above training is combined with the decoders corresponding to multiple tasks to obtain the initial processing network, and then the initial processing network is trained, a remote sensing data training set is selected for initial processing network training, and multi-task annotation is performed on each remote sensing data in the remote sensing data training set, including labels for instance segmentation tasks (category labels for different object instances), labels for semantic segmentation tasks (category to which each pixel belongs), and labels for super-resolution reconstruction tasks (high-resolution images corresponding to remote sensing data). During training, the remote sensing data training set is input into the initial processing network, and the initial processing network outputs instance segmentation results, semantic segmentation results, and super-resolution reconstruction results. The instance segmentation results, semantic segmentation results, and super-resolution reconstruction results output during the training process are used to calculate loss values ​​together with the annotations of the remote sensing data to obtain instance segmentation loss, semantic segmentation loss, and super-resolution reconstruction loss. The three losses are added and back-propagated to update the parameters in the initial processing network, including fine-tuning the feature extraction module and adjusting the parameters of the decoders corresponding to multiple tasks. Finally, when the three losses are less than a preset threshold, the training is completed to obtain the above-mentioned multi-task processing network. The target extraction module in the multi-task processing network is obtained by fine-tuning the feature extraction module.

[0065] The trained feature extraction model is adapted to different downstream tasks. By combining decoders of different tasks, different downstream tasks can be achieved. The efficient feature extraction model is used to replace the initialization stage of multi-task processing network training, which greatly reduces the training time.

[0066] Optionally, determining the instance segmentation result, semantic segmentation result, and super-resolution reconstruction result corresponding to the target remote sensing data according to the target features includes: decoding the target features through a super-resolution reconstruction task decoder to obtain a super-resolution reconstruction result, wherein a resolution of the super-resolution reconstruction result is greater than a resolution of the target remote sensing data; performing feature extraction on the super-resolution reconstruction result to obtain a third feature; and determining the instance segmentation result and the semantic segmentation result according to the third feature and the target feature.

[0067] Optionally, determining the instance segmentation result and the semantic segmentation result based on the third feature and the target feature includes: fusing the third feature and the target feature to obtain a fourth feature; decoding the fourth feature by an instance segmentation task decoder to obtain a fifth feature; decoding the fourth feature by a semantic segmentation task decoder to obtain a sixth feature; and determining the instance segmentation result and the semantic segmentation result based on the fifth feature and the sixth feature.

[0068] Specifically, the extracted target feature is copied into three copies, recorded as feature 1, feature 2 and feature 3. Feature 1 is input into the super-resolution reconstruction task decoder to obtain a higher super-resolution reconstruction result, and feature extraction is performed on the super-resolution reconstruction result again to obtain feature 4 (corresponding to the third feature). At this time, when extracting features, the features can be input into the target extraction module for extraction, or other feature extraction methods can be used for extraction; feature 4 is fused with features 2 and 3 respectively to obtain the fused feature 2 ′ (i.e. corresponding to the fourth feature) and feature 3 ′ (i.e. corresponding to the fourth feature). v and feature 3 ′ Input into the instance segmentation task decoder and the semantic segmentation task decoder for feature decoding, obtain the fifth feature and the sixth feature, and copy and exchange the fifth feature and the sixth feature, that is, copy the features extracted by the instance segmentation task decoder to the semantic segmentation task decoder, and copy the features extracted by the semantic segmentation task decoder to the instance segmentation task decoder. In this way, each decoder can obtain feature representations from different tasks, and finally obtain the instance segmentation result and the semantic segmentation result, among which the instance segmentation result will be affected by the features extracted by the semantic segmentation task decoder, and the semantic segmentation result will be affected by the features extracted by the instance segmentation task decoder.

[0069] In this embodiment, the feature interaction and information transfer between instance segmentation and semantic segmentation tasks can provide a deep understanding of the relationship between different tasks.

[0070] like Figure 6 As shown, a training device 600 for extracting a mine land feature model provided by an embodiment of the present invention includes:

[0071] An acquisition unit 610 is used to acquire an original training sample set, wherein the original training sample set includes a plurality of remote sensing data;

[0072] A first enhancement unit 620, configured to perform a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set;

[0073] A second enhancement unit 630 is used to perform a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods;

[0074] The iteration unit 640 is used to input the first training sample set and the second training sample set into the model to be trained respectively, and perform iterative training until the similarity loss of the first feature and the second feature meets a preset condition, and then stop the iterative training to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process.

[0075] Optionally, the iteration unit 640 is specifically used to: input the first training sample set into the first network branch in the model to be trained to obtain the first feature; input the second training sample set into the second network branch in the model to be trained to obtain the second feature; determine the similarity between the first feature and the second feature to obtain the similarity loss; adjust the parameters of the model to be trained according to the similarity loss, and use the adjusted model to be trained for the next training.

[0076] Optionally, the iteration unit 640 is also specifically used to adjust the parameters of the model to be trained according to the similarity loss in the following manner: adjust the parameters of the first network branch according to the similarity loss to obtain first parameters; share the first parameters to the second network branch to adjust the parameters of the second network branch to the first parameters.

[0077] Optionally, the training device 600 of the mine area feature extraction model is also used to: obtain a third training sample set, wherein the third training sample set is a subset of the original training sample set; input the third training sample set and the data label set corresponding to the third training sample set into the third network branch in the model to be trained to obtain the prediction label set corresponding to the third training sample set; determine the loss of the data label set and the prediction label set to obtain the supervision loss; adjust the parameters of the model to be trained according to the supervision loss and the similarity loss, and use the adjusted model to be trained for the next training.

[0078] Optionally, after obtaining the feature extraction model, the training device 600 of the mine area feature extraction model is also used to: input the target remote sensing data into the multi-task processing network, and perform feature extraction on the target remote sensing data through the target extraction module in the multi-task processing network to obtain target features, wherein the target extraction module is obtained based on the feature extraction model; determine the instance segmentation result, semantic segmentation result, and super-resolution reconstruction result corresponding to the target remote sensing data based on the target features.

[0079] Optionally, the training device 600 of the mine area feature extraction model is also used to determine the instance segmentation result, semantic segmentation result, and super-resolution reconstruction result corresponding to the target remote sensing data according to the target feature in the following manner: decoding the target feature through a super-resolution reconstruction task decoder to obtain a super-resolution reconstruction result, wherein the resolution of the super-resolution reconstruction result is greater than the resolution of the target remote sensing data; performing feature extraction on the super-resolution reconstruction result to obtain a third feature; determining the instance segmentation result and the semantic segmentation result according to the third feature and the target feature.

[0080] Optionally, optionally, the training device 600 of the mine area feature extraction model is also used to determine the instance segmentation result and the semantic segmentation result according to the third feature and the target feature in the following manner: fusing the third feature and the target feature to obtain a fourth feature; decoding the fourth feature by an instance segmentation task decoder to obtain a fifth feature; decoding the fourth feature by a semantic segmentation task decoder to obtain a sixth feature; determining the instance segmentation result and the semantic segmentation result according to the fifth feature and the sixth feature.

[0081] like Figure 7 As shown, an electronic device 700 provided by an embodiment of the present invention includes a memory 710 and a processor 720; the memory 710 is used to store a computer program; the processor 720 is used to implement the training method of the mine area feature extraction model as described above when executing the computer program.

[0082] In other words, an electronic device 700 includes a memory 710 and a processor 720 coupled to the memory 710; the memory 710 is configured to store a computer program; and the processor 720 is configured to perform the following operations when executing the computer program:

[0083] Acquire an original training sample set, wherein the original training sample set includes a plurality of remote sensing data;

[0084] Performing a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set;

[0085] Performing a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods;

[0086] The first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition, and the iterative training is stopped to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process.

[0087] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the training method of the mine area feature extraction model as described above is implemented.

[0088] In other words, a non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following operations:

[0089] Acquire an original training sample set, wherein the original training sample set includes a plurality of remote sensing data;

[0090] Performing a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set;

[0091] Performing a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods;

[0092] The first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition, and the iterative training is stopped to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process.

[0093] An electronic device 700 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 700 is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 700 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described herein and / or required.

[0094] The electronic device 700 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0095] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0096] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A training method for a mine area feature extraction model, characterized in that: include: Acquire an original training sample set, wherein the original training sample set includes a plurality of remote sensing data; Performing a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set; Performing a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods; The first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition, and the iterative training is stopped to obtain a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process, and the feature extraction model is used to extract the mine occupation feature in the input data; Inputting the target remote sensing data into the multi-task processing network, and extracting features of the target remote sensing data through a target extraction module in the multi-task processing network to obtain target features, wherein the target extraction module is obtained according to the feature extraction model; Decoding the target feature through a super-resolution reconstruction task decoder to obtain a super-resolution reconstruction result, wherein the resolution of the super-resolution reconstruction result is greater than the resolution of the target remote sensing data; Performing feature extraction on the super-resolution reconstruction result to obtain a third feature; The third feature and the target feature are combined to obtain a fourth feature; Decoding the fourth feature by an instance segmentation task decoder to obtain a fifth feature; Decoding the fourth feature through a semantic segmentation task decoder to obtain a sixth feature; The instance segmentation result and the semantic segmentation result are determined according to the fifth feature and the sixth feature.

2. The method for training a mine area feature extraction model according to claim 1, characterized in that: The first training sample set and the second training sample set are respectively input into the model to be trained, and iterative training is performed until the similarity loss between the first feature and the second feature meets a preset condition, and the iteration is stopped to obtain a feature extraction model, including: Inputting the first training sample set into the first network branch in the model to be trained to obtain the first feature; Inputting the second training sample set into the second network branch in the model to be trained to obtain the second feature; Determine the similarity between the first feature and the second feature to obtain the similarity loss; Parameters of the model to be trained are adjusted according to the similarity loss, and the adjusted model to be trained is used for the next training.

3. The method for training a mine area feature extraction model according to claim 2, characterized in that: The step of adjusting parameters of the model to be trained according to the similarity loss includes: Adjusting the parameters of the first network branch according to the similarity loss to obtain first parameters; The first parameter is shared with the second network branch to adjust the parameter of the second network branch to the first parameter.

4. The method for training a mine area feature extraction model according to claim 2, characterized in that: The method further comprises: Acquire a third training sample set, wherein the third training sample set is a subset of the original training sample set; Inputting the third training sample set and the data label set corresponding to the third training sample set into the third network branch in the model to be trained to obtain a prediction label set corresponding to the third training sample set; Determining the loss of the data label set and the predicted label set to obtain a supervision loss; Parameters of the model to be trained are adjusted according to the supervision loss and the similarity loss, and the adjusted model to be trained is used for the next training.

5. A training device for a mine area feature extraction model, characterized in that: include: An acquisition unit, used for acquiring an original training sample set, wherein the original training sample set includes a plurality of remote sensing data; A first enhancement unit, configured to perform a first enhancement process on the remote sensing data in the original training sample set to obtain a first training sample set; A second enhancement unit is used to perform a second enhancement process on the remote sensing data in the original training sample set to obtain a second training sample set, wherein the first enhancement process and the second enhancement process are different data enhancement processing methods; an iterative unit, used to input the first training sample set and the second training sample set into the model to be trained respectively, perform iterative training until the similarity loss between the first feature and the second feature satisfies a preset condition and then stop the iterative training, thereby obtaining a feature extraction model, wherein the first feature is a feature corresponding to the first training sample set output by the model to be trained during the iteration process, and the second feature is a feature corresponding to the second training sample set output by the model to be trained during the iteration process, and the feature extraction model is used to extract the mine occupation feature in the input data; The device is also used to: input the target remote sensing data into a multi-task processing network, perform feature extraction on the target remote sensing data through a target extraction module in the multi-task processing network to obtain target features, wherein the target extraction module is obtained according to the feature extraction model; decode the target features through a super-resolution reconstruction task decoder to obtain a super-resolution reconstruction result, wherein the resolution of the super-resolution reconstruction result is greater than the resolution of the target remote sensing data; perform feature extraction on the super-resolution reconstruction result to obtain a third feature; fuse the third feature with the target feature to obtain a fourth feature; decode the fourth feature through an instance segmentation task decoder to obtain a fifth feature; decode the fourth feature through a semantic segmentation task decoder to obtain a sixth feature; determine the instance segmentation result and the semantic segmentation result according to the fifth feature and the sixth feature.

6. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the training method of the mine area feature extraction model as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the training method for the mine area feature extraction model as described in any one of claims 1 to 4 is implemented.

Images