Remote sensing data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a remote sensing data processing method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: in response to a region selection operation for initial remote sensing data, obtaining a region image segmentation vector and a region image text vector of a target region; the regional image segmentation vector represents respective position distribution and element categories of a plurality of elements in the target region, and the regional image text vector represents respective element information of the plurality of elements; determining a category label set and a pixel coding set of the plurality of elements based on the region image segmentation vector, and determining text description information of the plurality of elements based on the region image text vector; wherein elements corresponding to the image areas corresponding to different pixel codes in the pixel code set are different; based on the sub remote sensing data corresponding to the pixel coding set, the category label set and the text description information, the area monitoring information of the target area is obtained, and the accuracy or efficiency of target area monitoring is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a remote sensing data processing method, device, electronic device and storage medium. Background Art

[0002] At present, remote sensing technology has been widely used in the agricultural field. For example, it is used in many aspects such as crop planting area, growth conditions, soil moisture, yield estimation and disaster monitoring.

[0003] Among them, for the monitoring of crop planting area, a variety of algorithms (for example, support vector machine (SVM) classification method, random forest (RF) classification method, normalized difference vegetation index (NDVI) difference method, machine learning algorithm and deep learning algorithm, etc.) can be used to identify and classify crops using remote sensing data (such as remote sensing images), and then estimate the planting area of ​​target crops.

[0004] However, each of the above algorithms has certain limitations, which results in low accuracy or efficiency in crop planting area monitoring (i.e., target area monitoring). Therefore, how to improve the accuracy or efficiency of target area monitoring is a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a remote sensing data processing method, device, electronic device and storage medium to improve the accuracy or efficiency of target area monitoring.

[0006] In a first aspect, an embodiment of the present application provides a remote sensing data processing method, the method comprising:

[0007] In response to the region selection operation for the initial remote sensing data, a region image segmentation vector and a region image text vector of the target region are obtained; wherein the region image segmentation vector represents the position distribution and feature category of each of the multiple elements in the target region, and the region image text vector represents the feature information of each of the multiple elements;

[0008] Determine a set of category labels and a set of pixel codes of multiple elements based on the regional image segmentation vector, and determine text description information of multiple elements based on the regional image text vector; wherein different pixel codes in the pixel code set correspond to different elements in the image region;

[0009] Based on the sub-remote sensing data corresponding to the pixel coding set, as well as the category label set and text description information, the regional monitoring information of the target area is obtained.

[0010] In an optional embodiment, obtaining a regional image segmentation vector and a regional image text vector of a target region includes:

[0011] Perform data preprocessing on the initial remote sensing data to obtain target remote sensing data;

[0012] Perform image segmentation encoding and image text encoding on the target remote sensing data respectively, and obtain a scene image segmentation vector and a scene image text vector of the target scene corresponding to the initial remote sensing data;

[0013] Based on the prompt code, scene image segmentation vector and scene image text vector corresponding to the region selection operation, a region image segmentation vector and a region image text vector are obtained; wherein the prompt code is used to indicate a target region in a target scene.

[0014] In an optional embodiment, data preprocessing is performed on the initial remote sensing data to obtain target remote sensing data, including:

[0015] Obtaining environmental information when collecting initial remote sensing data;

[0016] Based on the data correction methods respectively set for the multiple remote sensing data influencing factors included in the environmental information, the initial remote sensing data is preprocessed to obtain the target remote sensing data.

[0017] In an optional embodiment, obtaining the region image segmentation vector and the region image text vector based on the prompt code, the scene image segmentation vector and the scene image text vector corresponding to the region selection operation includes:

[0018] Performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain a scene image fusion vector, and obtaining a region image segmentation vector based on the scene image fusion vector and the prompt code;

[0019] And, based on the prompt encoding and the scene image text vector, the region image text vector is obtained.

[0020] In an optional embodiment, performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain the scene image fusion vector includes:

[0021] Convert the scene image text vector into a text vector with the same number of vector elements as the scene image segmentation vector;

[0022] Feature vector fusion is performed on the scene image segmentation vector and the converted scene image text vector to obtain a scene image fusion vector.

[0023] In an optional embodiment, based on the sub-remote sensing data corresponding to the pixel code set, as well as the category label set and the text description information, the regional monitoring information of the target area is obtained, including:

[0024] Based on the regional position corresponding to the pixel code set, sub-remote sensing data is obtained from the initial remote sensing data;

[0025] Perform image encoding on the sub-remote sensing data to obtain an image vector, and respectively perform word encoding on the category label set and the text description information to obtain the word vectors corresponding to the category label set and the text description information;

[0026] Based on the image vector and the two word vectors obtained, the region monitoring information is obtained.

[0027] In an optional embodiment, the regional monitoring information includes: the regional type of the target area, and at least one of the element categories, location distribution, resource occupancy and expected returns corresponding to the multiple elements.

[0028] In a second aspect, an embodiment of the present application further provides a remote sensing data processing device, the device comprising:

[0029] An acquisition module, for acquiring a regional image segmentation vector and a regional image text vector of a target region in response to a region selection operation on the initial remote sensing data; wherein the regional image segmentation vector represents the position distribution and feature category of each of a plurality of elements in the target region, and the regional image text vector represents the feature information of each of the plurality of elements;

[0030] A determination module, used to determine a set of category labels and a set of pixel codes of multiple elements based on the regional image segmentation vector, and to determine text description information of multiple elements based on the regional image text vector; wherein different pixel codes in the pixel code set correspond to different elements in the image region;

[0031] The processing module is used to obtain the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel coding set, the category label set and the text description information.

[0032] In an optional embodiment, when acquiring the regional image segmentation vector and the regional image text vector of the target area, the acquisition module is specifically used to:

[0033] Perform data preprocessing on the initial remote sensing data to obtain target remote sensing data;

[0034] Perform image segmentation encoding and image text encoding on the target remote sensing data respectively, and obtain a scene image segmentation vector and a scene image text vector of the target scene corresponding to the initial remote sensing data;

[0035] Based on the prompt code, scene image segmentation vector and scene image text vector corresponding to the region selection operation, a region image segmentation vector and a region image text vector are obtained; wherein the prompt code is used to indicate a target region in a target scene.

[0036] In an optional embodiment, when performing data preprocessing on the initial remote sensing data to obtain the target remote sensing data, the acquisition module is specifically used to:

[0037] Obtaining environmental information when collecting initial remote sensing data;

[0038] Based on the data correction methods respectively set for the multiple remote sensing data influencing factors included in the environmental information, the initial remote sensing data is preprocessed to obtain the target remote sensing data.

[0039] In an optional embodiment, when obtaining the region image segmentation vector and the region image text vector based on the prompt code, the scene image segmentation vector and the scene image text vector corresponding to the region selection operation, the acquisition module is specifically used to:

[0040] Performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain a scene image fusion vector, and obtaining a region image segmentation vector based on the scene image fusion vector and the prompt code;

[0041] And, based on the prompt encoding and the scene image text vector, the region image text vector is obtained.

[0042] In an optional embodiment, when the scene image segmentation vector and the scene image text vector are subjected to feature vector fusion to obtain the scene image fusion vector, the acquisition module is specifically used to:

[0043] Convert the scene image text vector into a text vector with the same number of vector elements as the scene image segmentation vector;

[0044] Feature vector fusion is performed on the scene image segmentation vector and the converted scene image text vector to obtain a scene image fusion vector.

[0045] In an optional embodiment, when obtaining the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information, the processing module is specifically used to:

[0046] Based on the regional position corresponding to the pixel code set, sub-remote sensing data is obtained from the initial remote sensing data;

[0047] Perform image encoding on the sub-remote sensing data to obtain an image vector, and respectively perform word encoding on the category label set and the text description information to obtain the word vectors corresponding to the category label set and the text description information;

[0048] Based on the image vector and the two word vectors obtained, the region monitoring information is obtained.

[0049] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0050] Processor; and

[0051] Memory for storing programs,

[0052] The program includes instructions, which, when executed by a processor, cause the processor to execute the remote sensing data processing method as described in the first aspect.

[0053] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the remote sensing data processing method as described in the first aspect.

[0054] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the remote sensing data processing method as described in the first aspect.

[0055] The beneficial effects of this application are as follows:

[0056] In the remote sensing data processing method provided in the embodiment of the present application, according to the region selection operation for the initial remote sensing data, the regional image segmentation vector and the regional image text vector of the target area are obtained, and regional monitoring can be performed only for the target area, thereby improving the efficiency of target area monitoring; and, after determining the category label set and the pixel coding set of multiple elements in the target area based on the regional image segmentation vector, and determining the text description information of multiple elements based on the regional image text vector, the regional monitoring information of the target area can be obtained based on the sub-remote sensing data corresponding to the pixel coding set, as well as the category label set and the text description information. In this way, through the full combination of image data (for example, the sub-remote sensing data corresponding to the target area) and text information, the understanding and analysis of the image data corresponding to the target area is improved, thereby improving the accuracy of target area monitoring.

[0057] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described here are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0059] Figure 1 A schematic diagram of an optional system architecture applicable to the embodiments of the present application;

[0060] Figure 2 A schematic diagram of an implementation flow of a remote sensing data processing method provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a scenario for acquiring remote sensing data provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of an implementation flow of a vector generation method provided in an embodiment of the present application;

[0063] Figure 5 A method based on the embodiment of the present application is provided Figure 4 Schematic diagram of the

[0064] Figure 6 A schematic diagram of a network structure for classifying land features provided in an embodiment of the present application;

[0065] Figure 7 A schematic diagram of an implementation flow of a method for determining regional monitoring information provided in an embodiment of the present application;

[0066] Figure 8 A method based on the embodiment of the present application is provided Figure 7 Schematic diagram of the

[0067] Fig. 9 A schematic diagram of the architecture of a remote sensing data processing system provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the structure of a remote sensing data processing device provided in an embodiment of the present application;

[0069] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0071] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0072] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0073] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0074] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0075] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0076] (1) NDVI: It can separate vegetation from water and soil and objectively reflect the changes in vegetation coverage. It is the best indicator of vegetation growth status and vegetation coverage.

[0077] (2) Encoder: A model structure in deep learning that converts raw data into a low-dimensional vector representation and can extract key features of the raw data.

[0078] (3) Decoder: In deep learning, the decoder converts the low-dimensional vector representation generated by the encoder back to the model structure of the original data space.

[0079] (4) Model interpretability: refers to the understanding of the internal mechanism of the model and the understanding of the model results, that is, the degree to which the decision rules and prediction results of the machine learning model can be understood. It involves understanding the decision-making process of the model and how the model makes predictions or classifications based on the input data.

[0080] (5) Pixel size: the size of a pixel. The pixel size and the number of pixels (resolution) together determine the size of the camera target surface. The pixel is the image unit, which is the smallest unit that makes up the digital image.

[0081] (6) Segment anything model (SAM): It can also be called the general industrial scene segmentation model. It is a general model for image segmentation tasks. Unlike image segmentation models that can only process a certain type of picture or image, SAM can process all types of images. Compared with previous image segmentation models, SAM can recognize various input prompts and determine the content that needs to be segmented in the image. It can also be flexibly integrated into other systems such as virtual reality / augmented reality, and can achieve better image segmentation effects.

[0082] (7) Contrastive language-image pre-training (CLIP) model: It is a multimodal learning framework that closely combines images and texts through contrastive learning. In other words, its main goal is to learn to match images and texts through contrastive learning.

[0083] Based on the above-mentioned nouns and related terminology explanations, the design concept of the embodiments of the present application is briefly introduced below:

[0084] Remote sensing technology has been widely used in the agricultural field, and with the development of technology, its research on helping agriculture is also deepening. For example, it is widely used in crop planting area, growth status, soil moisture, yield estimation and disaster monitoring. These monitoring results are crucial for agricultural situation analysis, food security early warning and agricultural policy formulation.

[0085] Although remote sensing technology has made significant progress in the field of agriculture, it still faces some serious challenges, such as how to improve the quantification and objectivity of monitoring information, and how to solve the impact of phenological changes, crop rotation and fallow phenomena, and vegetation index saturation on monitoring results. Therefore, in order to solve the above problems, a series of ground object recognition models have been proposed.

[0086] Taking crop planting area monitoring as an example, a series of algorithms can be used in the object recognition model, and remote sensing information can be used to identify and classify crops, so as to estimate the planting area of ​​target crops. For example, the aforementioned algorithms can include any one or combination of the following to meet different monitoring needs and improve the accuracy of crop planting area monitoring.

[0087] 1. SVM classification method: SVM classification technology is used to identify the crop planting areas in a specific area.

[0088] 2. RF classification method: The accuracy of extracting or monitoring crop planting area can be improved by RF classification method.

[0089] 3. NDVI difference method: The NDVI difference method can be used to track the growth of crops throughout the entire growth stage.

[0090] 4. Multi-temporal image analysis: Use multi-temporal remote sensing images to obtain crop planting areas, and extract crop planting areas through data correction and classification.

[0091] 5. Mixed pixel decomposition: Use mixed pixel linear analysis method to estimate the crop planting area.

[0092] 6. Phenological analysis of crops: Based on the different phenological calendar characteristics of crops, extract or monitor the planting area of ​​different crops.

[0093] 7. Combination of multispectral and microwave remote sensing images: Combine multispectral and microwave remote sensing image methods to improve the accuracy of crop planting area extraction.

[0094] 8. Time series analysis: Differentiate the planting areas of different crops by analyzing the changes in the time series vegetation index of crops.

[0095] 9. Machine Learning Methods: Apply machine learning algorithms for crop classification and area estimation.

[0096] 10. Deep learning algorithm: Use deep learning algorithm to extract or monitor crop planting area from remote sensing images.

[0097] 11. Spectral feature analysis: Classification and area estimation based on differences in crop spectral features.

[0098] 12. Comprehensive analysis of vegetation indices: Use a variety of vegetation indices (such as NDVI, ratio vegetation index (RVI), enhanced vegetation index (EVI), etc.) to classify crops.

[0099] However, each of the above algorithms has certain limitations, which results in low accuracy or efficiency in crop planting area monitoring (i.e., target area monitoring). For example, the limitations or shortcomings of each of the above algorithms are as follows:

[0100] 1. SVM classification method: Although SVM performs well in classification problems, it may require a large number of training samples and is highly sensitive to parameter selection and kernel functions.

[0101] 2. RF classification method: Although RF has a good performance in classification accuracy, the model may encounter overfitting problems and the model's interpretability is poor.

[0102] 3. NDVI difference method: NDVI is a simple and effective index, but it may be affected by soil brightness and cloud cover, limiting its application under certain conditions.

[0103] 4. Multi-temporal image analysis: Multi-temporal remote sensing images can provide rich information, but the data processing is relatively complex and requires a lot of computing resources and storage space.

[0104] 5. Hybrid pixel decomposition: May not be accurate enough at the pixel scale, especially in areas of high heterogeneity.

[0105] 6. Phenological analysis of crops: Phenological analysis relies on the biological cycle of crops. If crop planting patterns or growing conditions change, classification accuracy may be affected.

[0106] 7. Combining multispectral and microwave remote sensing images: Combining multispectral and microwave data can provide more comprehensive information, but data acquisition and processing may be more complex and expensive.

[0107] 8. Time series analysis: Time series analysis requires continuous imaging data. If data acquisition is interrupted or there are quality problems, the accuracy of the analysis results may be affected.

[0108] 9. Machine learning methods: Machine learning algorithms usually require a large amount of labeled data for training, and the generalization ability of the model may be limited by the diversity and representativeness of the training data.

[0109] 10. Deep learning algorithms: Deep learning models perform well in extracting features, but they may require a lot of computing resources and expertise in model training and tuning.

[0110] 11. Spectral feature analysis: may be affected by sensor noise and atmospheric conditions, requiring complex data preprocessing.

[0111] 12. Comprehensive analysis of vegetation indices: Although the integration of multiple vegetation indices can improve the accuracy of classification, different indices may have different sensitivities to different crops and environmental conditions and need to be carefully selected and optimized.

[0112] In view of this, in order to solve or improve the above-mentioned problems, an embodiment of the present application proposes a remote sensing data processing method, which may specifically include: in response to a region selection operation on initial remote sensing data, obtaining a regional image segmentation vector and a regional image text vector of the target area; wherein the aforementioned regional image segmentation vector can represent the position distribution and feature category of each of the multiple elements in the target area, and the aforementioned regional image text vector can represent the feature information of each of the multiple elements; then, based on the aforementioned regional image segmentation vector, determining the category label set and pixel code set of the multiple elements, and based on the aforementioned regional image text vector, determining the text description information of the multiple elements; wherein, the image areas corresponding to different pixel codes in the pixel code set correspond to different elements; finally, based on the sub-remote sensing data corresponding to the pixel code set, as well as the category label set and the text description information, obtaining the regional monitoring information of the target area.

[0113] Based on the above method, regional monitoring can be performed only on the target area, thereby improving the efficiency of target area monitoring; and, based on the sub-remote sensing data corresponding to the pixel coding set, as well as the category label set and text description information, the regional monitoring information of the target area can be obtained. Through the full combination of image data and text information, the understanding and analysis of the image data corresponding to the target area is improved, thereby improving the accuracy of target area monitoring.

[0114] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0115] See also Figure 1 As shown, it is a schematic diagram of a system architecture applicable to an embodiment of the present application, and the system architecture may include: terminal devices (101a, 101b, 101c) and a server 102. The terminal devices (101a, 101b, 101c) and the server 102 may exchange information through a communication network, wherein the communication mode adopted by the communication network may include: a wireless communication mode and a wired communication mode.

[0116] Exemplarily, the terminal devices (101a, 101b, 101c) can access the network through cellular mobile communication technology and communicate with the server 102. The cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th generation mobile networks, 5G) technology or the next generation mobile communication technology. Optionally, the terminal devices (101a, 101b, 101c) can access the network through short-range wireless communication mode and communicate with the server 102. The short-range wireless communication mode, for example, includes wireless fidelity (wireless fidelity, Wi-Fi) technology.

[0117] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above system architecture. For example, the above system architecture may include more terminal devices, or fewer terminal devices, or other network devices. Figure 1 As shown, only the terminal devices (101a, 101b, 101c) and the server 102 are described as examples, and the above communication devices and their respective functions are briefly introduced below.

[0118] The terminal device (101a, 101b, 101c) is a device that can provide voice and / or data connectivity to users, and can be a device that supports wired and / or wireless connection.

[0119] Exemplarily, the terminal devices (101a, 101b, 101c) may include, but are not limited to: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in unmanned driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0120] In addition, a related client may be installed on the terminal device (101a, 101b, 101c), and the client may be software, such as an application (APP), a browser, a short video software, etc., or a web page, a mini-program, etc.; it should be noted that in the embodiment of the present application, the terminal device (101a, 101b, 101c) may enable the above-mentioned client related to remote sensing data processing to send the area selection operation for the initial remote sensing data, or the collected initial remote sensing data, to the server 102, respectively, so as to perform subsequent remote sensing data processing and other method steps.

[0121] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0122] Optionally, a remote sensing data processing system (or remote sensing agricultural assistance system) is deployed on the server 102 in the embodiment of the present application, which can obtain the regional image segmentation vector and the regional image text vector of the target area in response to the area selection operation on the initial remote sensing data; determine the category label set and the pixel code set of multiple elements based on the regional image segmentation vector, and determine the text description information of multiple elements based on the regional image text vector; obtain the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, as well as the category label set and the text description information.

[0123] The remote sensing data processing method provided by the exemplary embodiment of the present application is described below in combination with the above-mentioned system architecture and with reference to the accompanying drawings. It should be noted that the above-mentioned system architecture is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.

[0124] See also Figure 2 As shown, it is a schematic diagram of an implementation process of a remote sensing data processing method provided in an embodiment of the present application. The execution subject takes the remote sensing data processing system in the server as an example. The specific implementation process of the method is as follows:

[0125] S201: In response to a region selection operation on initial remote sensing data, a region image segmentation vector and a region image text vector of a target region are obtained.

[0126] Among them, the above-mentioned initial remote sensing data is remote sensing data (such as remote sensing images) collected by remote sensing equipment. The aforementioned remote sensing equipment can be equipment such as drones or remote sensing satellites, and this embodiment of the present application does not limit this.

[0127] The above-mentioned target area is also the area corresponding to the area selection operation. The above-mentioned area selection operation can be an area delineation operation on the above-mentioned initial remote sensing data to achieve the selection of the target area. Optionally, the above-mentioned area selection operation can also be a click operation on the above-mentioned initial remote sensing data, and the target area can be selected according to the click position and the preset frame range. There can be multiple preset width ranges. The embodiment of the present application is not limited to this. It should be noted that the above-mentioned area selection operation can be understood as the interactive information between the user and the remote sensing data.

[0128] See also Figure 3 As shown, after receiving the region selection operation for the initial remote sensing data (eg, Source image), the remote sensing data processing system can obtain sub-remote sensing data corresponding to the target region, eg, Regional image, from the aforementioned initial remote sensing data Source image.

[0129] In addition, the above-mentioned regional image segmentation vector can represent the location distribution and element category of each of the multiple elements included in the target area, and the above-mentioned regional image text vector can represent the element information of each of the aforementioned multiple elements. For example, the aforementioned multiple elements can also be called multiple objects or multiple entities, for example, they can only be crops in the target area. Of course, other elements besides crops can also be included, such as land, water, etc.

[0130] It should be noted that the above-mentioned location distribution may include but is not limited to: the size of the area occupied by each element, the shape of the area occupied by each element, etc. The above-mentioned element category is also the type of element, for example, the type of crop or the type of plant. The above-mentioned element information is also the text description or text information of the element attribute. It should be understood that the above-mentioned element information is also the text description associated with the sub-remote sensing data (such as Regional image) corresponding to the target area.

[0131] In an optional implementation, when executing step S201, the remote sensing data processing system can obtain the regional image segmentation vector and the regional image text vector of the target area after determining the target area corresponding to the area selection operation, so as to ensure that the regional monitoring information of the target area is subsequently obtained based on the regional image segmentation vector and the regional image text vector.

[0132] See also Figure 4 As shown, the remote sensing data processing system can execute the following specific process when obtaining the regional image segmentation vector and the regional image text vector of the target area:

[0133] S401: Perform data preprocessing on the initial remote sensing data to obtain target remote sensing data.

[0134] Specifically, when executing step S401, considering that the remote sensing equipment is easily affected by environmental factors when collecting or acquiring remote sensing data, in order to improve the accuracy of the subsequent regional monitoring information of the target area, the environmental information when the aforementioned initial remote sensing data is collected can be obtained, and then the initial remote sensing data is preprocessed based on the data correction methods set for the multiple remote sensing data influencing factors included in the environmental information to obtain the target remote sensing data.

[0135] The above data correction method may include but is not limited to: performing pre-processing operations such as radiation correction, atmospheric correction, geometric correction and cloud occlusion processing on the above initial remote sensing data. It can be seen that this method can, to a certain extent, change the defects of the existing method. For example, the problem that the NDIV difference method may be affected by cloud occlusion can be improved, and for another example, the problem that the spectral feature analysis may be affected by atmospheric conditions can be improved.

[0136] S402: performing image segmentation coding and image text coding on the target remote sensing data respectively, and obtaining a scene image segmentation vector and a scene image text vector of the target scene corresponding to the initial remote sensing data.

[0137] Since SAM can provide accurate pixel-level segmentation when applied to object recognition, it is very useful for fine classification in object recognition. At the same time, SAM has strong generalization ability. SAM is pre-trained on a large amount of data and has good generalization ability, and can recognize and segment a variety of object types. In addition, the interactive nature of SAM allows users to continuously optimize the segmentation results by providing feedback, which is crucial for improving recognition accuracy. In addition, SAM can also perform object recognition and other related tasks at the same time, such as object classification or attribute prediction. Therefore, in the embodiment of the present application, the target remote sensing data can be image segmented and encoded by SAM to obtain the scene image segmentation vector of the target scene corresponding to the initial remote sensing data.

[0138] In addition, because the CLIP model can understand the connection between image content and associated text descriptions, it can achieve cross-modal semantic understanding; at the same time, the CLIP model has a strong zero-shot learning ability, and can effectively classify and retrieve images based on text descriptions even in categories that have never been seen. Moreover, the CLIP model is pre-trained on a large-scale multi-category dataset, so it has good generalization ability and can adapt to various image and text tasks; it adopts a contrastive learning method, does not require a large amount of labeled data, reduces the dependence on manual annotation, and does not require additional supervision; it can directly learn from the original data of images and texts, without complex feature engineering, and achieve end-to-end training; the feature representation learned during the training process has a certain robustness to noise and occlusion; and it can capture the contextual information in the image, understand the relationship between the objects in the image and their correspondence with the text description. That is, these advantages of the CLIP model make it a powerful tool in multimodal learning and image-text alignment tasks, especially in scenarios where the model needs to understand the image content and operate according to the text description. Therefore, in the embodiment of the present application, the target remote sensing data can be image-text encoded through the CLIP model to obtain the scene image text vector of the above target scene.

[0139] Exemplarily, when executing step S402, refer to Figure 5 As shown, the remote sensing data processing system can perform image segmentation encoding on the target remote sensing data (such as Target image) through the encoder in SAM, namely SAM-encoder, to obtain the scene image segmentation vector Scene_vector.1; and, through the encoder in CLIP, namely CLIP-encoder, perform image text encoding on the target remote sensing data (such as Target image) to obtain the scene image text vector Scene_vector.2.

[0140] S403: Obtaining a region image segmentation vector and a region image text vector based on the prompt code, the scene image segmentation vector and the scene image text vector corresponding to the region selection operation.

[0141] The prompt code can be used to indicate the target area in the target scene. Optionally, the prompt code can be obtained by encoding the target area (eg, Point) corresponding to the area selection operation using a preset boost encoder (eg, Prompt-encoder) in the remote sensing data processing system.

[0142] Specifically, when executing step S403, the remote sensing data processing system can determine the regional image segmentation vector corresponding to the target area from the scene image segmentation vector according to the prompt code corresponding to the target area, and determine the regional image text vector corresponding to the target area from the scene image text vector according to the prompt code corresponding to the target area. In other words, the scene image segmentation vector can be regarded as a set of regional image segmentation vectors, and the scene text segmentation vector can be regarded as a set of regional text segmentation vectors. The regional image segmentation vector and the regional image text vector corresponding to the target area are obtained from the scene image segmentation vector and the scene image text vector according to the prompt code (i.e., the region identifier) ​​of the target area.

[0143] In order to improve the accuracy of the regional monitoring information of the target area subsequently acquired, in an optional implementation, the remote sensing data processing system can also perform feature vector fusion on the above-mentioned scene image segmentation vector and the above-mentioned scene image text vector to obtain a scene image fusion vector, and obtain a regional image segmentation vector corresponding to the target area based on the scene image fusion vector and the prompt code corresponding to the target area; and obtain a regional image text vector corresponding to the target area based on the prompt code corresponding to the target area and the scene image text vector. In this way, the fusion of image information and text information can enhance semantic understanding and improve the understanding and analysis of the image content included in the initial remote sensing data.

[0144] Furthermore, when the remote sensing data processing system performs feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain the scene image fusion vector, the scene image text vector can be converted into a text vector with the same number of vector elements as the scene image segmentation vector, that is, the vector dimension of the scene image text vector is adjusted to a text vector with the same vector dimension as the scene image segmentation vector; then, the scene image segmentation vector and the converted scene image text vector are subjected to feature vector fusion to obtain the scene image fusion vector. In this way, by adjusting the number of vector elements of the scene image text vector and the scene image segmentation vector to be consistent, the speed of feature vector fusion can be accelerated.

[0145] Exemplarily, assuming that the scene image segmentation vector Scene_vector.1 is a 26-dimensional vector (i.e., the number of vector elements or the number is 26), and the scene image text vector Scene_vector.2 is a 20-dimensional vector (i.e., the number of vector elements or the number is 20). The remote sensing data processing system can convert the scene image text vector Scene_vector.2 containing 20 vector elements into the scene image text vector Scene_vector.3 containing 26 vector elements.

[0146] Optionally, the remote sensing data processing system may pad the number of vector elements of the scene image text vector Scene_vector.2 by adding "0". Of course, other methods may also be used to pad the number of vector elements of the scene image text vector Scene_vector.2, which is not limited in the embodiments of the present application.

[0147] S202: Determine a category label set and a pixel encoding set of multiple elements based on the regional image segmentation vector, and determine text description information of the multiple elements based on the regional image text vector.

[0148] Different category labels in the above category label set indicate different element categories, and image regions corresponding to different pixel codes in the above pixel code set correspond to different elements. That is, the above pixel code set includes at least one pixel code, and image regions corresponding to different pixel codes can be used to indicate or refer to different elements.

[0149] Assuming that the above pixel code set includes three pixel codes (e.g., pixel code "0", pixel code "1" and pixel code "2"), the image areas corresponding to the above three pixel codes can be used to indicate three different elements. For example, pixel code "0" corresponds to crop 1, pixel code "1" corresponds to crop 2 and pixel code "2" corresponds to crop 3.

[0150] Still taking the above-mentioned SAM and CLIP as an example, the remote sensing data processing system can generate a category label set (e.g., Labels) and a pixel encoding set (e.g., Masks) for the regional image segmentation vector (e.g., Region_vector.1) through the decoder in SAM, where Masks is used to represent the boundaries and regions of each object in the initial remote sensing data.

[0151] Furthermore, the remote sensing data processing system can generate text description information for the regional image text vector (eg, Region_vector.3) through the decoder in CLIP.

[0152] For example, see Figure 6As shown, it is a schematic diagram of a network structure for object classification provided by an embodiment of the present application. The network structure may include SAM and CLIP. The input of the network structure may be the target remote sensing data Target image obtained through data preprocessing and the area selection operation Point. The target remote sensing data Target image passes through the SAM-encoder and the CLIP-encoder respectively to obtain the scene image segmentation vector Scene_vector.1 and the scene image text vector Scene_vector.2. Then, the number of vector elements of the scene image text vector Scene_vector.2 can be adjusted to the scene image text vector Scene_vector.3 with the same number of vector elements as the scene image segmentation vector Scene_vector.1 through an adapter (such as Adapter). Furthermore, the scene image segmentation vector Scene_vector.1 and the scene image text vector Scene_vector.3 are feature fused through the fusion module Fusion, and the scene image fusion vector Fusion_vector.1 is obtained as the input of the SAM-decoder, and point is also input into the SAM-decoder after passing through the Prompt-encoder, and finally the SAM-decoder outputs the final Mask prediction result (i.e., the pixel encoding set Masks) and the corresponding category information (i.e., the category label set Labels). At the same time, the output of the CLIP-encoder (i.e., the scene image text vector Scene_vector.2) and the output of the Prompt-encoder (i.e., the prompt encoding) are merged and input into the CLIP-decoder, thereby obtaining the text description information Text corresponding to the target area.

[0153] It should be noted that the encoder in SAM can usually use a network structure based on vision transformer (VIT) as the backbone network to achieve this function. Optionally, the main structure of the encoder in CLIP is also based on the transformer network structure. This is not limited in the embodiments of the present application.

[0154] By combining SAM and CLIP, semantic understanding can be enhanced. The CLIP model learns a wide range of image and text pairings during pre-training and can provide rich semantic information. Combined with SAM's precise segmentation capabilities, the understanding and analysis of image content can be further improved. At the same time, SAM excels at image segmentation at the spatial level, while CLIP has advantages at the semantic level. The combination of the two can achieve a more comprehensive image understanding while considering the spatial structure and semantic content of the image. In addition, CLIP has a strong zero-shot learning ability. Combined with SAM's precise segmentation, it can improve the model's ability to recognize new categories without a large amount of labeled data, that is, improve zero-shot learning capabilities.

[0155] S203: Obtaining regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information.

[0156] The above-mentioned regional monitoring information may include, but is not limited to: the regional type of the target area, and at least one of the element category, location distribution, resource occupation and expected return corresponding to the multiple elements. In the embodiment of the present application, the specific classification of regional types is not limited. For example, agricultural regional types may include: horticulture, nomadism, mixed agriculture, dairy animal husbandry, pasture grazing, commercial grain agriculture, planting agriculture, rice planting and tropical rainforest migration agriculture, etc.

[0157] Taking multiple elements as crops as an example, the element categories, location distribution, resource occupation and expected returns corresponding to the above multiple elements can be understood as: the crop types, planting locations and planting areas, planting costs, and expected income corresponding to the multiple crops in the target area. Among them, the aforementioned location distribution is also the planting location and planting area, the aforementioned resource occupation is also the planting cost, and the aforementioned expected return is also the expected income.

[0158] Based on the above method, the remote sensing data processing system improves the ability to understand remote sensing data by combining the category label set and text description information corresponding to the target area with the sub-remote sensing data corresponding to the pixel coding set (or the target area), thereby improving the accuracy or efficiency of monitoring the target area.

[0159] In an alternative implementation, see Figure 7 As shown, when executing step S203, the remote sensing data processing system may specifically perform the following operations:

[0160] S701: Obtain sub-remote sensing data from initial remote sensing data based on the regional position corresponding to the pixel code set.

[0161] Specifically, when executing step S701, the remote sensing data processing system may obtain sub-remote sensing data corresponding to the region selection operation from the initial remote sensing data according to the region position corresponding to the above pixel code.

[0162] Exemplarily, taking the above-mentioned initial remote sensing data Source image as an example, the remote sensing data processing system can intercept or crop the remote sensing data of the regional position corresponding to the above-mentioned pixel code set from the initial remote sensing data Source image to obtain cropped remote sensing data Cropped image, that is, sub-remote sensing data Regional image.

[0163] S702: Perform image encoding on the sub-remote sensing data to obtain an image vector, and perform word encoding on the category label set and the text description information to obtain word vectors corresponding to the category label set and the text description information.

[0164] Exemplarily, when executing step S702, after the remote sensing data processing system obtains the sub-remote sensing data Regional image (or the cropped remote sensing data Cropped image) from the initial remote sensing data Sourceimage, the cropped remote sensing data Cropped image can be input into the image embedding layer (i.e., Image-embedding) to obtain the corresponding image vector image_vector. Also, the category label set Labels and the text description information Text can be input into the word embedding layer (i.e., Word-embedding) to obtain the word vector word_vector1 corresponding to the category label set Labels and the word vector word_vector2 corresponding to the text description information Text.

[0165] S703: Obtaining region monitoring information based on the image vector and the two obtained word vectors.

[0166] Since the large language model (LLM) in the image-text dialogue model usually refers to a deep learning model that can process and generate long texts, such as a generative pre-trained transformer (GPT) model. The image-text dialogue model combines image processing and natural language processing technologies, can realize the understanding of image content and generate related dialogues, can process long texts, and can better understand the contextual information of the dialogue. In the dialogue, LLM can generate more coherent and fluent replies, can maintain the consistency of the topic; and can generate diverse answers to improve the naturalness and readability of the dialogue. In particular, LLM is usually able to capture long-distance dependencies in the text, which is particularly important when dealing with complex or logically strong dialogues. The image-text dialogue model combines image and text information to more comprehensively understand the user's queries and needs. In a long dialogue, LLM can remember the information of the previous dialogue, making the dialogue more personalized and relevant. Therefore, in an embodiment of the present application, the image vector corresponding to the sub-remote sensing data, and the word vectors corresponding to the category label set and the text description information can be input into the LLM to obtain the regional detection information of the target area.

[0167] Based on the method steps described in steps S701 to S703 above, refer to Figure 8 As shown, the cropped remote sensing data (i.e., the sub-remote sensing data Regional image) can be input into Image-embedding to obtain the corresponding image vector image_vector; and the category label set Labels and the text description information Text are input into Word-embedding to obtain the word vector word_vector1 corresponding to the category label set Labels and the word vector word_vector2 corresponding to the text description information Text. Further, the image vector image_vector, the word vector word_vector1 corresponding to the category label set Labels and the word vector word_vector2 corresponding to the text description information Text are input into the trained LLM (e.g., Finetuned LLM), thereby obtaining the regional monitoring information of the target area, for example, the regional type of the target area, and the feature categories, location distribution, resource occupancy and expected returns corresponding to multiple features.

[0168] Optionally, in order to reduce the complexity of word vector generation, the category label set Labels and the text description information Text can be jointly word-embedded through Word-embedding.

[0169] Based on the above method, the output of SAM and CLIP are combined with the original image (i.e., sub-remote sensing data) and input into LLM, which not only improves the ability to understand remote sensing data, but also interacts well with users, so that a variety of indicators can be automatically generated and the efficiency of result analysis can be improved.

[0170] In an alternative implementation, see Fig. 9 As shown, it is a schematic diagram of the architecture of a remote sensing data processing system provided in an embodiment of the present application. The remote sensing data processing system may include: a data preprocessing module, a data processing module (e.g., a SAM-CLIP module) and a multimodal large model analysis module (e.g., an LLM model) to realize interactive remote sensing data analysis. By calibrating the position of the remote sensing data, the target area of ​​the input remote sensing data is parsed through the data processing module, and Masks and Labels are automatically generated. Finally, the text description information Text of the target area and the sub-remote sensing data are sent to the multimodal large model analysis module for processing, and then output interactively. In other words, the segmentation results of objects in the remote sensing data are generated according to the prompts provided by the user.

[0171] In summary, in the remote sensing data processing method provided in the embodiment of the present application, according to the region selection operation for the initial remote sensing data, the regional image segmentation vector and the regional image text vector of the target area are obtained, and regional monitoring can be performed only for the target area, thereby improving the efficiency of target area monitoring; and, after determining the category label set and pixel coding set of multiple elements in the target area based on the regional image segmentation vector, and determining the text description information of multiple elements based on the regional image text vector, the regional monitoring information of the target area can be obtained based on the sub-remote sensing data corresponding to the pixel coding set, as well as the category label set and the text description information. In this way, through the full combination of image data (for example, the sub-remote sensing data corresponding to the target area) and text information, the understanding and analysis of the image data corresponding to the target area is improved, thereby improving the accuracy of target area monitoring.

[0172] Further, based on the same technical concept, the present application embodiment provides a remote sensing data processing device, which is used to implement the above method flow of the present application embodiment. Fig.10 As shown, the remote sensing data processing device 1000 may include: an acquisition module 1001, a determination module 1002 and a processing module 1003, wherein:

[0173] An acquisition module 1001 is used to obtain a regional image segmentation vector and a regional image text vector of a target region in response to a region selection operation on initial remote sensing data; wherein the regional image segmentation vector represents the position distribution and feature category of each of a plurality of features in the target region, and the regional image text vector represents feature information of each of the plurality of features;

[0174] A determination module 1002 is used to determine a set of category labels and a set of pixel codes of multiple elements based on the regional image segmentation vector, and to determine text description information of multiple elements based on the regional image text vector; wherein different pixel codes in the pixel code set correspond to different elements in the image region;

[0175] The processing module 1003 is used to obtain the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information.

[0176] In an optional embodiment, when acquiring the regional image segmentation vector and the regional image text vector of the target area, the acquisition module 1001 is specifically used to:

[0177] Perform data preprocessing on the initial remote sensing data to obtain target remote sensing data;

[0178] Perform image segmentation encoding and image text encoding on the target remote sensing data respectively, and obtain a scene image segmentation vector and a scene image text vector of the target scene corresponding to the initial remote sensing data;

[0179] Based on the prompt code, scene image segmentation vector and scene image text vector corresponding to the region selection operation, a region image segmentation vector and a region image text vector are obtained; wherein the prompt code is used to indicate a target region in a target scene.

[0180] In an optional embodiment, when performing data preprocessing on the initial remote sensing data to obtain the target remote sensing data, the acquisition module 1001 is specifically used to:

[0181] Obtaining environmental information when collecting initial remote sensing data;

[0182] Based on the data correction methods respectively set for the multiple remote sensing data influencing factors included in the environmental information, the initial remote sensing data is preprocessed to obtain the target remote sensing data.

[0183] In an optional embodiment, when obtaining the region image segmentation vector and the region image text vector based on the prompt code, the scene image segmentation vector and the scene image text vector corresponding to the region selection operation, the acquisition module 1001 is specifically used to:

[0184] Performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain a scene image fusion vector, and obtaining a region image segmentation vector based on the scene image fusion vector and the prompt code;

[0185] And, based on the prompt encoding and the scene image text vector, the region image text vector is obtained.

[0186] In an optional embodiment, when performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain the scene image fusion vector, the acquisition module 1001 is specifically used to:

[0187] Convert the scene image text vector into a text vector with the same number of vector elements as the scene image segmentation vector;

[0188] Feature vector fusion is performed on the scene image segmentation vector and the converted scene image text vector to obtain a scene image fusion vector.

[0189] In an optional embodiment, when obtaining the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information, the processing module 1003 is specifically used to:

[0190] Based on the regional position corresponding to the pixel code set, sub-remote sensing data is obtained from the initial remote sensing data;

[0191] Perform image encoding on the sub-remote sensing data to obtain an image vector, and respectively perform word encoding on the category label set and the text description information to obtain the word vectors corresponding to the category label set and the text description information;

[0192] Based on the image vector and the two word vectors obtained, the region monitoring information is obtained.

[0193] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0194] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0195] An embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute the method according to the embodiment of the present application.

[0196] See also Fig.11 As shown, the structured block diagram of the electronic device 1100 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0197] like Fig.11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 to a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0198] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including: an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. The input unit 1106 can be any type of device that can input information to the electronic device 1100, and the input unit 1106 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 1107 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1108 can include but is not limited to a disk, an optical disk. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a worldwide interoperability for microwave access (WiMax) device, a cellular communication device, and / or the like.

[0199] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above. For example, in some embodiments, the remote sensing data processing method described above may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. In some embodiments, the computing unit 1101 may be configured to perform the remote sensing data processing method described above in any other appropriate manner (e.g., by means of firmware).

[0200] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0201] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0202] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0203] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tub (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0204] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0205] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0206] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present application.

Claims

1. A remote sensing data processing method, characterized in that: include: In response to the region selection operation for the initial remote sensing data, a region image segmentation vector and a region image text vector of the target region are obtained; wherein the region image segmentation vector represents the position distribution and the feature category of each of the multiple elements in the target region, and the region image text vector represents the feature information of each of the multiple elements; Determine a set of category labels and a set of pixel codes of the plurality of elements based on the regional image segmentation vector, and determine text description information of the plurality of elements based on the regional image text vector; wherein different pixel codes in the pixel code set correspond to different elements in the image region; Based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information, the regional monitoring information of the target area is obtained.

2. The method according to claim 1, characterized in that The step of obtaining the region image segmentation vector and the region image text vector of the target region includes: Performing data preprocessing on the initial remote sensing data to obtain target remote sensing data; Performing image segmentation coding and image text coding on the target remote sensing data respectively, to obtain a scene image segmentation vector and a scene image text vector of the target scene corresponding to the initial remote sensing data; Based on the prompt code corresponding to the region selection operation, the scene image segmentation vector and the scene image text vector, the region image segmentation vector and the region image text vector are obtained; wherein the prompt code is used to indicate the target region in the target scene.

3. The method according to claim 2, characterized in that The step of performing data preprocessing on the initial remote sensing data to obtain target remote sensing data includes: Acquiring environmental information when collecting the initial remote sensing data; Based on the data correction methods respectively set for the multiple remote sensing data influencing factors included in the environmental information, the initial remote sensing data is preprocessed to obtain the target remote sensing data.

4. The method according to claim 2, characterized in that The obtaining of the region image segmentation vector and the region image text vector based on the prompt code corresponding to the region selection operation, the scene image segmentation vector and the scene image text vector comprises: Performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain a scene image fusion vector, and obtaining the region image segmentation vector based on the scene image fusion vector and the prompt code; And, based on the prompt code and the scene image text vector, the regional image text vector is obtained.

5. The method according to claim 4, characterized in that The step of performing feature vector fusion on the scene image segmentation vector and the scene image text vector to obtain a scene image fusion vector includes: Converting the scene image text vector into a text vector having the same number of vector elements as the scene image segmentation vector; Feature vector fusion is performed on the scene image segmentation vector and the converted scene image text vector to obtain the scene image fusion vector.

6. The method according to any one of claims 1 to 5, characterized in that The obtaining of the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information includes: Based on the regional position corresponding to the pixel code set, acquiring the sub-remote sensing data from the initial remote sensing data; Performing image encoding on the sub-remote sensing data to obtain an image vector, and performing word encoding on the category label set and the text description information to obtain word vectors corresponding to the category label set and the text description information respectively; The region monitoring information is obtained based on the image vector and the two obtained word vectors.

7. The method according to any one of claims 1 to 5, characterized in that The regional monitoring information includes: the regional type of the target area, and at least one of the element categories, location distribution, resource occupancy and expected returns corresponding to the multiple elements respectively.

8. A remote sensing data processing device, characterized in that: include: An acquisition module, configured to acquire a regional image segmentation vector and a regional image text vector of a target region in response to a region selection operation on the initial remote sensing data; wherein the regional image segmentation vector represents the position distribution and the feature category of each of the multiple elements in the target region, and the regional image text vector represents the feature information of each of the multiple elements; A determination module, used to determine a set of category labels and a set of pixel codes of the plurality of elements based on the regional image segmentation vector, and to determine text description information of the plurality of elements based on the regional image text vector; wherein different pixel codes in the pixel code set correspond to different elements in the image region; A processing module is used to obtain the regional monitoring information of the target area based on the sub-remote sensing data corresponding to the pixel code set, the category label set and the text description information.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images