Data processing method and device, nonvolatile storage medium and electronic equipment
Through image analysis and semantic analysis of bi-time phase remote sensing images, mask sequence and semantic description information are generated, and combined with the Q&A interactive mode, the problem of long and slow time caused by large amount of data during remote sensing image analysis is solved, and the rapid and efficient Data processing.
Patent Information
- Application Number
- CN202411391168.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-12-20
AI Technical Summary
In the prior art, a large amount of data is required to be processed during remote sensing image analysis, resulting in a long analysis time and slow speed, and high requirements for the generalization ability of the model, requiring a large amount of labeled data and computing resources.
By receiving dual-time phase remote sensing images and query information, image analysis and semantic analysis processing are carried out, mask sequence and semantic description information are generated, and combined with the Q&A interactive mode, it reduces data processing, reduces resource consumption, and improves analysis speed.
A comprehensive analysis of remote sensing data changes is realized, which reduces data processing time, reduces the consumption of machine learning model learning resources, improves data processing speed, and solves the problems of long and slow analysis time.
Smart Images
Figure CN119166846A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of remote sensing change detection, and more specifically, to a data processing method and device, a non-volatile storage medium, and an electronic device. Background Art
[0002] In the related art, remote sensing images are analyzed by time series analysis methods or machine learning methods to determine changes in the area corresponding to the remote sensing images. Among them, the time series analysis method targets multi-phase remote sensing image sequences and detects changes by analyzing the temporal change trends of the objects. However, for a large monitoring area, the amount of time series data is large, the analysis complexity is high, and the accuracy of the remote sensing data is required to be high. The machine learning method automatically learns the characteristics of the changes in the objects by training a neural network model to achieve the purpose of identifying the changes in the area corresponding to the remote sensing image. This method usually requires a large amount of labeled data and computing resources, and has certain requirements on the generalization ability of the model. Therefore, the methods used to analyze remote sensing images in the related art all have the problem of processing a large amount of remote sensing data, and require a lot of manual intervention and post-analysis. Therefore, there are problems of long data processing time and slow speed.
[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0004] The embodiments of the present application provide a data processing method and device, a non-volatile storage medium, and an electronic device to at least solve the technical problem of long analysis time and slow analysis speed caused by the need for a large amount of remote sensing data when analyzing changes in a certain area through remote sensing images in related technologies.
[0005] According to one aspect of an embodiment of the present application, a method for data processing is provided, comprising: receiving input information, wherein the input information comprises: a dual-phase remote sensing image of a target area, query information for querying the target area, the dual-phase remote sensing image comprising a plurality of remote sensing images of the target area acquired at different times; performing image analysis processing and semantic analysis processing on the input information to obtain a mask sequence and semantic description information, wherein the mask sequence and the semantic description information are used to describe the difference between the dual-phase remote sensing images, and the difference between the dual-phase remote sensing images is the different parts in different remote sensing images; generating an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
[0006] Optionally, the semantic description information is obtained by the following method: determining the position information and category information of each object in multiple remote sensing images; determining objects with the same category information and different position information as first target objects; determining multiple pixels corresponding to the first target object in multiple remote sensing images; and generating semantic description information based on the multiple pixels.
[0007] Optionally, semantic description information is generated based on multiple pixels, including: determining the remote sensing image to which each pixel belongs, and determining the generation time corresponding to the remote sensing image as the generation time corresponding to the pixel; classifying pixels corresponding to the same generation time into a set to obtain multiple pixel sets; determining pixel change information based on the multiple pixel sets, and generating semantic description information based on the change information.
[0008] Optionally, the mask sequence is obtained by the following method: extracting spectral information of each remote sensing image, and determining a second target object in multiple remote sensing images based on the spectral information, wherein the second target object is an object with exactly the same spectral information; obtaining multiple position information of the second target object in multiple remote sensing images; performing a spatial transformation operation on the multiple remote sensing images based on the multiple position information, wherein the multiple remote sensing images after the spatial transformation operation are located in the same coordinate system; and generating a mask sequence based on the multiple remote sensing images after the spatial transformation operation.
[0009] Optionally, a mask sequence is generated based on multiple remote sensing images after a spatial transformation operation, including: determining position information corresponding to each pixel in the multiple remote sensing images after the spatial transformation operation; comparing pixel values of pixels with the same position information to obtain a comparison result; generating a first type of mask when the comparison result indicates that the pixel value has changed; generating a second type of mask when the comparison result indicates that the pixel value has not changed; and generating a mask sequence based on multiple first type masks and multiple second type masks.
[0010] Optionally, generating an output corresponding to the query information according to the mask sequence and the semantic description information includes: performing semantic analysis on the query information to obtain an analysis result, wherein the analysis result is used to indicate a query purpose corresponding to the query information; when the analysis result indicates that the query purpose is to display changes in a dual-phase remote sensing image, generating an output corresponding to the query information according to the mask sequence and the semantic description information includes: generating a target image according to the mask sequence, wherein the mask sequence contains masks with different values, and when generating the target image, masks with different values have different corresponding colors.
[0011] Optionally, an output corresponding to the query information is generated according to the mask sequence and the semantic description information, including: when the analysis result indicates that the query purpose is to determine the carbon sequestration amount of the target area, an output corresponding to the query information is generated according to the mask sequence and the semantic description information, including: determining the difference area according to the target image, wherein the difference area is the area where the target area has changed; determining the category of crops in the area corresponding to the difference area according to the semantic description information; determining the carbon sequestration amount according to the attribute information of each type of crop and the difference area, wherein the attribute information includes: carbon absorption rate, water content.
[0012] According to another aspect of an embodiment of the present application, a data processing device is also provided, including: a receiving module, used to receive input information, wherein the input information includes: a dual-phase remote sensing image of a target area, query information for querying the target area, and the dual-phase remote sensing image includes multiple remote sensing images of the target area collected at different times; a processing module, used to perform image analysis processing and semantic analysis processing on the input information to obtain a mask sequence and semantic description information, the mask sequence and semantic description information are used to describe the difference between the dual-phase remote sensing images, and the difference between the dual-phase remote sensing images is the different parts in different remote sensing images; an output module, used to generate an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
[0013] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a computer program is stored, wherein the above-mentioned data processing method is executed by running the computer program on a device where the non-volatile storage medium is located.
[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above-mentioned data processing method through the computer program.
[0015] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, which implement the steps of the above-mentioned data processing method when the computer instructions are executed by a processor.
[0016] In an embodiment of the present application, input information is received, wherein the input information includes: a dual-phase remote sensing image of a target area, query information for querying the target area, the dual-phase remote sensing image includes multiple remote sensing images of the target area collected at different times; image analysis processing and semantic analysis processing are performed on the input information to obtain a mask sequence and semantic description information, the mask sequence and the semantic description information are used to describe the difference between the dual-phase remote sensing images, and the difference between the dual-phase remote sensing images is the different parts in different remote sensing images; output corresponding to the query information is generated according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs, and the output is generated by the result. It combines semantic analysis and processing technology with image analysis and processing technology to compare the pixel-level differences of remote sensing images of the same area at different times, realizes a comprehensive analysis of the changes in remote sensing data, and achieves the purpose of reducing the amount of data processing; at the same time, it supports question-and-answer interaction, and can obtain customized analysis results through simple question-and-answer interaction, thereby achieving the purpose of reducing the resources consumed in the process of machine learning models learning object features, thereby achieving the technical effect of increasing data processing speed and reducing data processing time, and further solving the technical problem of long analysis time and slow analysis speed caused by the need for a large amount of remote sensing data when analyzing changes in a certain area through remote sensing images in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0018] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a data processing method according to an embodiment of the present application;
[0019] Figure 2 is a flowchart of a method for data processing according to an embodiment of the present application;
[0020] Figure 3 is a schematic diagram of a question-and-answer interaction mode according to an embodiment of the present application;
[0021] Figure 4 It is a structural diagram of a data processing device according to an embodiment of the present application. DETAILED DESCRIPTION
[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0024] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:
[0025] Remote sensing image: an image of the earth's surface or other celestial bodies obtained through remote sensing technology.
[0026] Carbon sink: the "increment" of carbon dioxide fixed in a certain area or ecosystem over a certain period of time.
[0027] In the related art, when analyzing the changes of remote sensing images, the image processing method is mainly used to determine the changes of remote sensing images by comparing multi-temporal remote sensing images (remote sensing images of a certain area obtained at multiple time points); or, a large amount of remote sensing data is used to train a neural network model in advance, so that the trained neural network model learns the characteristics of ground object changes and performs accurate change detection. Therefore, there are problems such as large data processing volume, long processing time and slow speed in the whole process. In order to solve this problem, a relevant solution is provided in the embodiment of the present application, which is described in detail below.
[0028] According to an embodiment of the present application, a method embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a data processing method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0030] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0031] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned data processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .
[0034] The present application embodiment provides a data processing method that can be run in the above-mentioned operating environment. Figure 2 is a flowchart of the steps of the data processing method provided in the embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:
[0035] Step S202, receiving input information, wherein the input information includes: a dual-temporal remote sensing image of the target area and query information for querying the target area, and the dual-temporal remote sensing image includes multiple remote sensing images of the target area collected at different times.
[0036] The method provided in the embodiment of the present application supports the interactive mode of question and answer. Therefore, in step S202, the input information received by the device / system / equipment executing the method provided in the embodiment of the present application includes not only remote sensing images, but also query information. Since the method provided in the embodiment of the present application is used to analyze the changes of a certain area at different times, the number of remote sensing images obtained in step S202 is greater than 1. Since the method provided in the embodiment of the present application has the advantage of not needing to process a large amount of data, the remote sensing image obtained in step S202 may only be a remote sensing image of a certain area at two different times (i.e., a dual-phase remote sensing image), and any area to be detected on the surface can be used as a target area; and the query information is information used to query the relevant information of the target area. For example, the query information can be used to query the changes in the target area, or to query other changes caused by the changes in the target area.
[0037] Step S204, performing image analysis and semantic analysis on the input information to obtain a mask sequence and semantic description information, wherein the mask sequence and semantic description information are used to describe the difference between the dual-temporal remote sensing images, and the difference between the dual-temporal remote sensing images is the different parts in different remote sensing images.
[0038] The method provided in the embodiment of the present application integrates semantic analysis processing and image analysis processing, and determines the changes in the remote sensing image by analyzing the remote sensing image (contained in the input information) at the semantic level and the image level, thereby achieving a technical effect of comprehensive analysis; wherein, the image processing analysis is only for the remote sensing image, while the semantic analysis processing can be for both the remote sensing image and the query information. It should be noted that the remote sensing image is in the form of an image, and the query information is in the form of characters. Therefore, the process of semantic analysis processing of the remote sensing image is different from the process of semantic processing and analysis of the query information; the result of image analysis processing of the remote sensing image is a mask sequence, and the result of semantic analysis processing of the remote sensing image is semantic description information; and the information corresponding to the mask sequence and the semantic description information are both used to describe the changes occurring in the target area. When the change is presented in a dual-phase remote sensing image, different parts of the same target area contained in different remote sensing images are generated.
[0039] Optionally, the semantic description information is obtained by the following method: determining the position information and category information of each object in multiple remote sensing images; determining objects with the same category information and different position information as first target objects; determining multiple pixels corresponding to the first target object in multiple remote sensing images; and generating semantic description information based on the multiple pixels.
[0040] As mentioned in step S204, the result of semantic analysis processing of the remote sensing image is semantic description information, wherein the semantic analysis processing can be performed by a neural network / large language model (Large Language Model, LLM) included in the equipment / device / system that executes the method provided by the embodiment of the present application; the process of semantic analysis processing is as follows: first, multiple remote sensing images contained in the dual-phase remote sensing image are analyzed; specifically, when analyzing each remote sensing image, the position information and category information of each object contained in the remote sensing image are obtained, wherein the position information is used to indicate the spatial position of the object in the target area contained in the analyzed remote sensing image (which can be represented by three-dimensional coordinates), and the category information is used to indicate the category to which the object belongs, and is used to describe the object in more detail, wherein the category of the object includes: plants, buildings, etc., and the category of the object can also be more detailed information such as rice, office buildings, etc. Semantic description information is generated based on relevant information of an object whose position has changed in a target area (i.e., a first target object), wherein the object whose position has changed in the target area (i.e., the first target object) can be determined by the category information and position information of each object. Specifically, if the position information of an object of the same category in remote sensing images at different times is different, then this object is the first target object. After the object used to generate the description information is determined (i.e., the first target object), a plurality of pixels constituting the image of the first target object are determined in different remote sensing images respectively, and semantic description information for describing the change of the target area can be obtained by processing the plurality of pixels.
[0041] According to an optional embodiment of the present application, semantic description information is generated based on multiple pixels, including: determining the remote sensing image to which each pixel belongs, and determining the generation time corresponding to the remote sensing image as the generation time corresponding to the pixel; classifying the pixels corresponding to the same generation time into a set to obtain multiple pixel sets; determining pixel change information based on the multiple pixel sets, and generating semantic description information based on the change information.
[0042] As mentioned in the above embodiment, the semantic description information for describing the change of the target area can be obtained by processing the multiple pixels constituting the image of the first target object, wherein the generation time of different remote sensing images is different, and the generation time of the pixels in different remote sensing images is different (the generation time of the remote sensing image is the generation time of the pixels in the remote sensing image); since each remote sensing image contains the first target object, there must be pixels with different generation times among the multiple pixels constituting the first target object; when generating the semantic description information, the pixels with the same generation time are first classified into a set, and the multiple pixels in each set are arranged according to their position information in the image of the first target object; the pixels in different sets are compared to obtain the change information of the same pixel at the same position in the first target object; the change information of the pixel is converted into the form of characters, that is, the semantic description information is obtained; for example, a feature vector is generated according to the change information of the pixel, the feature vector is encoded by an encoder to obtain a coding sequence, and the coding sequence is decoded by a decoder to obtain the semantic description information. For each set of dual-phase remote sensing images, multiple semantic description information can be generated.
[0043] Optionally, the mask sequence is obtained by the following method: extracting spectral information of each remote sensing image, and determining a second target object in multiple remote sensing images based on the spectral information, wherein the second target object is an object with exactly the same spectral information; obtaining multiple position information of the second target object in multiple remote sensing images; performing a spatial transformation operation on the multiple remote sensing images based on the multiple position information, wherein the multiple remote sensing images after the spatial transformation operation are located in the same coordinate system; and generating a mask sequence based on the multiple remote sensing images after the spatial transformation operation.
[0044] As mentioned in step S202, the result of image analysis processing on the remote sensing image is a mask sequence, wherein the image analysis processing can be performed by another neural network included in the equipment / device / system that executes the method provided by the embodiment of the present application; the process of image analysis processing is as follows, first, the image is subjected to standardization processing such as denoising and correction. In the present embodiment, the image correction is achieved based on the spectrum of the image. Specifically, the spectral information in each remote sensing image is extracted, wherein the spectral information is affected by the color, brightness and other information of the image; only the spectral information of completely identical images will be completely identical, therefore, objects that have not changed at all in multiple remote sensing images (i.e., the second target object) can be determined based on the spectral information, or, in order to avoid the influence of sunlight, the average value of the spectrum of an image corresponding to a certain object can be calculated, and when the average values of the two spectra generated by the two remote sensing images are in the same value range, it is also determined that the object is the first target object, wherein the second target object is usually a building. Furthermore, taking the second target object as a reference, a spatial transformation operation is performed on multiple remote sensing images included in the dual-phase remote sensing image according to the position information of the second target object, so that the multiple remote sensing images included in the dual-phase remote sensing image are located in the same coordinate system after the spatial transformation; wherein the spatial transformation operation includes: translation, rotation, projection, etc.; after the spatial transformation operation, multiple position information of the second target object in the multiple remote sensing images are exactly the same; after the remote sensing image is spatially transformed, a mask sequence can be generated based on the remote sensing image.
[0045] According to another optional embodiment of the present application, a mask sequence is generated based on multiple remote sensing images after a spatial transformation operation, including: determining the position information corresponding to each pixel in the multiple remote sensing images after the spatial transformation operation; comparing the pixel values of pixels with the same position information to obtain a comparison result; when the comparison result indicates that the pixel value has changed, generating a first type of mask; when the comparison result indicates that the pixel value has not changed, generating a second type of mask; generating a mask sequence based on multiple first type masks and multiple second type masks.
[0046] The mask sequence changes the target area at the pixel level. In this embodiment, the mask sequence is generated by the following method: for multiple remote sensing images after the spatial transformation operation, the position information of each pixel contained in each remote sensing image is obtained, and the position information is the position of the pixel in the remote sensing image to which it belongs; further, the pixel values of pixels with the same position information are compared; that is, the pixel values of different pixels indicated by the same position in different remote sensing images are compared. If the pixel values of different pixels indicated by the same position in different remote sensing images are the same, the mask at this position is determined as the second type of mask (for example, 0); if the pixel values of different pixels indicated by the same position in different remote sensing images are different, it means that the pixel point has changed, and the mask at this position is determined as the second type of mask (for example, 0). The mask is determined to be a first-class mask (for example, 255). After traversing all pixels in the remote sensing image according to the above method, each pixel position of the remote sensing image has a corresponding mask. In this embodiment, a mask layer (mask layer) is generated according to the mask corresponding to the pixel. Specifically, multiple first-class masks and multiple second-class masks are arranged as a mask layer (mask layer) according to the position information of the mask, wherein the position information of the mask is the same as the position information of the pixel corresponding to the mask, and different masks have different corresponding colors when displayed; in this embodiment, the first-class mask (for example, 255) will be displayed in white, while the second-class mask (for example, 0) will be displayed in black, so only the changed area in the remote sensing image is displayed in the displayed mask layer.
[0047] Through the above method, it is ensured that each pair of dual-phase remote sensing images is accompanied by a mask sequence for describing pixel-level changes, as well as a semantic analysis result of the textual description of the changes. This complementary annotation provides a variety of change interpretations, thereby enhancing the comprehensiveness of the change interpretation model. These data can not only be used to subsequently generate customized answers corresponding to the query information, but also can be used to train a machine learning model that executes the method provided in the embodiment of the present application.
[0048] The neural network, LLM, and machine learning model for executing the method provided by the embodiment of the present application mentioned in the above embodiment can be stored in the system / device / equipment for executing the method provided by the embodiment of the present application. When applied, the neural network / machine learning model / LLM can be loaded into the memory. For example, the raw data of the neural network can be loaded from the non-volatile memory to the volatile memory so that the processor can run the neural network. The raw data of the neural network refers to the unprocessed data, which usually includes the parameters and structural data of the neural network. The structural data can be a calculation relationship based on the parameters, such as the forward propagation calculation relationship between the intermediate layers and the neurons. Specifically, the structural data can include the structure-related code of the neural network, such as the code for executing the related calculations between the intermediate layers and the neurons.
[0049] In one embodiment, an area for loading a neural network can be divided in the memory, which can include a structure data storage area and a parameter storage area. The structure data storage area is used to store structure-related codes, and the parameters referenced by it can point to the address of a specific parameter in the parameter storage area through a pointer. During the training process of the neural network, the parameters need to be updated frequently, and the parameter values in the parameter storage area can be updated.
[0050] Step S206: Generate an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
[0051] Since the method provided in the embodiment of the present application supports the interactive mode of question and answer, when the input information includes a query information field and the query information field has relevant content, in step S206, an output corresponding to the query information can be generated according to the result (mask sequence and semantic description information) obtained by processing the remote sensing image in step S204. It should be noted that the query information field can also be empty. In this case, the input information does not include the query information, but the system / terminal / device that executes the method provided in the embodiment of the present application will also have an output, which is information used to describe the difference between the dual-phase remote sensing images.
[0052] Optionally, generating an output corresponding to the query information according to the mask sequence and the semantic description information includes: performing semantic analysis on the query information to obtain an analysis result, wherein the analysis result is used to indicate a query purpose corresponding to the query information; when the analysis result indicates that the query purpose is to display changes in a dual-phase remote sensing image, generating an output corresponding to the query information according to the mask sequence and the semantic description information includes: generating a target image according to the mask sequence, wherein the mask sequence contains masks with different values, and when generating the target image, masks with different values have different corresponding colors.
[0053] As mentioned in the above embodiment, the method provided in the embodiment of the present application not only performs semantic analysis processing on the remote sensing image, but also performs semantic analysis processing on the query information, wherein the result of the semantic analysis processing on the query information (i.e., the analysis result) is used to indicate the result to be obtained by querying the dual-temporal remote sensing image (i.e., the query purpose), Figure 3 is a schematic diagram of the question-answer interaction mode, such as Figure 3As shown, when the query purpose is to display the changes of the dual-phase remote sensing image (that is, when the query information is "a mask layer that gives the difference in changes between the two images"), the output obtained is a mask layer, which is generated according to the mask sequence; each mask in the mask sequence carries position information, and the position information of each mask is the same as the position information of the corresponding pixel point in the remote sensing image. When generating the target image (that is, the mask layer), each mask in the mask sequence is first arranged according to the position information it carries. Next, since the value corresponding to each mask is different, as mentioned in the above embodiment, different masks will be displayed in different colors. In the embodiment of the present application, the masks in the mask sequence are divided into only two categories: masks for pixel change areas (that is, first-class masks) and masks for pixel unchanged areas (that is, second-class masks); therefore, Figure 3 As shown, the generated target image (ie, the mask layer) contains only black and white, wherein the image in the white area is the area where changes occur in the remote sensing image.
[0054] According to an optional embodiment of the present application, an output corresponding to the query information is generated according to the mask sequence and the semantic description information, including: when the analysis result indicates that the query purpose is to determine the carbon sequestration amount of the target area, an output corresponding to the query information is generated according to the mask sequence and the semantic description information, including: determining the difference area according to the target image, wherein the difference area is the area where the target area has changed; determining the category of crops in the area corresponding to the difference area according to the semantic description information; determining the carbon sequestration amount according to the attribute information of each type of crop and the difference area, wherein the attribute information includes: carbon absorption rate, water content.
[0055] According to another optional embodiment of the present application, the query information can also be used to query other information of the target area, for example, to query the carbon sequestration of the area contained in the remote sensing image (i.e., the target area), or the carbon sequestration of the changing area, or the carbon sequestration of a specific area in the remote sensing image; when the purpose of the query is to query the carbon sequestration, the output obtained is a numerical value representing the carbon sequestration, or a specific process for calculating the carbon sequestration; for example, if the carbon sequestration of the changing area is to be queried, it is first necessary to determine the area of the changing area (i.e., the difference area), and the type of crops grown in the changing area, and further, the carbon sequestration of the changing area is determined based on the attribute information of each type of crop and the area of the changing area (i.e., the difference area), wherein the attribute information of each type of crop mentioned above includes: the carbon coefficient rate of the crop, the moisture content of the crop, and other factors affecting the carbon sequestration. For example, if Figure 3 As shown in the figure, the query purpose is "the carbon sequestration of tobacco fields in the change area of the remote sensing image". When calculating the carbon sequestration, the change mask interpretation module is first used to identify the difference in the tobacco field plot layers in a set of dual-phase remote sensing images, and the difference area A is calculated.j At the same time, the types of crops in the tobacco field are identified, and the carbon sink generated by the crops is calculated based on the crop attribute information. For example, when the crops in the tobacco field are all tobacco, according to the formula Determine the carbon sink C produced by crops crop , where C is the carbon absorption rate of tobacco plants (for example, 0.450), Y is the crop yield, r is the water content of tobacco plants (between 70% and 80%, for example, 75%), and H is the economic coefficient of tobacco (valued at 0.55). Since soil has the characteristic of carbon fixation, when calculating carbon sink, it is also necessary to calculate the soil carbon fixation amount (C soil )C soil =∑A j ×θ j , where θ j is the average soil carbon fixation rate under different land use patterns, θ j The value range of is [-0.035,0.072]; Figure 3 As shown, the calculation process of carbon sink is demonstrated on the interface.
[0056] Through the above steps, an interactive remote sensing change detection method based on a large language model is provided, which can realize efficient and accurate detection of dynamic monitoring of carbon sinks. By combining the understanding ability of the large language model and the spatial information of remote sensing data, it is possible to more accurately identify the differences in vegetation plots at the same location at different time nodes, and by combining the corresponding carbon sink calculation method, the changes in carbon sinks caused by plot changes can be obtained, and a user-friendly interactive interface is provided to achieve real-time detection and personalized demand customization.
[0057] Figure 4 is a structural diagram of a data processing device according to an embodiment of the present application, such as Figure 4 As shown, the data processing device includes: a receiving module 40, which is used to receive input information, wherein the input information includes: a dual-phase remote sensing image of the target area, and query information for querying the target area, and the dual-phase remote sensing image includes multiple remote sensing images of the target area collected at different times; a processing module 42, which is used to perform image analysis processing and semantic analysis processing on the input information to obtain a mask sequence and semantic description information, and the mask sequence and semantic description information are used to describe the difference between the dual-phase remote sensing images, and the difference between the dual-phase remote sensing images is the different parts in different remote sensing images; an output module 44, which is used to generate an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
[0058] It should be noted that Figure 4 The preferred implementation of the illustrated embodiment can be found in Figure 2The relevant description of the illustrated embodiment will not be repeated here.
[0059] An embodiment of the present application further provides a non-volatile storage medium, in which a computer program is stored, wherein the above data processing method is executed by running the computer program on a device where the non-volatile storage medium is located.
[0060] The above-mentioned non-volatile storage medium is used to store a program that performs the following functions: receiving input information, wherein the input information includes: a dual-phase remote sensing image of a target area, query information for querying the target area, and the dual-phase remote sensing image includes multiple remote sensing images of the target area collected at different times; performing image analysis processing and semantic analysis processing on the input information to obtain a mask sequence and semantic description information, and the mask sequence and semantic description information are used to describe the difference between the dual-phase remote sensing images, and the difference between the dual-phase remote sensing images is the different parts in different remote sensing images; generating an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
[0061] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute the above data processing method through the computer program.
[0062] The processor in the above-mentioned electronic device is used to run a program that performs the following functions: receiving input information, wherein the input information includes: a dual-phase remote sensing image of a target area, query information for querying the target area, and the dual-phase remote sensing image includes multiple remote sensing images of the target area collected at different times; performing image analysis processing and semantic analysis processing on the input information to obtain a mask sequence and semantic description information, wherein the mask sequence and semantic description information are used to describe the difference between the dual-phase remote sensing images, and the difference between the dual-phase remote sensing images is the different parts in different remote sensing images; generating an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
[0063] An embodiment of the present application also provides a computer program product, including computer instructions, which implement the steps of the above data processing method when executed by a processor.
[0064] It should be noted that the various modules in the above-mentioned data processing device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.
[0065] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0066] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0067] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0068] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0069] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.
[0071] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for data processing, characterized in that: include: Receive input information, wherein the input information includes: a dual-temporal remote sensing image of a target area and query information for querying the target area, wherein the dual-temporal remote sensing image includes a plurality of remote sensing images of the target area acquired at different times; Performing image analysis and semantic analysis on the input information to obtain a mask sequence and semantic description information, wherein the mask sequence and the semantic description information are used to describe the difference between the dual-temporal remote sensing images, and the difference between the dual-temporal remote sensing images is the different parts in different remote sensing images; An output corresponding to the query information is generated according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
2. The method according to claim 1, characterized in that The semantic description information is obtained by the following method: Determining location information and category information of each object in a plurality of the remote sensing images; Determine the object with the same category information and different position information as a first target object; Determining a plurality of pixels corresponding to the first target object in a plurality of the remote sensing images; The semantic description information is generated according to the plurality of pixels.
3. The method according to claim 2, characterized in that Generating the semantic description information according to the plurality of pixels includes: Determine the remote sensing image to which each pixel belongs, and determine the generation time corresponding to the remote sensing image as the generation time corresponding to the pixel; Classifying pixels corresponding to the same generation time into a set to obtain multiple pixel sets; The pixel change information is determined according to the multiple pixel sets, and the semantic description information is generated according to the change information.
4. The method according to claim 1, characterized in that: The mask sequence is obtained by the following method: Extracting spectral information of each of the remote sensing images, and determining a second target object in the plurality of remote sensing images according to the spectral information, wherein the second target object is an object having completely the same spectral information; Acquire a plurality of position information of the second target object in a plurality of the remote sensing images; Performing a spatial transformation operation on the plurality of remote sensing images according to the plurality of position information, wherein the plurality of remote sensing images after the spatial transformation operation are located in the same coordinate system; The mask sequence is generated according to the plurality of remote sensing images after the spatial transformation operation is performed.
5. The method according to claim 4, characterized in that Generating the mask sequence according to the plurality of remote sensing images after the spatial transformation operation, comprising: Determine the position information corresponding to each pixel in the plurality of remote sensing images after the spatial transformation operation is performed; Comparing the pixel values of the pixels with the same position information to obtain a comparison result; In a case where the comparison result indicates that the pixel value has changed, generating a first type of mask; If the comparison result indicates that the pixel value has not changed, generating a second type of mask; The mask sequence is generated according to a plurality of the first-type masks and a plurality of the second-type masks.
6. The method according to claim 1, characterized in that Generating an output corresponding to the query information according to the mask sequence and the semantic description information, including: Performing the semantic analysis on the query information to obtain an analysis result, wherein the analysis result is used to indicate a query purpose corresponding to the query information; When the analysis result indicates that the query purpose is to display the change of the dual-temporal remote sensing image, generating an output corresponding to the query information according to the mask sequence and the semantic description information includes: A target image is generated according to the mask sequence, wherein the mask sequence includes masks with different values. When generating the target image, the masks with different values correspond to different colors.
7. The method according to claim 6, characterized in that Generating an output corresponding to the query information according to the mask sequence and the semantic description information, including: When the analysis result indicates that the query purpose is to determine the carbon sequestration amount of the target area, generating an output corresponding to the query information according to the mask sequence and the semantic description information includes: Determining a difference area according to the target image, wherein the difference area is an area of the target region that has changed; Determining the category of crops in the area corresponding to the difference area according to the semantic description information; The carbon sink amount is determined according to the attribute information of each type of the crops and the difference area, wherein the attribute information includes: carbon absorption rate and water content.
8. A data processing device, characterized in that: include: A receiving module, used for receiving input information, wherein the input information includes: a dual-temporal remote sensing image of a target area, and query information for querying the target area, wherein the dual-temporal remote sensing image includes a plurality of remote sensing images of the target area collected at different times; A processing module, used for performing image analysis and semantic analysis on the input information to obtain a mask sequence and semantic description information, wherein the mask sequence and the semantic description information are used to describe the difference between the dual-temporal remote sensing images, and the difference between the dual-temporal remote sensing images is the different parts in the different remote sensing images; An output module is used to generate an output corresponding to the query information according to the mask sequence and the semantic description information, wherein different query information corresponds to different outputs.
9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein the data processing method according to any one of claims 1 to 7 is executed by running the computer program on the device where the non-volatile storage medium is located.
10. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to execute the data processing method according to any one of claims 1 to 7 through the computer program.
11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the data processing method described in any one of claims 1 to 7 are implemented.