Land parcel change detection method, related device, equipment and storage medium
By generating and decoding the encoding characteristics of land -based changes and problem sentences, the problem of poor quality of land change test results in existing technology has been solved, and more efficient and accurate land change testing is achieved.
Patent Information
- Application Number
- CN202411212407.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2025-01-28
AI Technical Summary
The prior art has problems of high technical threshold and poor generation quality in the generation of detection results for characterizing plot changes, which require special artificial complex supplementation and correction.
By shooting the first encoding feature based on the target plot at different times, and based on the target problem statement of the target shooting image, the second encoding feature is generated. The characteristics of the two are the same. Answer the sentence.
The target of the change of the land to answer the production effect of the statement, and improve the quality of the test results and the convenience and pertinence of the user.
Smart Images
Figure CN119380207A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of spatial data detection technology, and in particular to a land parcel change detection method and related devices, equipment and storage media. Background Art
[0002] As the demand for environmental resource management continues to increase, images of geospatial data are used to identify changes in land features and understand the impact of human activities on the natural environment.
[0003] In the existing technology, image comparison technology is usually used to obtain the perception results of the captured images, and then professional technicians manually process the perception results to obtain the detection results that characterize the changes in the land. On the one hand, the technical threshold for generating the detection results is high, and on the other hand, complex supplementation and correction of the perception results are required manually, resulting in poor generation quality of the detection results. In view of this, how to improve the generation effect of the target answer sentence that characterizes the changes in the land has become an urgent problem to be solved. Summary of the invention
[0004] The main technical problem solved by the present application is to provide a land parcel change detection method and related devices, equipment and storage media, which can improve the generation effect of target answer sentences that characterize land parcel changes.
[0005] In order to solve the above technical problems, the first aspect of the present application provides a land plot change detection method, including: generating a first coding feature based on target captured images of the target land plot at different times, and generating a second coding feature based on a target question statement about the target captured images; wherein the feature dimensions of the first coding feature and the second coding feature are consistent; decoding is performed based on the first coding feature and the second coding feature to obtain a target answer statement for answering the target question statement.
[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides a land change detection device, including: an encoding module and a decoding module, the encoding module is used to generate a first encoding feature based on target captured images of the target land at different times, and to generate a second encoding feature based on a target question statement about the target captured images; wherein the feature dimensions of the first encoding feature and the second encoding feature are consistent; the decoding module is used to decode based on the first encoding feature and the second encoding feature to obtain a target answer statement for answering the target question statement.
[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, the memory storing program instructions, and the processor being used to execute the program instructions to implement the land change detection method in the above first aspect.
[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the land change detection method described in the first aspect above.
[0009] The above scheme generates a first coding feature based on the target captured images of the target plot at different times, and generates a second coding feature based on the target question sentence about the target captured images, and the feature dimensions of the first coding feature and the second coding feature are consistent, and decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information to improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection of the target plot, and the generated target answer sentence can facilitate the user to understand the changes in the plot as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the plot can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 It is a flowchart of an embodiment of a method for detecting land parcel changes in the present application;
[0011] FIG. 2( a ) is a schematic diagram of an embodiment of a target captured image of a land parcel change detection method of the present application;
[0012] FIG2( b ) is a schematic diagram of an embodiment of the output result of the land parcel change detection method of the present application;
[0013] Figure 3 It is a schematic diagram of the framework of an embodiment of a land parcel change detection model of the present application;
[0014] Figure 4 It is a schematic diagram of the framework of an embodiment of a land parcel change detection device of the present application;
[0015] Figure 5 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;
[0016] Figure 6 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION
[0017] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0018] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0019] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.
[0020] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of a land parcel change detection method of the present application. Specifically, it may include the following steps:
[0021] Step S10: Generate a first coding feature based on target photographed images of the target land parcel at different times, and generate a second coding feature based on a target question statement about the target photographed images.
[0022] In the embodiments of the present disclosure, there is no limitation on the method of acquiring the target captured images of the target land parcel at different times. For example, the target captured images can be acquired based on capturing by an imaging device, or based on detection by an electromagnetic wave sensor, etc. Specifically, as a possible implementation method, the target captured images are remote sensing images, which are acquired by sensing the electromagnetic waves, visible light, infrared rays, etc. reflected or radiated by the target land parcel from a long distance through remote sensing platforms such as aircraft and satellites, so as to acquire information on the earth's surface.
[0023] It should be noted that the number of target images of the target plot at different times is not limited in the present application, for example, two, three, etc., that is, the target images of the target plot are obtained at least at two times.
[0024] In one implementation scenario, the target question statement about the target captured image is a natural language text. The natural language text is based on the natural, informal language system used by humans and can be used for communication, expression of opinions, and transmission of information. It should be noted that the language type of the target question statement is not limited in this application, such as Chinese, English, French, etc.
[0025] In a specific implementation scenario, the input type of the target question sentence includes but is not limited to text, audio, image, etc., and examples are not given one by one here.
[0026] In a specific implementation scenario, the target question statement is used to ask questions about the regional changes of the target plot. For example, if there are two target images, the target question statement is "Please tell me which areas of the two images have changed, and what are the corresponding category attributes of these areas before and after the changes", or, if there are multiple target images, the target question statement is "Please tell me the changing trend of the size of the lake area based on the images, and which areas have changed in the two images closest to the current moment", etc.
[0027] In one implementation scenario, the target images of the target plot at different times can be obtained based on different shooting angles, or can be obtained based on the same shooting angle. For example, the target images of the target plot at different times are dual-phase remote sensing images, that is, they have the same shooting angle. Alternatively, there are multiple target images of the target plot at different times, including images taken at the same angle and images taken at different shooting angles.
[0028] In one implementation scenario, a first coding feature is generated based on target images of the target plot at different times. For example, the pixel characteristics, frequency characteristics, spatial correlation, etc. of the color components of the target captured images are used for encoding. For details, please refer to the technical details of the image coding technology. For the sake of brevity, they will not be repeated here.
[0029] In a specific implementation scenario, an image feature representation of a target captured image is extracted, and a position coding representation characterizing position information of each pixel of the target captured image is obtained, and a first coding feature is obtained based on the image feature representation and the position coding representation of the target captured image.
[0030] In a specific implementation scenario, as a possible implementation method, an image coding model can be pre-trained, and the image coding model can include but is not limited to a convolutional neural network (CNN), a recurrent neural network (RNN), a network model of a Transformer architecture, etc., and the target captured image is input into the image coding model, and the output result of the image coding model is used as the first coding feature of the target captured image. In order to ensure the generation accuracy of the image coding model as much as possible, sample captured images of the sample plot at different times can be collected, and the sample real features can be annotated on the sample captured images. On this basis, the sample captured images can be processed based on the image coding model to obtain the sample predicted features of the sample captured images, so that the network parameters of the image coding model can be adjusted based on the difference between the sample real features and the sample predicted features until the image coding model training converges, and the target captured image can be processed based on the image coding model with converged training to obtain the first coding feature. It should be noted that the specific processing process of the image coding model can refer to the technical details such as convolutional neural network (CNN), recurrent neural network (RNN), network model of Transformer architecture, etc., which will not be repeated here. The above method uses the trained image coding model to generate the first coding feature, so the generation efficiency and generation accuracy of the first coding feature can be improved.
[0031] In a specific implementation scenario, when the target captured images of the target plot at different times include at least a first captured image and a second captured image, the first image feature of the first captured image is extracted, and the second image feature of the second captured image is extracted. Specifically, the extraction can be based on the convolution layer in the convolutional neural network, which is not limited in the present application. Since the first image feature and the second image feature have the same feature dimension, the first image feature and the second image feature are spliced to obtain a spliced image feature, that is, the spliced image feature has the feature representation of the first captured image and the second captured image. Exemplarily, the first image feature and the second image feature can be spliced in the channel dimension to obtain a spliced image feature. On this basis, the spliced image feature can be segmented based on the spatial position of the target plot represented by the first captured image and the spatial position of the target plot represented by the second captured image, and the segmented image feature of the same spatial position area represented by the first captured image and the second captured image is obtained. In other words, for the segmented image feature, it contains image feature information of the first captured image and the second captured image in the same spatial position area in the target plot. For example, both the first captured image and the second captured image capture area a and area b, the segmented image feature a includes features representing area a in the first image features and features representing area a in the second image features, and the segmented image feature b includes features representing area b in the first image features and features representing area b in the second image features. It should be noted that the above embodiment is only a possible implementation method for generating segmented image features, and the specific segmentation method is not limited in this application. Based on the segmented image features, the image feature representation is extracted, and the image feature representation can characterize the changes in the land area between the first captured image and the second captured image.
[0032] In one implementation scenario, a second encoding feature is generated based on a target question statement about the target captured image. For example, the target question statement is converted into binary digits according to certain rules. For details, please refer to the technical details of the text encoding technology. For the sake of brevity, they will not be repeated here.
[0033] In a specific implementation scenario, the second encoding feature is generated based on a word segmenter technology, including rule-based, statistical, machine learning-based, deep learning-based, hybrid method-based, and pre-trained model-based word segmenters. The main function of these word segmenters is to accurately encode the input sentence and convert it into a second encoding feature.
[0034] In one implementation scenario, the feature dimensions of the first coding feature and the second coding feature are consistent. Specifically, the number of network channels used to generate the first coding feature is consistent with the number of network channels used to generate the second coding feature. It should be noted that the above embodiment is only one possible implementation method, and the method of maintaining the consistency of the feature dimensions of the first coding feature and the second coding feature is not limited in this application.
[0035] Step S20: Decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence.
[0036] In one implementation scenario, decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information, which can improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection of the target plot, and the generated target answer sentence can facilitate the user to understand the changes in the plot as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the plot can be improved.
[0037] In one implementation scenario, the decoding method includes but is not limited to decoding strategies such as greedy search and beam search. For details, please refer to the technical details of the decoding technology, etc., which will not be repeated here.
[0038] In an implementation scenario, based on the first coding feature and the second coding feature, a fused coding feature is obtained, that is, the fused coding feature simultaneously contains the image information of the target captured image and the text information of the target question sentence, and decoding is performed based on the fused coding feature to obtain a decoded character sequence, and based on the decoded character sequence, a target answer sentence is generated.
[0039] In a specific implementation scenario, since the first coding feature and the second coding feature have consistent feature dimensions, after the first coding feature and the second coding feature are encoded and aligned, a splicing operation is performed, and the coding feature obtained after splicing is used as a fused coding feature to improve decoding efficiency.
[0040] In a specific implementation scenario, a paradigm is set for generating a character sequence. Based on the set paradigm, the fused coding features can be decoded autoregressively to obtain a character sequence for describing fine-grained changes in the target captured image. The character sequence is converted into a target answer sentence that conforms to natural language logic, such as "the area with coordinates [x1, y1, x2, y2, ..., xn, yn] in the figure has changed. The original [land] in the area has become a [pond]. In addition, another area [x11, y11, x21, y21, ..., xn1, yn1] has also changed. The area was originally a piece of [land], but now it has become a [building]."
[0041] In a specific implementation scenario, the decoded character sequence and the target answer sentence can be generated based on a large language model. The large language model can understand human natural language input, and obtain keyword information about the input text and the logical relationship between each element based on the input text, and generate semantically relevant output. For specific principles, please refer to more technical details of the large language model, which will not be repeated here. It should be noted that the large language model may include but is not limited to: LLAMA, Bloom and other open source large models, and the network architecture of the large language model is not limited here.
[0042] In a specific implementation scenario, the keywords in the target answer sentence are detected. Taking the aforementioned target answer sentence as an example, "x1, y1, x2, y2,…, xn, yn", "land", "pond", "x11, y11, x21, y21,…, xn1, yn1", "land", and "building" are all keywords in the target answer sentence. They can be displayed using special marks, such as adding brackets, changing font color, etc., which are not limited here.
[0043] In a specific implementation scenario, identifiers are added to the content about the changes in the land area in the target answer sentence. For example, the coordinate identifier is set to "cls" and the area category identifier is set to "area". Taking the aforementioned target answer sentence as an example, the target answer sentence after adding the identifier is "The area with coordinates [cls, x1, y1, x2, y2, ..., xn, yn] in the figure has changed. The original [area, land] in the area has become [area, pond]. In addition, another area [cls, x11, y11, x21, y21, ..., xn1, yn1] has also changed. The area was originally a [area, land], and now it has become a [area, building]."
[0044] Please refer to Figure 2(a) and Figure 2(b) in combination. Figure 2(a) is a schematic diagram of an embodiment of a target captured image of a land change detection method of the present application, and Figure 2(b) is a schematic diagram of an embodiment of an output result of a land change detection method of the present application. In an implementation scenario, the target captured image is shown in 2(a), and the target question statement is "Please tell me which areas of the two images have changed, and what are the corresponding category attributes of these areas before and after the change", and the generated target answer statement is "The area with coordinates [cls, x1, y1, x2, y2, ..., xn, yn] in the figure has changed, and the original [area, land] in the area has become [area, pond]. In addition, another area [cls, x11, y11, x21, y21, ..., xn1, yn1] has also changed. The area was originally a piece of [area, land] and now has become a piece of [area, building]", that is, the target answer statement at least includes the target area where the target plot changes at different times. After decoding based on the first coding feature and the second coding feature to obtain the target answer statement for answering the target question statement, a target prompt image is generated based on the target captured image and the target area, and the target area is highlighted in the target prompt image, as shown in Figure 2(b), and the target prompt image is output. It should be noted that the highlighting method of the target area is not limited in this application. The above scheme answers the target question sentence in a text-to-illustration manner, improves the visibility of the target plot area change, and thus can improve the generation effect of the target answer sentence representing the plot change situation.
[0045] In a specific implementation scenario, the target answer statement also includes the change type of the target area, and the target prompt image marks the change areas of different change types in different styles. For example, the area type before the change is "land" in the change area is marked in yellow, the area type after the change is "pond" is marked in blue, or only the area after the change is marked.
[0046] In a specific implementation scenario, region change data including the coordinate position and change type of each change region is generated based on the target answer sentence. Specifically, the region change data is structured data, for example:
[0047] "Variation area: x1, y1, x2, y2, ..., xn, yn
[0048] Area Type: Pond
[0049] Variation area: x11, y11, x21, y21, …, xn1, yn1
[0050] Area Type: Building
[0051] The region change data is output together with the target cue image.
[0052] In a specific implementation scenario, the region change data may be data in a specific format, such as a table, a key-value pair, etc., which is not limited in this application.
[0053] In a specific implementation scenario, different change types of change areas are marked in different styles in the target prompt image, and the area change data is output together with the target prompt image. For details, please refer to the above embodiment, which will not be repeated here for brevity.
[0054] The above scheme generates a first coding feature based on the target captured images of the target plot at different times, and generates a second coding feature based on the target question sentence about the target captured images, and the feature dimensions of the first coding feature and the second coding feature are consistent, and decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information to improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection of the target plot, and the generated target answer sentence can facilitate the user to understand the changes in the plot as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the plot can be improved.
[0055] See also Figure 3 , Figure 3 3 is a schematic diagram of the framework of an embodiment of the land parcel change detection model 30 of the present application. Figure 3 As shown, the land parcel change detection model 30 at least includes an image encoding network 31, a sentence encoding network 32 and a multimodal decoding network 33, the image encoding network 31 is used to generate a first encoding feature, the sentence encoding network 32 is used to generate a second encoding feature, the multimodal decoding network 33 is used to decode to obtain a target answer sentence, and the land parcel change detection model 30 is trained based on sample data, and the sample data includes: sample captured images of sample plots at different times, sample question sentences about the sample captured images, and sample answer sentences for answering the sample question sentences.
[0056] It should be noted that the image encoding network 31, the sentence encoding network 32 and the multimodal decoding network 33 can be found in the detailed description of the aforementioned embodiments, and will not be described again for the sake of brevity.
[0057] In one implementation scenario, the land parcel change detection model 30 further includes a post-processing network (not shown), which is used to generate a target prompt image based on the target captured image and the target area after decoding based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence, and output the target prompt image. For details, please refer to the aforementioned embodiment, which will not be described here for brevity.
[0058] In an implementation scenario, the training step of the land parcel change detection model 30 includes predicting sample captured images of sample land parcels at different times and sample question statements about the sample captured images based on the land parcel change detection model 30, obtaining predicted answer statements for answering the sample question statements, the sample question statements are annotated with sample answer statements, and based on the differences between the sample answer statements and the predicted answer statements, jointly adjusting the network parameters of the image encoding network 31 and the multimodal decoding network 33, and fixing the network parameters of the sentence encoding network 32.
[0059] In a specific implementation scenario, the difference between the sample answer statement and the predicted answer statement can be calculated based on a loss function, such as cross entropy loss, reinforcement learning loss, etc., which is not limited in this application.
[0060] In a specific implementation scenario, sample captured images are annotated with key-value pairs representing various areas of the sample plots that have changed at different times. Based on the key-value pairs, prompt words are constructed to indicate the generation of a large language model task. The sample question sentences and prompt words are processed based on the large language model to generate sample answer sentences for answering the sample question sentences. For example, the sample captured image is annotated with "Changed area: x1, y1, x2, y2, ..., xn, yn; Area type: pond; Change area: x11, y11, x21, y21, ..., xn1, yn1; Area type: building". It can be understood that "Changed area" is the "key", "x1, y1, x2, y2, ..., xn, yn" and "x11, y11, x21, y21, ..., xn1, yn1" are different "values", and the sample question text is "Please tell me which areas in these two images have become ponds", and the constructed prompt words are "Please answer the input question based on (the specific content of the key-value pair)". Therefore, the sample answer sentence generated by the large language model to answer the sample question sentence may be "The area with coordinates x1, y1, x2, y2, ..., xn, yn changed from land to pond after a period of time". The above scheme, with the help of the sentence understanding ability of the large language model, generates sample answer sentences for training the land change detection model 30, which can improve the detection accuracy of the land change detection model 30 and improve the generation effect of the target answer sentence that represents the land change situation.
[0061] In a specific implementation scenario, sample question statements about sample captured images are selected based on a preset question statement library, which stores several relatively common preset question statements, such as "what changes have occurred in the land area?" and "what are the coordinates of the changed area?".
[0062] In a specific implementation scenario, the preset question sentences in the preset question sentence library are processed based on the large language model. The large language model can rewrite, expand, change the word order, etc. of the preset question sentences, and use the processed sentences as sample question sentences used in the training of the land change detection model 30, thereby improving the richness of the sample data.
[0063] In the above scheme, the land parcel change detection model 30 generates a first coding feature based on the target captured images of the target land parcel at different times, and generates a second coding feature based on the target question sentence about the target captured images, and the feature dimensions of the first coding feature and the second coding feature are consistent, and decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information, which can improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection on the target land parcel, and the generated target answer sentence can facilitate the user to understand the changes in the land parcel as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the land parcel can be improved.
[0064] See also Figure 4 , Figure 4 Schematic diagram of the framework of an embodiment of the land parcel change detection device 40 of the present application. Figure 4 As shown, the land parcel change detection device 40 includes: an encoding module 41 and a decoding module 42, the encoding module 41 is used to generate a first encoding feature based on target captured images of the target land parcel at different times, and to generate a second encoding feature based on a target question statement about the target captured images; wherein the feature dimensions of the first encoding feature and the second encoding feature are consistent; the decoding module 42 is used to decode based on the first encoding feature and the second encoding feature to obtain a target answer statement for answering the target question statement.
[0065] In the above scheme, the land change detection device 40 generates a first coding feature based on the target captured images of the target land at different times, and generates a second coding feature based on the target question sentence about the target captured images, and the feature dimensions of the first coding feature and the second coding feature are consistent, and decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information, which can improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection on the target land, and the generated target answer sentence can facilitate the user to understand the changes in the land as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the land can be improved.
[0066] In some disclosed embodiments, the decoding module 42 also includes a coding fusion module (not shown) for obtaining a fused coding feature based on the first coding feature and the second coding feature; the decoding module 42 also includes a character decoding module (not shown) for decoding based on the fused coding feature to obtain a decoded character sequence; the decoding module 42 also includes a sentence generation module (not shown) for generating a target answer sentence based on the decoded character sequence.
[0067] In some disclosed embodiments, the encoding module 41 also includes a feature extraction module (not shown) for extracting an image feature representation of the target captured image; the encoding module 41 also includes a feature encoding module (not shown) for obtaining a first encoding feature based on the image feature representation and a position encoding representation of the target captured image; wherein the position encoding representation represents position information of each pixel of the target captured image.
[0068] In some disclosed embodiments, the target captured images of the target plot at different times include at least a first captured image and a second captured image, the feature extraction module (not shown) further includes a first extraction module (not shown) for extracting first image features of the first captured image, the feature extraction module (not shown) further includes a second extraction module (not shown) for extracting second image features of the second captured image; the feature extraction module (not shown) further includes a feature stitching module (not shown) for stitching the first image feature representation and the second image feature to obtain a stitched image feature; the feature extraction module (not shown) further includes a feature segmentation module (not shown) for segmenting the stitched image feature based on the spatial position of the target plot represented by the first captured image and the spatial position of the target plot represented by the second captured image to obtain a segmented image feature; the feature extraction module (not shown) further includes an image feature representation module (not shown) for extracting an image feature representation based on the segmented image feature.
[0069] In some disclosed embodiments, the target answer statement includes at least a target area where the target plot changes at different times. After decoding based on the first coding feature and the second coding feature to obtain the target answer statement for answering the target question statement, the plot change detection device 40 also includes a prompt image generation module (not shown) for generating a target prompt image based on the target captured image and the target area; wherein the target prompt image highlights the target area; the plot change detection device 40 also includes a prompt image output module (not shown) for outputting the target prompt image.
[0070] In some disclosed embodiments, the target answer statement also includes the change type of the target area, and different styles mark the change areas of different change types in the target prompt image; and / or, the land change detection device 40 also includes a change data output module (not shown) for outputting area change data together with the target prompt image; wherein the area change data includes the coordinate position and change type of each change area.
[0071] In some disclosed embodiments, the land parcel change detection device 40 also includes a model building module (not shown), and the target answer sentence is predicted by the land parcel change detection model, which includes at least an image encoding network, a sentence encoding network, and a multimodal decoding network. The image encoding network is used to generate a first encoding feature based on target captured images of the target land parcel at different times, and the sentence encoding network is used to generate a second encoding feature based on a target question sentence about the target captured image. The multimodal decoding network is used to obtain the target answer sentence based on the first encoding feature and the second encoding feature, and the land parcel change detection model is trained based on sample captured images of the sample land parcel at different times and sample question sentences about the sample captured images.
[0072] In some disclosed embodiments, the model building module (not shown) also includes a model training module (not shown), which is used to predict sample captured images of sample plots at different times and sample question statements about the sample captured images based on the plot change detection model, and obtain predicted answer statements for answering the sample question statements; wherein the sample question statements are annotated with sample answer statements; based on the difference between the sample answer statements and the predicted answer statements, the network parameters of the image encoding network and the multimodal decoding network are jointly adjusted, and the network parameters of the sentence encoding network are fixed.
[0073] In some disclosed embodiments, the sample captured images are annotated with key-value pairs representing various areas of the sample plot that have changed at different times. The model training module (not shown) also includes a sample answer sentence generation module (not shown) for constructing prompt words for indicating the generation of a large language model task based on the key-value pairs; and processing the sample question sentences and the prompt words based on the large language model to generate sample answer sentences for answering the sample question sentences.
[0074] See also Figure 5 , Figure 5 1 is a schematic diagram of a framework of an embodiment of an electronic device 50 of the present application. The electronic device 50 includes a memory 51 and a processor 52. The memory 51 stores program instructions, and the processor 52 is used to execute the program instructions to implement the steps in any of the above-mentioned land change detection method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. The electronic device 50 may specifically include but is not limited to: a server, a smart phone, a laptop computer, a tablet computer, a self-service machine, etc., which are not limited here.
[0075] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps in any of the above-mentioned land change detection method embodiments. The processor 52 can also be called a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 52 can be implemented by an integrated circuit chip.
[0076] In the above scheme, the electronic device 50 generates a first coding feature based on the target captured images of the target plot at different times, and generates a second coding feature based on the target question sentence about the target captured images, and the feature dimensions of the first coding feature and the second coding feature are consistent, and decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information, which can improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection of the target plot, and the generated target answer sentence can facilitate the user to understand the changes in the plot as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the plot can be improved.
[0077] See also Figure 6 , Figure 6Schematic diagram of a framework of an embodiment of a computer-readable storage medium 60 of the present application. The computer-readable storage medium 60 stores program instructions 61 that can be executed by a processor, and the program instructions 61 are used to implement the steps in any of the above land parcel change detection method embodiments.
[0078] In the above scheme, the computer-readable storage medium 60 generates a first coding feature based on the target captured images of the target plot at different times, and generates a second coding feature based on the target question sentence about the target captured images, and the feature dimensions of the first coding feature and the second coding feature are consistent, and decoding is performed based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence. On the one hand, the target captured image and the target question sentence are used together as multimodal input information, which can improve the generation quality of the target answer sentence. On the other hand, the input of the target question sentence can improve the convenience and pertinence of the user's change detection on the target plot, and the generated target answer sentence can facilitate the user to understand the changes in the plot as much as possible. Therefore, the generation effect of the target answer sentence that characterizes the changes in the plot can be improved.
[0079] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0080] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0081] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0082] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0083] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0084] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0085] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.
Claims
1. A method for detecting land parcel changes, characterized in that: include: Generate a first coding feature based on target photographed images of the target plot at different times, and generate a second coding feature based on a target question statement about the target photographed images; wherein the feature dimensions of the first coding feature and the second coding feature are consistent, and the target question statement is used to inquire about the regional changes of the target plot; Decoding is performed based on the first encoding feature and the second encoding feature to obtain a target answer sentence for answering the target question sentence.
2. The method according to claim 1, characterized in that The decoding based on the first coding feature and the second coding feature to obtain a target answer sentence for answering the target question sentence includes: Based on the first coding feature and the second coding feature, obtaining a fused coding feature; Decoding is performed based on the fused coding feature to obtain a decoded character sequence; Based on the decoded character sequence, the target answer sentence is generated.
3. The method according to claim 1, characterized in that The generating of the first coding feature based on the target captured images of the target land plot at different times includes: Extracting image feature representation of the target captured image; The first coding feature is obtained based on the image feature representation and the position coding representation of the target captured image; wherein the position coding representation represents the position information of each pixel of the target captured image.
4. The method according to claim 3, characterized in that The target captured images of the target land parcel at different times include at least a first captured image and a second captured image, and extracting image feature representations of the target captured images includes: extracting a first image feature of the first captured image, and extracting a second image feature of the second captured image; splicing the first image feature and the second image feature to obtain a spliced image feature; Based on the spatial position of the target plot represented by the first captured image and the spatial position of the target plot represented by the second captured image, segmenting the stitched image features to obtain segmented image features; Based on the segmented image features, the image feature representation is extracted.
5. The method according to claim 1, characterized in that The target answer sentence at least includes the target area where the target plot changes at the different times. After decoding based on the first coding feature and the second coding feature to obtain the target answer sentence for answering the target question sentence, the method further includes: Based on the target captured image and the target area, a target prompt image is generated; wherein the target prompt image highlights the target area; The target prompt image is output.
6. The method according to claim 5, characterized in that The target answer sentence also includes the change type of the target area, and different styles of change areas of different change types are marked in the target prompt image; And / or, the method further includes: outputting region change data together with the target prompt image; wherein the region change data includes the coordinate position and change type of each of the changed regions.
7. The method according to claim 1, characterized in that The target answer sentence is predicted by a land parcel change detection model, and the land parcel change detection model at least includes an image encoding network, a sentence encoding network and a multimodal decoding network, the image encoding network is used to generate the first encoding feature, the sentence encoding network is used to generate the second encoding feature, and the multimodal decoding network is used to decode and obtain the target answer sentence, and the land parcel change detection model is trained based on sample data, and the sample data includes: sample captured images of sample plots at different times, sample question sentences about the sample captured images, and sample answer sentences used to answer the sample question sentences.
8. The method according to claim 7, characterized in that The training steps of the land parcel change detection model include: Based on the land parcel change detection model, sample images of the sample land parcel at different times and sample question statements about the sample images are predicted to obtain a predicted answer statement for answering the sample question statement; wherein the sample question statement is annotated with a sample answer statement; Based on the difference between the sample answer sentence and the predicted answer sentence, the network parameters of the image encoding network and the multimodal decoding network are jointly adjusted, and the network parameters of the sentence encoding network are fixed.
9. The method according to claim 7, characterized in that: The sample shot image is annotated with key-value pairs representing various areas of the sample plot that have changed at different times, and the method for obtaining the sample answer statement includes: Based on the key-value pair, construct a prompt word for indicating the generation of a large language model task; The sample question sentence and the prompt word are processed based on the large language model to generate a sample answer sentence for answering the sample question sentence.
10. A remote sensing land change detection device, characterized in that: include: An encoding module, configured to generate a first encoding feature based on target images of a target land parcel at different times, and to generate a second encoding feature based on a target question statement about the target images; wherein the feature dimensions of the first encoding feature and the second encoding feature are consistent; A decoding module is used to perform decoding based on the first encoding feature and the second encoding feature to obtain a target answer sentence for answering the target question sentence.
11. An electronic device, characterized in that: The method at least comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the land change detection method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the land change detection method described in any one of claims 1 to 9.