Image processing method, storage medium and system

Abstract

The present invention discloses an image processing method, storage medium and system. The method includes: obtaining a first image and a second image within the target area; performing multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; merging the multiple classification results to determine the object change detection result between the first image and the second image. The present invention solves the technical problem of poor detection accuracy of the object change detection model in the related art, which is a processing method in which the object change detection model directly learns the finest-grained single-level classification label.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing method, storage medium and system. Background Art

[0002] In remote sensing technology, the comparison of two remote sensing images is commonly used to monitor changes in land features. With advancements in computing power, deep learning methods have been applied to this two-phase comparison, enabling binary and even multi-class change detection. However, changes between different land features often vary significantly, posing a challenge for deep learning.

[0003] In related technologies, there is currently no effective solution for how to process remote sensing images to improve the accuracy of ground object change detection results. Summary of the Invention

[0004] The embodiments of the present invention provide an image processing method, storage medium and system to at least solve the technical problem in the related art that the ground object change detection model directly learns the finest-grained single-level classification labels, and its detection results have poor accuracy.

[0005] According to one aspect of an embodiment of the present invention, an image processing method is provided, including: acquiring a first image and a second image within a target area; performing multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; and merging the multiple classification results to determine a result of object change detection between the first image and the second image.

[0006] According to another aspect of an embodiment of the present invention, an image processing method is also provided, including: obtaining a first urban planning remote sensing image and a second urban planning remote sensing image within a target area; performing multi-level land object classification on the first urban planning remote sensing image and the second urban planning remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of land object classification in the multi-level land object classification, and the types of land object changes contained in each level of land object classification in the multi-level land object classification increase step by step; merging the multiple classification results to determine the urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0007] According to another aspect of an embodiment of the present invention, an image processing method is also provided to obtain a first natural resource remote sensing image and a second natural resource remote sensing image within a target area; perform multi-level land feature classification on the first natural resource remote sensing image and the second natural resource remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of land feature classification in the multi-level land feature classification, and the types of land feature changes contained in each level of land feature classification in the multi-level land feature classification increase step by step; merge the multiple classification results to determine the natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image.

[0008] According to another aspect of an embodiment of the present invention, an image processing method is also provided, which receives a first image and a second image within a target area from a client; performs multi-level object classification on the first image and the second image to obtain multiple classification results, and merges the multiple classification results to determine a result of object change detection between the first image and the second image, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; and returns the object change detection result to the client.

[0009] According to another aspect of an embodiment of the present invention, a storage medium is further provided. The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned image processing methods.

[0010] According to another aspect of an embodiment of the present invention, an image processing system is also provided, including: a processor; and a memory, connected to the above-mentioned processor, for providing the above-mentioned processor with instructions for processing the following processing steps: obtaining a first image and a second image within the target area; performing multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; merging the multiple classification results to determine the object change detection result between the first image and the second image.

[0011] In an embodiment of the present invention, a first image and a second image within the target area are acquired, and a multi-level feature classification method is used to perform multi-level feature classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of feature classification in the multi-level feature classification, and the types of feature changes contained in each level of feature classification in the multi-level feature classification increase step by step. The multiple classification results are then merged to determine the feature change detection result between the first image and the second image. Thus, the purpose of optimizing the feature change detection model through image-based multi-level feature classification is achieved, thereby achieving the technical effect of improving the accuracy of the feature change detection results, and thus solving the technical problem of poor detection accuracy of the feature change detection model in the related art, which is the processing method of directly learning the finest-grained single-level classification label by the feature change detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0013] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown;

[0014] Figure 2 is a flowchart of an image processing method according to an embodiment of the present invention;

[0015] Figure 3 is a schematic diagram of an optional multi-level classification label system according to an embodiment of the present invention;

[0016] Figure 4 is a schematic diagram of another optional multi-level decoder structure according to an embodiment of the present invention;

[0017] Figure 5 is a flowchart of another image processing method according to an embodiment of the present invention;

[0018] Figure 6 is a schematic diagram of an optional urban planning change detection result according to an embodiment of the present invention;

[0019] Figure 7 is a flowchart of another image processing method according to an embodiment of the present invention;

[0020] Figure 8 is a flowchart of another image processing method according to an embodiment of the present invention;

[0021] Figure 9 is a schematic diagram of performing image processing on a cloud server according to an embodiment of the present invention;

[0022] Figure 10 is a structural block diagram of an image processing device according to an embodiment of the present invention;

[0023] Figure 11 is a structural block diagram of another computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] First, some nouns or terms that appear in the description of the embodiments of the present invention are subject to the following explanations:

[0027] Remote sensing images: refers to films or photographs used to record the electromagnetic wave size of various ground objects. Remote sensing images are mainly divided into aerial photographs and satellite photographs.

[0028] Object classification: A computer-generated statistical method that categorizes pixels of similar brightness in remote sensing images. This is also known as computer-generated automatic recognition.

[0029] Example 1

[0030] According to an embodiment of the present invention, an image processing method embodiment is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] The method embodiment provided in the first embodiment of the present invention may be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0032] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present invention, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0033] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned image processing method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0034] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0035] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0036] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the aforementioned computer device (or mobile device).

[0037] In recent years, deep learning algorithms have been successfully applied to the comparison of two remote sensing images, often focusing on binary classification of changed and unchanged areas. However, this binary change detection approach is not suitable for more complex scenarios and requirements. Therefore, a more detailed multi-classification change detection approach is proposed, which can not only distinguish between changed and unchanged areas, but also further distinguish the changes in specific ground features within the changed areas.

[0038] The changes between different land features often vary greatly. Some land feature changes occur in large quantities and are easy to learn (for example, from landfill to construction), but more land feature changes occur in small quantities and are difficult to learn (for example, from greenhouse to construction). Therefore, for land feature change detection models that apply deep learning, multi-classification change detection models are more difficult to optimize than binary classification change detection models. In addition, in related technologies, multi-classification land feature change detection models that apply deep learning usually directly learn the most fine-grained land feature change labels. The drawback of this method is that the detection results at the finest granularity have low recall and poor accuracy.

[0039] Unlike related art methods, this invention analyzes the finest-grained feature change labels to derive their hierarchical relationships. In a multi-classification feature change detection model using deep learning, a hierarchical cascaded decoder is used to synchronously learn feature change labels at multiple levels, thereby improving the recall and accuracy of feature change detection results.

[0040] Under the above operating environment, the present invention provides Figure 2 An image processing method is shown. Figure 2 is a flow chart of an image processing method according to an embodiment of the present invention. Figure 2 As shown, the image processing method includes:

[0041] Step S202, acquiring a first image and a second image within the target area;

[0042] Step S204: performing multi-level object classification on the first image and the second image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step;

[0043] Step S206 , merging the multiple classification results to determine a ground object change detection result between the first image and the second image.

[0044] In an embodiment of the present invention, a first image and a second image within the target area are acquired, and a multi-level feature classification method is used to perform multi-level feature classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of feature classification in the multi-level feature classification, and the types of feature changes contained in each level of feature classification in the multi-level feature classification increase step by step. The multiple classification results are then merged to determine the feature change detection result between the first image and the second image. Thus, the purpose of optimizing the feature change detection model through image-based multi-level feature classification is achieved, thereby achieving the technical effect of improving the accuracy of the feature change detection results, and thus solving the technical problem of poor detection accuracy of the feature change detection model in the related art, which is the processing method of directly learning the finest-grained single-level classification label by the feature change detection model.

[0045] It should be noted that the embodiments of the present invention can be applicable to, but not limited to, actual application scenarios of land object classification, actual application scenarios of change detection, actual application scenarios of remote sensing image classification, and actual application scenarios of land object recognition. For example, it can also be applied to the following technical fields: meteorological field (for example, cloud extraction, weather forecast, weather warning, etc.); natural resources and ecological environment field (for example, weather forecast, change detection, ecological red line change detection, multi-classification change detection, land object classification, greenhouse extraction, road network extraction, building extraction, building change detection (satellite, drone), etc.); water conservancy field (for example, water area change detection, greenhouse extraction, water body extraction (optical, radar), forest extraction, cage aquaculture extraction, sand mining site extraction, riverside house extraction, dam extraction, photovoltaic power plant extraction, etc.); agriculture and forestry Industry fields (for example, crop extraction (wheat, rice, potatoes, etc.), drone crop identification (corn, flue-cured tobacco, Job's tears rice, etc.), plot identification, growth monitoring (index calculation), agricultural yield estimation, pest and disease monitoring, planting recommendation push, etc.); secondary disaster fields (for example, disaster monitoring, disaster warning, etc.); life services (travel, food delivery, logistics) fields (for example, travel route planning, travel recommendation push, personnel transfer, price adjustment, etc.); urban planning fields (for example, road network extraction (satellite, drone), building extraction, building change detection (satellite, drone), fire protection, etc.).

[0046] The target area may be an area where changes in ground objects are to be detected. The first image may be a remote sensing image of the target area at a first moment, and the second image may be a remote sensing image of the target area at a second moment.

[0047] Each level of feature classification in the multi-level feature classification can include multiple feature variation categories, and the number of feature variation categories included in each level of feature classification in the multi-level feature classification increases gradually. Each level of feature classification in the multi-level feature classification can correspond to a classification result. Performing multi-level feature classification on the first image and the second image can produce multiple classification results.

[0048] The types of feature changes contained in each level of feature classification in the above multi-level feature classification increase step by step. For example, when s types of features are classified into three levels: in the first level of feature classification, the s types of features are divided into changed and unchanged types, that is, the number of feature change types is 2; in the second level of feature classification, based on the s types of features, each type of feature can be classified into new additions and disappearances, that is, the number of feature change types is 2s; in the third level of feature classification, the changes between each of the s types of features can be considered, that is, the number of feature change types is s×(s-1).

[0049] By combining the above-mentioned multiple classification results, the above-mentioned ground object change detection result can be obtained. The ground object change detection result can be the ground object change detection result between the first image and the second image within the above-mentioned target area.

[0050] It should be noted that the more types of feature changes a certain level of feature classification includes, the finer the classification results of the feature classification at that level; conversely, the coarser the classification results of the feature classification at that level. In the above multi-level feature classification, the number of feature change types included in each level of feature classification increases step by step.

[0051] In an optional embodiment, the image processing method further includes the following method steps:

[0052] Step S208: obtaining a first-level classification label, wherein the first-level classification label is used to represent the feature change between every two different types of features in the multiple types of features;

[0053] Step S210: establishing a second-level classification label corresponding to each type of ground feature among the multiple types of ground features based on the first-level classification label, wherein the second-level classification label is used to indicate the ground feature change category corresponding to each type of ground feature;

[0054] Step S212: establishing a third-level classification label corresponding to each type of ground feature in the multiple types of ground features based on the second-level classification label, wherein the third-level classification label is used to indicate whether each type of ground feature in the multiple types of ground features has changed;

[0055] Step S214 , constructing multi-level classification labels corresponding to multi-level feature classification using the first-level classification labels, the second-level classification labels, and the third-level classification labels.

[0056] In the aforementioned optional embodiment, the multiple types of features can be a set of feature types predefined based on the actual feature change detection task. For example, when performing feature change detection in an urban area, the multiple types of features can include buildings, roads, green spaces, water bodies, vehicles, etc. Due to the influence of human production and life as well as natural changes, the multiple types of features often change. The aforementioned first-level classification labels can be used to represent the feature changes between each two features in the multiple types of features.

[0057] Furthermore, the feature change that occurs for each of the multiple feature types may be the addition of a feature of that type or the loss of that feature. Based on the first-level classification labels, a second-level classification label corresponding to each of the multiple feature types can be established. The second-level classification label can be used to indicate the feature change category corresponding to each feature type. The feature change category can include both addition and loss.

[0058] Furthermore, based on the changes that have occurred to each of the multiple types of features, the multiple types of features can be categorized as a whole into two types: changed and unchanged. Based on the second-level classification labels, the third-level classification labels corresponding to the multiple types of features can be established. These third-level classification labels can be used to indicate whether each type of feature in the multiple types of features has changed. That is, the third-level classification labels include two labels: changed and unchanged. Therefore, these third-level classification labels can also be called binary classification labels.

[0059] The multi-level classification labels corresponding to the multi-level feature classification can be constructed using the first-level classification labels, the second-level classification labels, and the third-level classification labels. The multi-level classification labels can include the classification labels of the three levels and the corresponding relationships between the classification labels of the three levels. In the multi-level classification labels, each level can include multiple classification labels, and the more classification labels included, the finer the classification criteria for feature classification at that level.

[0060] For example, when performing remote sensing feature change detection on a certain area, Area 1, the method provided in this embodiment can be used. Feature types in Area 1 are counted in advance to determine a feature type set H. For example, feature type set H may include green space, buildings, water areas, groundbreaking, hardened vacant land, and buildings under construction. The number of feature types included in feature type set H is denoted as h.

[0061] The first-level classification label Y0 of area Area1 can include variations between any two features from the h features in the feature type set H. For example, the first-level classification label Y0 can include "green space - building," "building - green space," "green space - water area," "water area - land," "green space - groundbreaking," "groundbreaking - green space," and so on. That is, the first-level classification label Y0 of area Area1 can include n = h × (h - 1) variations.

[0062] However, in real-world applications, only a few changes in the first-level classification labels Y0 are prevalent in Area1, while other changes rarely or not at all occur. For example, changes such as "green space to building" and "building to hardened open space" are prevalent in Area1, while changes such as "building to water area" and "water area to hardened open space" rarely occur in Area1. As a result, the various changes in the first-level classification labels Y0 are fragmented, disorganized, and extremely unbalanced. This makes it difficult for the ground feature change detection model to learn from changes with little or no data, thus affecting the accuracy of the detection results.

[0063] Figure 3is a schematic diagram of an optional multi-level classification label system according to an embodiment of the present invention, such as Figure 3 As shown, according to the method provided in this embodiment, a derivation operation can be performed based on the first-level classification label Y0 in area Area1 to obtain three levels of classification labels from coarse to fine (equivalent to the above-mentioned multi-level classification labels).

[0064] Specifically, the n variations in the first-level classification labels Y0 can be used as classification labels at the finest level, which are recorded as N classifications. That is, N classifications include n=h×(h-1) classification labels.

[0065] Specifically, based on whether each feature in the feature set H is newly added or removed, the second level of multi-level classification can be performed, denoted as M classification. That is, M classification includes m = h × 2 classification labels, which are labels for newly added and removed features of h types, respectively.

[0066] Specifically, based on whether the h types of features within Area 1 have changed, we can perform the third level of multi-level classification, the coarsest level, which is called binary classification. In this binary classification, Area 1 is divided into two categories: a changed area and an unchanged area, i.e., only two classification labels are included.

[0067] By constructing multi-level classification labels corresponding to multi-level feature classification, when training the decoder in the feature change detection model, multiple decoding layers in the decoder can learn from the classification labels of one level respectively, thereby obtaining a multi-level decoder, optimizing the feature change detection model, and thus improving the accuracy of the feature change detection results.

[0068] In an optional embodiment, in step S204, multi-level object classification is performed on the first image and the second image to obtain multiple classification results, including the following method steps:

[0069] Step S241, extracting features from the first image and the second image using an encoder to obtain features to be classified;

[0070] Step S242, obtain the target decoder corresponding to the multi-level classification label, and perform multi-level object classification on the features to be classified based on the target decoder to obtain multiple classification results, wherein the target decoder is a multi-level decoder, and the decoder of each level includes: a fusion device and a classifier, the fusion device is used to fuse the features obtained at the current level and the features obtained at each level before the current level, and the classifier is used to obtain the classification result corresponding to the current level. In the above optional embodiment, the above-mentioned features to be classified can be obtained by extracting features from the above-mentioned first image and the second image through the above-mentioned encoder. The first image and the second image can be remote sensing images of the above-mentioned target area, and the features to be classified can be the features of the features corresponding to the target area.

[0071] The target decoder can be obtained by training an initial decoder. Specifically, the initial decoder can be a module for information decoding in a deep learning neural network model, and the initial decoder can include multiple sub-decoders. The target decoder can be obtained by training a sub-decoder in the initial decoder using the classification labels of each level in the multi-level classification labels. The target decoder can be a multi-level decoder, and the number of levels of the target decoder can correspond to the number of levels of the classification labels.

[0072] The decoder of each level in the multi-level decoder can include: a fusion unit and a classifier, wherein the fusion unit is used to fuse the features obtained at the current level and the features obtained at the levels before the current level, and the classifier is used to obtain the classification result corresponding to the current level. Based on the target decoder, multi-level feature classification is performed on the features to be classified, and multiple classification results can be obtained. That is, the input of the target decoder is the features to be classified, and the output of the target decoder is the classification result obtained by the classifier of the decoder of each level in the multiple levels of decoders.

[0073] Optionally, the fuser structure of each decoder in the target decoder (i.e., a multi-level decoder) can be identical, and the classifier structure is determined by the type of ground feature change corresponding to the decoder at that level. Each fuser and classifier can be organically composed of multiple convolutional layers, multiple sampling operations, etc.

[0074] For example, when performing remote sensing ground feature change detection on a certain area, Area 1, the method provided in this embodiment can be used. For the ground feature change detection task in Area 1, a multi-level decoder including three sub-decoders can be used. Training the multi-level decoder can include the following method steps:

[0075] In the first step, the first sub-decoder in the three-layer decoder is trained using the classification labels in the N categories;

[0076] In the second step, the classification labels in the M classification are used to train the second sub-decoder in the three-layer decoder;

[0077] In the third step, the classification labels in the two-class classification are used to train the third sub-decoder in the three-layer decoder.

[0078] Unlike the processing method in related technologies in which the feature change detection model directly learns the most fine-grained single-level classification labels, according to the method provided in this embodiment, the decoder in the feature change detection model uses a multi-level decoder, and the multi-level decoder can simultaneously learn from multiple levels of classification labels, which can optimize the feature change detection model and thereby improve the accuracy of the feature change detection results.

[0079] It should be noted that the method provided in this embodiment produces a relatively small amount of detection results, making it easier to deploy on terminal devices and cloud servers. Furthermore, while meeting various performance indicators, the model employed in this embodiment can detect changes faster than conventional methods.

[0080] In an optional embodiment, the target decoder includes: a first layer decoder, a second layer decoder, and a third layer decoder; the features to be classified include: a first feature to be classified, a second feature to be classified, a third feature to be classified, and a fourth feature to be classified; in step S242, multi-level feature classification is performed on the features to be classified based on the target decoder to obtain multiple classification results, including the following method steps:

[0081] Step S2421: In the fusion unit of the first layer decoder, feature fusion is performed on the first feature to be classified to obtain a second feature to be classified, and in the classifier of the first layer decoder, ground object classification is performed on the second feature to be classified to obtain a first classification result, wherein the first classification result is used to determine areas where changes occur and areas where no changes occur between the first image and the second image;

[0082] Step S2422: In the fusion unit of the second layer decoder, feature fusion is performed on the second feature to be classified to obtain a third feature to be classified, and in the classifier of the second layer decoder, ground object classification is performed on the third feature to be classified to obtain a second classification result, wherein the second classification result is used to determine the ground object change category to which the area where there is a change between the first image and the second image belongs;

[0083] In step S2423, in the fusion unit of the third layer decoder, the third feature to be classified is subjected to feature fusion to obtain a fourth feature to be classified, and in the classifier of the third layer decoder, the fourth feature to be classified is subjected to object classification to obtain a third classification result, wherein the third classification result is used to determine the change category corresponding to the object change category between the first image and the second image. In the above optional embodiment, the target decoder may be a multi-level decoder, which may include: the above-mentioned first layer decoder, the above-mentioned second layer decoder, and the above-mentioned third layer decoder. Based on the target decoder, multi-level object classification is performed on the above-mentioned feature to be classified, and the feature to be classified may be subjected to object classification by the first layer decoder, the second layer decoder, and the third layer decoder. The above-mentioned multiple classification results may include: the above-mentioned first classification result, the above-mentioned second classification result, and the above-mentioned third classification result.

[0084] The above-mentioned features to be classified may include but are not limited to: a first feature to be classified, a second feature to be classified, a third feature to be classified, and a fourth feature to be classified. In the above-mentioned optional embodiment, in the fusion unit of the first layer decoder, the first feature to be classified may be subjected to feature fusion to obtain a second feature to be classified. In the classifier of the first layer decoder, the second feature to be classified may be subjected to object classification to obtain a first classification result. In the fusion unit of the second layer decoder, the second feature to be classified may be subjected to feature fusion to obtain a third feature to be classified. In the classifier of the second layer decoder, the third feature to be classified may be subjected to object classification to obtain a second classification result. In the fusion unit of the third layer decoder, the third feature to be classified may be subjected to feature fusion to obtain a fourth feature to be classified. In the classifier of the third layer decoder, the fourth feature to be classified may be subjected to object classification to obtain a third classification result.

[0085] The first and second images may be remote sensing images of the target area, and the features to be classified may be features of land objects corresponding to the target area. Land object classification of the features to be classified based on the first-layer decoder may yield the first classification result, which is used to determine areas with changes and areas without changes between the first and second images. Land object classification of the features to be classified based on the second-layer decoder may yield the second classification result, which is used to determine the land object change category to which the areas with changes between the first and second images belong. Land object change categories refer to the categories corresponding to the trend of land object morphological changes occurring in the areas where changes occur between different images. For example, a trend of land object morphological change from nonexistence to presence is classified as a new change, while a trend of land object morphological change from presence to nonexistence is classified as a disappearing change. Land object classification of the features to be classified based on the third-layer decoder may yield the third classification result, which is used to determine the change category corresponding to the land object change category between the first and second images.

[0086] For example, when performing remote sensing feature change detection on a certain area Area1, the method provided in this embodiment can be used. Two remote sensing images corresponding to area Area1 are obtained, namely Pt0 (the remote sensing image at time t0, equivalent to the first remote sensing image mentioned above) and PT (the remote sensing image at time T, equivalent to the second remote sensing image mentioned above). By using a preset encoder to extract features from the remote sensing images Pt0 and PT, the feature to be classified Ch1 of area Area1 can be obtained. The feature to be classified includes the feature Ch1t0 of the feature at time t0 and the feature Ch1T of the feature at time T.

[0087] Figure 4 is a schematic diagram of another optional multi-level decoder structure according to an embodiment of the present invention, such as Figure 4As shown, the multi-level decoder 40 includes a sub-decoder 401, a sub-decoder 402, and a sub-decoder 403. Each of the three sub-decoders includes a fusion unit and a classifier. The fusion units of the three sub-decoders have the same structure, but different classifiers, denoted as classifier D, classifier E, and classifier F, respectively. The feature Ch1 to be classified from area Area1 is input into the fusion unit of sub-decoder 401. After fusion, a fused feature Ch2 is obtained. This fused feature Ch2 is transmitted to classifier D and the fusion unit of sub-decoder 402. This fused feature Ch2 is fused by the fusion unit of sub-decoder 402 to obtain a fused feature Ch3. This fused feature Ch3 is transmitted to classifier E and the fusion unit of sub-decoder 403. This fused feature Ch3 is fused by the fusion unit of sub-decoder 403 to obtain a fused feature Ch4. This fused feature Ch4 is transmitted to classifier F. The three classifiers classify the fused features, respectively, and obtain three classification results.

[0088] Specifically, based on the sub-decoder 401 (equivalent to the first-layer decoder described above), the fused feature Ch2 is subjected to ground object classification to obtain a classification result D (equivalent to the first classification result described above). The classification result D is used to determine the area C1 where changes occur and the area C0 where no changes occur between the remote sensing image Pt0 and the remote sensing image PT.

[0089] Specifically, sub-decoder 402 (equivalent to the second-layer decoder) performs feature classification on the fused feature Ch3, resulting in a classification result E (equivalent to the second classification result). This classification result E is used to determine whether the region where changes occur between the remote sensing image Pt0 and the remote sensing image PT is a new change or a missing change. The region covered by the classification operation performed by classifier E is denoted as C3, and the uncovered region is denoted as C2.

[0090] Specifically, based on the sub-decoder 403 (equivalent to the third-layer decoder described above), the fused feature Ch4 is classified to obtain a classification result F (equivalent to the third classification result described above). This classification result F is used to determine the specific change category corresponding to the new or missing changes between the remote sensing image Pt0 and the remote sensing image PT. The covered area of ​​the classification operation performed by the classifier F is denoted as C5, and the uncovered area is denoted as C4.

[0091] It should be noted that the aforementioned fusion units and classifiers are each composed of multiple convolutional layers or sampling operations. The specific structures of classifiers A through F are determined by the object classification in the corresponding sub-decoders. The three classification results obtained by classifiers D, E, and F are respectively learned from the object change detection labels at the corresponding levels. This achieves the effect of simultaneously learning from object change detection labels at different levels within the same object change detection model, thereby improving the accuracy of the object change detection model's detection results.

[0092] To further improve the accuracy of the detection results of the above-mentioned ground feature change detection model, the loss function corresponding to the model at multiple levels (three levels in this example) can be determined through the following method steps:

[0093] The first step is to determine the multiple layers in the change detection model and their loss function weight coefficients. In this example, the three layers are denoted as binary classification, M classification, and N classification, and the corresponding loss function weight coefficients are denoted as w_b, w_m, and w_f respectively.

[0094] The second step is to calculate the cross entropy (CE) loss function for binary classification, M classification and N classification respectively:

[0095] (1) The CE loss function of the two-class classification is calculated by the following formula (1): bCE :

[0096]

[0097] (2) The CE loss function Loss of the M classification is calculated by the following formula (2): mCE :

[0098]

[0099] (3) The CE loss function Loss of N categories is calculated by the following formula (3): nCE :

[0100]

[0101] In the above formulas (1), (2), and (3), t i represents the i-th ground feature change detection label, s i Represents the label t i The corresponding predicted value, m represents the number of ground object change detection labels in M ​​categories, and n represents the number of ground object change detection labels in N categories.

[0102] The third step is to calculate the mutual information-based loss function for M and N classifications respectively:

[0103] (1) The mutual information loss function Loss of the M classification is calculated by the following formula (4): mRMI :

[0104]

[0105] (2) The loss function based on mutual information for N categories is calculated by the following formula (5): nRMI :

[0106]

[0107] In the above formulas (4) and (5), t i represents the i-th ground feature change detection label, s i Represents the label t i The corresponding predicted value, m represents the number of ground object change detection labels in the M category, n represents the number of ground object change detection labels in the N category, and J represents the number of samples in a batch. In addition, the intermediate variables are calculated It can be calculated by the following formula (6) and formula (7):

[0108]

[0109] In the above formulas (6) and (7), Y represents the set of ground feature change detection labels, P represents the set of predicted values ​​corresponding to the ground feature change detection labels, Tr represents the calculation of the rank operation, d represents the calculation of the rank operation correlation coefficient, and S represents the calculation of the intermediate variable.

[0110] The fourth step is to calculate the loss functions of binary classification, M classification and N classification respectively:

[0111] (1) The CE loss function of the two-class classification is regarded as the two-class classification loss function, that is, Loss_b = Loss bCE .

[0112] (2) The M classification loss function Loss is calculated by the following formula (8): m :

[0113] Loss m = w_m_CE×Loss nCE +w_m_RMI×Loss mRMI Formula (8)

[0114] In the above formula (8), w_m_CE represents the weight coefficient of the CE loss function in the M-classification loss function, and w_m_RMI represents the weight coefficient of the mutual information-based loss function in the M-classification loss function.

[0115] (3) The N-classification loss function Loss_f is calculated using the following formula (9):

[0116] Loss f = w_f_CE×Loss nCE +w_f_RMI×Loss nRMI Formula (9)

[0117] In the above formula (9), w_f_CE represents the weight coefficient of the CE loss function in the N-classification loss function, and w_f_RMI represents the weight coefficient of the mutual information-based loss function in the N-classification loss function.

[0118] The fifth step is to calculate the total loss function Loss of the ground feature change detection model using the following formula (10):

[0119] Loss = w_b×Loss_b + w_m × Loss_m + w_f × Loss_f Formula (10)

[0120] The method provided in this embodiment builds a feature change detection model from feature change detection labels at different levels. This model includes a multi-level decoder and a multi-level loss function. This allows the same feature change detection model to simultaneously learn from feature change detection labels at different levels and optimize the model using the multi-level loss function. This solves the technical problem in related art where feature change detection models directly learn from the finest-grained, single-level classification labels, resulting in poor detection accuracy.

[0121] In an optional embodiment, in step S206, merging the multiple classification results to determine the ground object change detection result between the first image and the second image includes one of the following method steps:

[0122] Step S261: determining a first difference result based on the first classification result and the second classification result; merging the first difference result and the second classification result in response to the second classification result satisfying a first preset condition to determine a ground feature change detection result between the first image and the second image; wherein the first preset condition is determined by a predicted value corresponding to the second classification result and the second-level classification label;

[0123] Step S262: determining a second difference result based on the second classification result and the third classification result; responsive to the third classification result satisfying a second preset condition, merging the second difference result and the third classification result to determine a ground feature change detection result between the first image and the second image, wherein the second preset condition is determined by the second classification result, the third classification result, the predicted values ​​corresponding to the second-level classification label, and the third-level classification label;

[0124] Step S263: determine a first difference result based on the first classification result and the second classification result; in response to the second classification result satisfying the first preset condition, merge the first difference result and the second classification result to obtain a merged result; determine a third difference result based on the merged result and the third classification result; in response to the third classification result satisfying the second preset condition, merge the third difference result and the merged result to determine the ground object change detection result between the first image and the second image, wherein the first preset condition is determined by the predicted value corresponding to the second classification result and the second-level classification label, and the second preset condition is determined by the predicted value corresponding to the second classification result, the third classification result, the second-level classification label and the third-level classification label.

[0125] In the above optional embodiment, the plurality of classification results may include: the first classification result, the second classification result, and the third classification result. The first classification result is used to determine areas with changes and areas without changes between the first image and the second image, the second classification result is used to determine the ground feature change category to which the areas with changes between the first image and the second image belong, and the third classification result is used to determine the change category corresponding to the ground feature change category between the first image and the second image.

[0126] Determining whether the second classification result satisfies the first preset condition may include the following steps:

[0127] (1) Based on the second classification result, select some labels from the second-level classification labels;

[0128] (2) Obtain the second largest value among the multiple predicted values ​​corresponding to the above partial labels;

[0129] (3) Determine whether the second largest value is greater than a preset threshold.

[0130] Determining whether the third classification result satisfies the second preset condition may include the following steps:

[0131] (1) According to the second classification result, a first part of labels is selected from the second-level classification labels to obtain a sub-label set of the second-level classification labels;

[0132] (2) According to the third classification result, select the second part of labels from the third level classification labels;

[0133] (3) From the second part of labels, select the label with the second largest prediction value;

[0134] (4) Determine whether the label with the second largest prediction value belongs to the sub-label set of the second-level classification label.

[0135] Combining the multiple classification results to determine the ground object change detection result between the first image and the second image may include one of the following three methods:

[0136] Method A: comprises the following steps.

[0137] The first step is to determine the first difference result based on the first classification result and the second classification result;

[0138] The second step is to determine whether the second classification result meets the first preset condition;

[0139] In the third step, when the second classification result satisfies the first preset condition, the first difference result is combined with the second classification result to obtain a ground feature change detection result between the first image and the second image.

[0140] Method B: comprises the following steps.

[0141] The first step is to determine the second difference result based on the second classification result and the third classification result;

[0142] The second step is to determine whether the third classification result meets the second preset condition;

[0143] In the third step, when the third classification result satisfies the second preset condition, the second difference result is combined with the third classification result to obtain a ground feature change detection result between the first image and the second image.

[0144] Method C: comprises the following steps.

[0145] The first step is to determine the first difference result based on the first classification result and the second classification result;

[0146] The second step is to determine whether the second classification result meets the first preset condition;

[0147] In the third step, when the second classification result satisfies the first preset condition, the first difference result and the second classification result are merged to obtain a merged result;

[0148] Step 4: Based on the above merging result and the above third classification result, determine the above third difference result;

[0149] Step 5: Determine whether the third classification result meets the second preset condition;

[0150] In step 6, when the third classification result satisfies the second preset condition, the merging result is combined with the third difference result to obtain a ground feature change detection result between the first image and the second image.

[0151] For example, when performing remote sensing feature change detection on a certain area Area1, the method provided in this embodiment can be used. Based on the three classification results obtained by the decoder 40, the feature change detection result between the remote sensing image Pt0 (equivalent to the first image) and the remote sensing image PT (equivalent to the second image) can be obtained. The specific method steps can be as follows:

[0152] The first step is to determine the area that belongs to area C1 and does not belong to area C3 (equivalent to the first difference result mentioned above), and record it as area D0, that is, D0 = C1-C3;

[0153] The second step is to set a threshold x1, select the second largest value C3x2 among the predicted values ​​of the ground feature change corresponding to the area C3, and compare the second largest value C3x2 with the threshold x1;

[0154] In the third step, when the second largest value C3x2 is greater than the threshold x1, a partial region is selected from region D0 according to the preset merging strategy, and is recorded as D0_1;

[0155] In the fourth step, the above region D0_1 is merged into region C3 to obtain region C3a (equivalent to the above merging result).

[0156] Step 5: Determine the area that belongs to area C3a and does not belong to area C5 (equivalent to the third difference result mentioned above), and record it as area D1, that is, D1 = C3a-C5;

[0157] Step 6: Obtain the ground feature change detection label set corresponding to region C3a, recorded as Y1 (equivalent to the sub-label set of the second-level classification label above);

[0158] Step 7: Select the second largest value C5x3 among the predicted values ​​of the feature change corresponding to the region C5, determine the feature change detection label tx corresponding to the second largest value C5x3, and determine the subordinate relationship between the feature change detection label tx and the feature change detection label set Y1;

[0159] In the eighth step, when the feature change detection label tx belongs to the feature change detection label set Y1, a partial area is selected from the area D1 according to the preset merging strategy, which is recorded as D1_1;

[0160] In the ninth step, the region D1_1 is merged into the region C5 to obtain the region C5a, and the ground feature change detection result between the remote sensing image Pt0 and the remote sensing image PT is obtained (ie, the ground feature change detection result after the regions C3 and C5 are updated to the regions C3a and C5a).

[0161] Different from determining the ground feature change detection result directly based on the classification result, according to the above method provided by this embodiment, the classification result is processed through a preset merging strategy, which can further improve the accuracy of the ground feature change detection result.

[0162] In an optional embodiment, a graphical user interface is provided by a terminal device, and the content displayed by the graphical user interface at least partially includes a ground object change detection scene. The image processing method further includes the following method steps:

[0163] Step S302: Displaying a plurality of candidate multi-level classification label templates in a graphical user interface, wherein each multi-level classification label template corresponds to a different category of ground feature change detection scenario;

[0164] Step S304 , in response to a first touch operation on the graphical user interface, determining a target multi-level classification label template from a plurality of candidate objects;

[0165] Step S306, in response to the second touch operation acting on the multi-level classification label template, the ground object change detection result between the first image and the second image of the target object is displayed in the graphical user interface. In the above optional embodiment, the user can at least partially obtain the above-mentioned ground object change detection scene through the graphical user interface content displayed by the terminal device. The above-mentioned multiple candidate multi-level classification label templates can be displayed in the graphic user interface, and each of the multiple candidate multi-level classification label templates corresponds to a different category of ground object change detection scene. For example, the mobile terminal can display a multi-level classification label template corresponding to a large area, and the large area can include multiple small areas. Ground object change detection can be performed on each of the multiple small areas, and each small area corresponds to a multi-level classification label template.

[0166] In the optional embodiment, when the first touch operation is received on the graphical user interface, the target multi-level classification label template is determined from the plurality of candidate objects. For example, the user may select a small area for feature change detection from the plurality of small areas by clicking a corresponding area on the touch screen of the mobile terminal, thereby selecting the multi-level classification label template for feature change detection.

[0167] Furthermore, upon receiving the second touch operation for the target multi-level classification label template, the graphical user interface displays the ground feature change detection results for the target object between the first image and the second image. For example, a user can long-press a designated area of ​​the mobile terminal's touch screen to display the ground feature change detection results for a selected small area between the two remote sensing images.

[0168] In particular, the above-mentioned touch operation refers to the user touching the display screen of the above-mentioned terminal device with his fingers and controlling the operation of the terminal device. The touch operation may include single-point touch and multi-point touch, wherein the touch operation of each touch point may include click, long press, hard press, swipe and other actions and their combination actions.

[0169] In an optional embodiment, the image processing method further includes the following method steps:

[0170] Step S308: In response to a first editing operation on the ground feature change detection result, adjusting the display range of the ground feature change detection result to obtain a first adjustment result; and / or, in response to a second editing operation on the ground feature change detection result, adjusting the display color of the ground feature change detection result to obtain a second adjustment result; and / or, in response to a sliding operation on the ground feature change detection result, viewing an image change of the ground feature change detection result between the first image and the second image;

[0171] Step S310: updating the ground feature change detection result based on the first adjustment result and / or the second adjustment result.

[0172] In the above optional embodiment, the above ground feature change detection result can be edited to obtain an adjusted ground feature change detection result, and the ground feature change detection result can be updated. Specifically, at least one of the following three methods can be included:

[0173] First, upon receiving the first edit operation on the feature change detection result, the display range of the feature change detection result is adjusted to obtain the first adjusted result. The feature change detection result is updated to the first adjusted result. For example, the user can enlarge or reduce the display range of the feature change detection result by dragging a zoom level control handle in the graphical user interface.

[0174] Second, upon receiving the second edit operation on the feature change detection result, the display color of the feature change detection result is adjusted to obtain the second adjusted result. The feature change detection result is updated to the second adjusted result. For example, the user can adjust the display color of the feature change detection result by clicking a display color selection pane in a graphical user interface.

[0175] The third method is to check the image change of the ground object change detection result between the first image and the second image when the sliding operation on the ground object change detection result is received.

[0176] According to the method provided by this embodiment, a more accurate ground feature change detection result can be obtained, and an adjustable, editable and real-time updated graphical user interface can be provided to the user based on the ground feature change detection result, which has practical significance in relevant application scenarios.

[0177] Under the above operating environment, the present invention provides Figure 5 An image processing method is shown. Figure 5 is a flow chart of another image processing method according to an embodiment of the present invention. Figure 5 As shown, the image processing method includes:

[0178] Step S502: Acquire a first urban planning remote sensing image and a second urban planning remote sensing image within the target area;

[0179] Step S504: performing multi-level object classification on the first urban planning remote sensing image and the second urban planning remote sensing image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step;

[0180] Step S506 : merging the multiple classification results to determine an urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0181] In an embodiment of the present invention, by acquiring a first urban planning remote sensing image and a second urban planning remote sensing image within the target area, a method of multi-level feature classification is adopted for the first urban planning remote sensing image and the second urban planning remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of feature classification in the multi-level feature classification, and the types of feature changes contained in each level of feature classification in the multi-level feature classification increase step by step, and then the multiple classification results are merged to determine the urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image. Thus, the purpose of optimizing the urban planning change detection model through multi-level feature classification based on urban planning remote sensing images is achieved, thereby achieving the technical effect of improving the accuracy of the urban planning change detection results, and thus solving the technical problem of poor detection accuracy of the feature change detection model in the related art, which is a processing method in which the feature change detection model directly learns the finest-grained single-level classification label.

[0182] It should be noted that the embodiments of the present invention can be applied to, but not limited to, the field of urban planning (for example, road network extraction (satellite, drone), building extraction, building change detection (satellite, drone), fire protection, etc.). For example, it can also be applied to the following technical fields: meteorological field (for example, cloud extraction, weather forecast, weather warning, etc.); natural resources and ecological environment field (for example, weather forecast, change detection, ecological red line change detection, multi-classification change detection, land feature classification, greenhouse extraction, road network extraction, building extraction, building change detection (satellite, drone), etc.); water conservancy field (for example, water area change detection, greenhouse extraction, water body extraction (optical, radar), forest extraction, cage aquaculture extraction, sand mining site extraction, riverside house extraction, dam extraction, photovoltaic power plant extraction, etc.); agriculture and forestry field (for example, crop extraction (wheat, rice, potato, etc.), drone crop identification (corn, flue-cured tobacco, coix rice, etc.), plot identification, growth monitoring (index calculation), agricultural yield estimation, pest and disease monitoring, planting suggestion push, etc.); secondary disaster field (for example, disaster monitoring, disaster warning, etc.); life service (travel, takeout, logistics) field (for example, travel route planning, travel suggestion push, personnel transfer, price adjustment, etc.).

[0183] The target area may be a city planning area where the ground object change is to be detected. The first city planning remote sensing image may be a remote sensing image of the target area at a first moment, and the second city planning remote sensing image may be a remote sensing image of the target area at a second moment.

[0184] Each level of feature classification in the multi-level feature classification can include multiple feature change categories, and the number of feature change categories included in each level of feature classification in the multi-level feature classification increases gradually. Each level of feature classification in the multi-level feature classification can correspond to a classification result. Performing multi-level feature classification on the first urban planning remote sensing image and the second urban planning remote sensing image can produce multiple classification results.

[0185] The types of feature changes included in each level of feature classification in the above multi-level feature classification increase step by step. For example, when s urban planning features are classified into three levels: in the first level of feature classification, the s urban planning features are divided into changed and unchanged types, that is, the number of feature change types is 2; in the second level of feature classification, based on the s urban planning features, each urban planning feature can be classified as added or lost, that is, the number of feature change types is 2s; in the third level of feature classification, the changes between each of the s urban planning features can be considered, that is, the number of feature change types is s×(s-1).

[0186] By merging the above-mentioned multiple classification results, the above-mentioned urban planning change detection result can be obtained. The urban planning change detection result can be the urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image within the above-mentioned target area.

[0187] It should be noted that the more types of feature changes a certain level of feature classification includes, the finer the classification results of the feature classification at that level; conversely, the coarser the classification results of the feature classification at that level. In the above multi-level feature classification, the number of feature change types included in each level of feature classification increases step by step.

[0188] In an optional embodiment, the image processing method further includes the following method steps:

[0189] Step S508: obtaining a first-level classification label, wherein the first-level classification label is used to represent the urban planning changes occurring between every two different urban planning features among the multiple urban planning features;

[0190] Step S510: establishing a second-level classification label corresponding to each of the multiple urban planning features based on the first-level classification label, wherein the second-level classification label is used to indicate the urban planning change category corresponding to each urban planning feature;

[0191] Step S512: establishing a third-level classification label corresponding to each of the multiple urban planning features based on the second-level classification label, wherein the third-level classification label is used to indicate whether each of the multiple urban planning features has changed;

[0192] Step S514 : constructing multi-level classification labels corresponding to multi-level feature classification using the first-level classification labels, the second-level classification labels, and the third-level classification labels.

[0193] In the aforementioned optional embodiment, the various urban planning features may be a set of feature types predefined based on the actual urban planning change detection task. For example, when performing urban planning change detection on an urban area, the various urban planning features may include buildings, roads, green spaces, water bodies, vehicles, etc. Due to the influence of human production and life as well as natural changes, the various urban planning features often change. The aforementioned first-level classification labels may be used to represent the urban planning changes between each pair of features in the aforementioned plurality of urban planning features.

[0194] Furthermore, the urban planning changes that occur to each of the multiple urban planning features may be the addition of the urban planning feature or the loss of the urban planning feature. Based on the first-level classification labels, the second-level classification labels corresponding to each of the multiple urban planning features may be established. The second-level classification labels may be used to indicate the urban planning change category corresponding to each urban planning feature. The urban planning change categories may include addition changes and loss changes.

[0195] Furthermore, based on the changes that have occurred to each of the multiple urban planning features, the multiple urban planning features can be categorized as a whole into two types: changed and unchanged. Based on the second-level classification labels, the third-level classification labels corresponding to the multiple urban planning features can be established. These third-level classification labels can be used to indicate whether each of the multiple urban planning features has changed. Specifically, the third-level classification labels include two labels: changed and unchanged. Therefore, these third-level classification labels can also be referred to as binary classification labels.

[0196] The multi-level classification labels corresponding to the multi-level feature classification can be constructed using the first-level classification labels, the second-level classification labels, and the third-level classification labels. The multi-level classification labels can include the classification labels of the three levels and the corresponding relationships between the classification labels of the three levels. In the multi-level classification labels, each level can include multiple classification labels, and the more classification labels included, the finer the classification criteria for feature classification at that level.

[0197] In an optional embodiment, in step S504, multi-level object classification is performed on the first urban planning remote sensing image and the second urban planning remote sensing image to obtain multiple classification results, including the following method steps:

[0198] Step S541, extracting features from the first urban planning remote sensing image and the second urban planning remote sensing image using an encoder to obtain features to be classified;

[0199] Step S542, obtain the target decoder corresponding to the multi-level classification label, and perform multi-level object classification on the features to be classified based on the target decoder to obtain multiple classification results, wherein the target decoder is a multi-level decoder, and the decoder of each level includes: a fusion device and a classifier, the fusion device is used to fuse the features obtained at the current level and the features obtained at each level before the current level, and the classifier is used to obtain the classification result corresponding to the current level.

[0200] In the above optional embodiment, the encoder performs feature extraction on the first and second urban planning remote sensing images to obtain the features to be classified. The first and second urban planning remote sensing images may be remote sensing images of the target area, and the features to be classified may be features of land objects corresponding to the target area.

[0201] The target decoder can be obtained by training an initial decoder. Specifically, the initial decoder can be a module for information decoding in a deep learning neural network model, and the initial decoder can include multiple sub-decoders. The target decoder can be obtained by training a sub-decoder in the initial decoder using the classification labels of each level in the multi-level classification labels. The target decoder can be a multi-level decoder, and the number of levels of the target decoder can correspond to the number of levels of the classification labels.

[0202] Each decoder in the multi-level decoder can include a fuser and a classifier. The fuser is used to fuse the features acquired at the current level with the features acquired at the previous levels, and the classifier is used to obtain the classification result corresponding to the current level. Based on the target decoder, the multi-level feature classification is performed on the features to be classified, resulting in multiple classification results.

[0203] Optionally, the fuser structure of each decoder in the target decoder (i.e., a multi-level decoder) can be identical, and the classifier structure is determined by the type of ground feature change corresponding to the decoder at that level. Each fuser and classifier can be organically composed of multiple convolutional layers, multiple sampling operations, etc.

[0204] In an optional embodiment, the target decoder includes: a first layer decoder, a second layer decoder, and a third layer decoder; the features to be classified include: a first feature to be classified, a second feature to be classified, a third feature to be classified, and a fourth feature to be classified; in step S542, multi-level feature classification is performed on the features to be classified based on the target decoder to obtain multiple classification results, including the following method steps:

[0205] Step S5421: In the fusion unit of the first layer decoder, feature fusion is performed on the first feature to be classified to obtain a second feature to be classified, and in the classifier of the first layer decoder, ground object classification is performed on the second feature to be classified to obtain a first classification result, wherein the first classification result is used to determine areas where changes occur and areas where no changes occur between the first urban planning remote sensing image and the second urban planning remote sensing image;

[0206] Step S5422: In the fusion unit of the second layer decoder, feature fusion is performed on the second feature to be classified to obtain a third feature to be classified, and in the classifier of the second layer decoder, ground object classification is performed on the third feature to be classified to obtain a second classification result, wherein the second classification result is used to determine the urban planning change category to which the area where there is a change between the first urban planning remote sensing image and the second urban planning remote sensing image belongs;

[0207] Step S5423: In the fusion device of the third layer decoder, the third feature to be classified is subjected to feature fusion to obtain a fourth feature to be classified, and in the classifier of the third layer decoder, the fourth feature to be classified is subjected to ground object classification to obtain a third classification result, wherein the third classification result is used to determine the change category corresponding to the urban planning change category between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0208] In the aforementioned optional embodiment, the target decoder may be a multi-layer decoder, including: the aforementioned first-layer decoder, the aforementioned second-layer decoder, and the aforementioned third-layer decoder. Multi-layer object classification may be performed on the feature to be classified based on the target decoder, and the feature to be classified may be classified using the first-layer decoder, the second-layer decoder, and the third-layer decoder. The plurality of classification results may include: the aforementioned first classification result, the aforementioned second classification result, and the aforementioned third classification result.

[0209] The first urban planning remote sensing image and the second urban planning remote sensing image may be remote sensing images of the target area, and the features to be classified may be features of land objects corresponding to the target area. Based on the first-layer decoder, the features to be classified are subjected to land object classification, and the first classification result may be obtained. The first classification result is used to determine the areas where changes occur between the first urban planning remote sensing image and the second urban planning remote sensing image, and the areas where no changes occur. Based on the second-layer decoder, the features to be classified are subjected to land object classification, and the second classification result may be obtained. The second classification result is used to determine the urban planning change category to which the areas where changes occur between the first urban planning remote sensing image and the second urban planning remote sensing image belong. The land object change category refers to the category corresponding to the land object morphological change trend that occurs in the area where changes occur between different images. For example, the category corresponding to the land object morphological change trend from nothing to something belongs to new changes, or the category corresponding to the land object morphological change trend from something to nothing belongs to disappearing changes. Based on the third-layer decoder, the features to be classified are classified into the third classification result, which is used to determine the change category corresponding to the urban planning change category between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0210] In an optional embodiment, in step S506, merging the multiple classification results to determine the urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image includes at least one of the following method steps:

[0211] Step S561: determining a first difference result based on the first classification result and the second classification result; merging the first difference result and the second classification result in response to the second classification result satisfying a first preset condition to determine a ground feature change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image, wherein the first preset condition is determined by a predicted value corresponding to the second classification result and the second-level classification label;

[0212] Step S562: determining a second difference result based on the second classification result and the third classification result; responsive to the third classification result satisfying a second preset condition, merging the second difference result and the third classification result to determine a ground feature change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image, wherein the second preset condition is determined by the second classification result, the third classification result, the predicted values ​​corresponding to the second-level classification label, and the third-level classification label;

[0213] Step S563: determine a first difference result based on the first classification result and the second classification result; in response to the second classification result satisfying the first preset condition, merge the first difference result and the second classification result to obtain a merged result; determine a third difference result based on the merged result and the third classification result; in response to the third classification result satisfying the second preset condition, merge the third difference result and the merged result to determine the ground feature change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image, wherein the first preset condition is determined by the predicted value corresponding to the second classification result and the second-level classification label, and the second preset condition is determined by the predicted value corresponding to the second classification result, the third classification result, the second-level classification label and the third-level classification label.

[0214] For example, when performing remote sensing urban planning feature change detection on urban planning area Area5, the method provided in this embodiment can be employed. Urban planning feature types are pre-counted for urban planning area Area5 to determine an urban planning feature type set H5. For example, urban planning feature type set H5 may include green spaces, roads, water bodies, buildings, vehicles, and the like. The number of urban planning feature types included in urban planning feature type set H5 is denoted as h5.

[0215] The first-level classification label Y50 for urban planning area Area5 can include variations between any two of the h5 types of urban planning features in urban planning feature type set H5. For example, the first-level classification label Y50 can include "green space - road," "road - green space," "green space - water area," "water area - building," and so on. That is, the first-level classification label Y50 for urban planning area Area5 can include n5 = h5 × (h5 - 1) variations.

[0216] However, in actual application scenarios, only a few changes in the first-level classification labels Y50 are prevalent in urban planning area Area5, while other changes rarely or not at all. For example, changes such as "green space to road" and "green space to building" are prevalent in urban planning area Area5, while changes such as "building to water area" and "water area to vehicle" rarely occur in urban planning area Area5. Therefore, the various changes in the first-level classification labels Y50 are scattered, disorganized, and extremely unbalanced. This makes it difficult for the urban planning feature change detection model to learn from changes with little or no data, thus affecting the accuracy of the detection results.

[0217] Based on the first-level classification label Y50 in the urban planning area Area5, a derivation operation is performed to obtain three levels of classification labels from coarse to fine (equivalent to the above-mentioned multi-level classification labels).

[0218] Specifically, the n5 variations in the first-level classification labels Y50 can be used as classification labels at the finest level, which is recorded as N classifications. That is, N classifications include n5=h5×(h5-1) classification labels.

[0219] Specifically, based on whether each urban planning feature in the urban planning feature type set H5 is newly added or removed, the second level of multi-level classification can be performed, denoted as M classification. That is, M classification includes m5 = h5 × 5 classification labels, representing the labels for newly added and removed h5 types of urban planning features, respectively.

[0220] Specifically, based on whether the h5 types of urban planning features within Area5 have changed, we can perform the third level of multi-level classification, the coarsest level, known as binary classification. In this binary classification, Area5 is divided into two categories: changed areas and unchanged areas, meaning only five classification labels are included.

[0221] For the urban planning feature change detection task in Area 5, a multi-layer decoder including three sub-decoders can be used. Training the multi-layer decoder can include the following method steps:

[0222] In the first step, the first sub-decoder in the three-layer decoder is trained using the classification labels in the N categories;

[0223] In the second step, the classification labels in the M classification are used to train the second sub-decoder in the three-layer decoder;

[0224] In the third step, the classification labels in the two-class classification are used to train the third sub-decoder in the three-layer decoder.

[0225] Obtain two remote sensing urban planning images corresponding to urban planning area Area5: P5t0 (the remote sensing urban planning image at time t0, equivalent to the first remote sensing urban planning image) and P5T (the remote sensing urban planning image at time T, equivalent to the second remote sensing urban planning image). Using a preset encoder, feature extraction is performed on remote sensing urban planning images P5t0 and P5T to obtain unclassified features Ch15 for urban planning area Area5. These unclassified features include urban planning feature features Ch15t0 at time t0 and urban planning feature features Ch15T at time T.

[0226] Inputting the unclassified feature Ch15 of urban planning area Area5 into decoder 40 yields three classification results. Sub-decoder 401 identifies the area C15 that has changed and the area C05 that has not changed between the remote sensing urban planning image P5t0 and the remote sensing urban planning image P5T. The covered area classified by sub-decoder 402 is denoted as C35, and the uncovered area as C25. The covered area classified by sub-decoder 403 is denoted as C55, and the uncovered area as C45.

[0227] Based on the three classification results obtained by the decoder 40, the urban planning feature change detection result between the remote sensing urban planning image P5t0 (equivalent to the first urban planning image) and the remote sensing urban planning image P5T (equivalent to the second urban planning image) can be obtained. The specific method steps can be as follows:

[0228] The first step is to determine the area that belongs to area C15 and does not belong to area C35 (equivalent to the first difference result mentioned above), and record it as area D05, that is, D05 = C15-C35;

[0229] The second step is to set a threshold x15, select the second largest value C35x55 among the predicted values ​​of urban planning features change corresponding to area C35, and compare the second largest value C35x55 with the threshold x15;

[0230] In the third step, when the second largest value C35x55 is greater than the threshold x15, a partial area is selected from the area D05 according to the preset merging strategy, and is recorded as D05_1;

[0231] Step 4: Merge the region D05_1 into region C35 to obtain region C35a (equivalent to the above merged result);

[0232] Step 5: Determine the area that belongs to area C35a and does not belong to area C55 (equivalent to the third difference result mentioned above), and record it as area D15, that is, D15 = C35a - C55;

[0233] Step 6: Obtain the urban planning feature change detection label set corresponding to area C35a, recorded as Y51 (equivalent to the sub-label set of the second-level classification label above);

[0234] Step 7: Select the second largest value C55x35 among the predicted values ​​of urban planning features changes corresponding to region C55, determine the urban planning features change detection label tx5 corresponding to the second largest value C55x35, and determine the subordinate relationship between the urban planning features change detection label tx5 and the urban planning features change detection label set Y51;

[0235] In the eighth step, when the urban planning feature change detection label tx5 belongs to the urban planning feature change detection label set Y51, a partial area is selected from the area D15 according to the preset merging strategy, and is recorded as D15_1;

[0236] In the ninth step, the region D15_1 is merged into the region C55 to obtain the region C55a. The urban planning feature change detection results between the remote sensing urban planning image P5t0 and the remote sensing urban planning image P5T are obtained (i.e., the urban planning feature change detection results after the regions C35 and C55 are updated to the regions C35a and C55a). Figure 6 is a schematic diagram of an optional urban planning change detection result according to an embodiment of the present invention, such as Figure 6 As shown, Figure 6 (1) is the urban planning change detection result directly output by the three classification results. In this case, it is almost impossible to extract the changes in urban roads; Figure 6 (2) is the urban planning change detection result based on a differential merging of the three classification results according to the preset merging strategy. At this time, the changes in urban roads can be extracted more completely; Figure 6 (3) is the result of urban planning change detection based on two differential merging of three classification results according to the preset merging strategy. At this time, it further makes up for Figure 6 The changes in urban roads are not fully extracted in (2).

[0237] Different from determining the urban planning change detection result directly based on the classification result, according to the above method provided by this embodiment, the classification result is processed through a preset merging strategy, which can further improve the accuracy of the urban planning change detection result.

[0238] In an optional embodiment, a graphical user interface is provided by a terminal device, and the content displayed by the graphical user interface at least partially includes an urban planning change detection scene. The image processing method further includes the following method steps:

[0239] Step S514, displaying multiple urban planning areas in the graphical user interface;

[0240] Step S516 , in response to the first touch operation on the graphical user interface, determining a target urban planning area from the plurality of urban planning areas;

[0241] Step S518 , in response to the second touch operation on the target urban planning area, displaying the urban planning change detection result of the target urban planning area between the first urban planning remote sensing image and the second urban planning remote sensing image in the graphical user interface.

[0242] In the above optional embodiment, the user can obtain at least part of the above urban planning change detection scenario through the graphical user interface content displayed on the terminal device. The above multiple urban planning areas can be multiple urban planning areas to be detected. For example, the mobile terminal can display a map of a city, which can include multiple counties, and ground feature change detection can be performed for each of the multiple counties.

[0243] In the optional embodiment, when the first touch operation is received on the graphical user interface, the target urban planning area is determined from the plurality of urban planning areas. For example, the user may select a county for which ground feature change detection is required from a plurality of counties by clicking a corresponding area on the touch screen of the mobile terminal.

[0244] In addition, when the second touch operation of the target urban planning area is received, the urban planning change detection result of the target object between the first urban planning remote sensing image and the second urban planning remote sensing image is displayed on the graphical user interface. For example, the user can long press the designated area of ​​the touch screen of the mobile terminal to display the urban planning change detection result of the selected county between the two remote sensing images.

[0245] In particular, the above-mentioned touch operation refers to the user touching the display screen of the above-mentioned terminal device with his fingers and controlling the operation of the terminal device. The touch operation may include single-point touch and multi-point touch, wherein the touch operation of each touch point may include click, long press, hard press, swipe and other actions and their combination actions.

[0246] Under the above operating environment, the present invention provides Figure 7 An image processing method is shown. Figure 7is a flow chart of another image processing method according to an embodiment of the present invention. Figure 7 As shown, the image processing method includes:

[0247] Step S702: Acquire a first natural resource remote sensing image and a second natural resource remote sensing image within the target area;

[0248] Step S704: performing multi-level object classification on the first natural resource remote sensing image and the second natural resource remote sensing image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step;

[0249] Step S706 : merging the multiple classification results to determine a natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image.

[0250] In an embodiment of the present invention, by acquiring a first natural resource remote sensing image and a second natural resource remote sensing image within the target area, a method of multi-level feature classification is adopted for the first natural resource remote sensing image and the second natural resource remote sensing image, to obtain multiple classification results, wherein the multiple classification results correspond to each level of feature classification in the multi-level feature classification, and the types of feature changes contained in each level of feature classification in the multi-level feature classification increase step by step, and then the multiple classification results are merged to determine the natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image. Thus, the purpose of optimizing the natural resource change detection model through multi-level feature classification based on natural resource remote sensing images is achieved, thereby achieving the technical effect of improving the accuracy of the natural resource change detection results, and thus solving the technical problem of poor detection result accuracy of the feature change detection model in the related art, which is a processing method in which the feature change detection model directly learns the finest-grained single-level classification label.

[0251] It should be noted that the embodiments of the present invention can be applied to, but not limited to, the fields of natural resources and ecological environment (for example, weather forecast, change detection, ecological red line change detection, multi-classification change detection, land feature classification, greenhouse extraction, road network extraction, building extraction, building change detection (satellite, drone), etc.). For example, it can also be applied to the following technical fields: meteorological field (for example, cloud extraction, weather forecast, weather warning, etc.); water conservancy field (for example, water area change detection, greenhouse extraction, water body extraction (optical, radar), forest extraction, cage aquaculture extraction, sand mining site extraction, riverside house extraction, dam extraction, etc.); Photovoltaic power plant extraction, etc.); agriculture and forestry (for example, crop extraction (wheat, rice, potato, etc.), drone crop identification (corn, flue-cured tobacco, Job's tears rice, etc.), plot identification, growth monitoring (index calculation), agricultural yield estimation, pest and disease monitoring, planting recommendation push, etc.); secondary disaster field (for example, disaster monitoring, disaster warning, etc.); life service (travel, food delivery, logistics) field (for example, travel route planning, travel recommendation push, personnel transfer, price adjustment, etc.); urban planning field (for example, road network extraction (satellite, drone), building extraction, building change detection (satellite, drone), fire protection, etc.).

[0252] The target area may be a natural resource area whose ground object changes are to be detected. The first natural resource remote sensing image may be a remote sensing image of the target area at a first moment, and the second natural resource remote sensing image may be a remote sensing image of the target area at a second moment.

[0253] Each level of feature classification in the multi-level feature classification can include multiple feature variation categories, and the number of feature variation categories included in each level of feature classification in the multi-level feature classification increases gradually. Each level of feature classification in the multi-level feature classification can correspond to a classification result. Performing multi-level feature classification on the first natural resource remote sensing image and the second natural resource remote sensing image can produce multiple classification results.

[0254] The types of feature changes contained in each level of feature classification in the above multi-level feature classification increase step by step. For example, when s types of features are classified into three levels: in the first level of feature classification, the s types of features are divided into changed and unchanged types, that is, the number of feature change types is 2; in the second level of feature classification, based on the s types of features, each type of feature can be classified into new additions and disappearances, that is, the number of feature change types is 2s; in the third level of feature classification, the changes between each of the s types of features can be considered, that is, the number of feature change types is s×(s-1).

[0255] By merging the multiple classification results, the natural resource change detection result can be obtained, which can be a natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image within the target area.

[0256] It should be noted that the more types of feature changes a certain level of feature classification includes, the finer the classification results of the feature classification at that level; conversely, the coarser the classification results of the feature classification at that level. In the above multi-level feature classification, the number of feature change types included in each level of feature classification increases step by step.

[0257] Furthermore, the image processing method provided by the present invention can also be applied to remote sensing agricultural feature change detection tasks in agricultural scenarios. For example, when performing remote sensing agricultural feature change detection on agricultural area Area2, the method provided by this embodiment can be employed. Agricultural feature types are pre-counted in agricultural area Area2 to determine an agricultural feature type set H2. For example, agricultural feature type set H2 may include farmland, roads, water bodies, houses, vehicles, and the like. The number of agricultural feature types included in this agricultural feature type set H2 is denoted as h2.

[0258] The first-level classification label Y20 for agricultural area Area2 can include variations between any two of the h2 types of agricultural features in the agricultural feature set H2. For example, the first-level classification label Y20 can include "farmland-road," "road-farmland," "farmland-water," "water-house," and so on. That is, the first-level classification label Y20 for agricultural area Area2 can include n2 = h2 × (h2 - 1) variations.

[0259] However, in real-world applications, only a few changes in the first-level classification labels Y20 are prevalent in the agricultural area Area2, while other changes rarely or not at all occur. For example, changes such as "farmland to road" and "house to farmland" are prevalent in the agricultural area Area2, while changes such as "house to water" and "water to vehicle" rarely occur in the agricultural area Area2. Therefore, the various changes in the first-level classification labels Y20 are fragmented, disorganized, and extremely unbalanced. This makes it difficult for the agricultural feature change detection model to learn from changes with little or no data, thus affecting the accuracy of the detection results.

[0260] Based on the first-level classification label Y20 in the agricultural area Area2, a derivation operation is performed to obtain three levels of classification labels from coarse to fine (equivalent to the above-mentioned multi-level classification labels).

[0261] Specifically, the n2 variations in the first-level classification labels Y20 can be used as classification labels at the finest level, which is recorded as N classifications. That is, N classifications include n2=h2×(h2-1) classification labels.

[0262] Specifically, based on whether each agricultural feature in the agricultural feature set H2 is newly added or reduced, the second level of multi-level classification can be performed, denoted as M classification. That is, M classification includes m2 = h2 × 2 classification labels, representing the newly added and reduced labels of the h2 types of agricultural features.

[0263] Specifically, based on whether the h2 types of agricultural features within Area2 have changed, we can perform the third level of multi-level classification, the coarsest level, which is called binary classification. In this binary classification, Area2 is divided into two categories: changed areas and unchanged areas, meaning only two classification labels are included.

[0264] For the agricultural land feature change detection task in the agricultural area Area2, a multi-layer decoder containing three sub-decoders can be used. Training the multi-layer decoder can include the following method steps:

[0265] In the first step, the first sub-decoder in the three-layer decoder is trained using the classification labels in the N categories;

[0266] In the second step, the classification labels in the M classification are used to train the second sub-decoder in the three-layer decoder;

[0267] In the third step, the classification labels in the two-class classification are used to train the third sub-decoder in the three-layer decoder.

[0268] Two remote sensing images corresponding to agricultural area Area2 are obtained: P2t0 (the remote sensing image at time t0, equivalent to the first remote sensing image) and P2T (the remote sensing image at time T, equivalent to the second remote sensing image). Using a preset encoder, feature extraction is performed on remote sensing images P2t0 and P2T to obtain the unclassified features Ch12 of agricultural area Area2. These unclassified features include the agricultural feature Ch12t0 at time t0 and the agricultural feature Ch12T at time T.

[0269] Inputting the unclassified feature Ch12 of agricultural area Area2 into decoder 40 yields three classification results. Sub-decoder 401 identifies the region C12 where changes occur between remote sensing images P2t0 and P2T, and the region C02 where no changes occur. The covered region classified by sub-decoder 402 is designated C32, while the uncovered region is designated C22. Sub-decoder 403 also classifies the covered region as C52, while the uncovered region is designated C42.

[0270] Based on the three classification results obtained by the decoder 40, the agricultural land feature change detection result between the remote sensing image P2t0 (equivalent to the first image) and the remote sensing image P2T (equivalent to the second image) can be obtained. The specific method steps can be as follows:

[0271] The first step is to determine the area that belongs to area C12 and does not belong to area C32 (equivalent to the first difference result mentioned above), and record it as area D02, that is, D02 = C12-C32;

[0272] The second step is to set a threshold x12, select the second largest value C32x22 among the predicted values ​​of agricultural land feature changes corresponding to region C32, and compare the second largest value C32x22 with the threshold x12;

[0273] In the third step, when the second largest value C32x22 is greater than the threshold x12, a partial area is selected from the area D02 according to the preset merging strategy, and is recorded as D02_1;

[0274] Step 4: Merge the region D02_1 into region C32 to obtain region C32a (equivalent to the above merged result);

[0275] Step 5: Determine the area that belongs to area C32a and does not belong to area C52 (equivalent to the third difference result mentioned above), and record it as area D12, that is, D12 = C32a - C52;

[0276] Step 6: Obtain the agricultural land feature change detection label set corresponding to region C32a, recorded as Y21 (equivalent to the sub-label set of the second-level classification label above);

[0277] Step 7: Select the second largest value C52x32 among the predicted agricultural land feature changes corresponding to region C52, determine the agricultural land feature change detection label tx2 corresponding to the second largest value C52x32, and determine the subordinate relationship between the agricultural land feature change detection label tx2 and the agricultural land feature change detection label set Y21;

[0278] In the eighth step, when the agricultural feature change detection label tx2 belongs to the agricultural feature change detection label set Y21, a partial area is selected from the area D12 according to the preset merging strategy, and is recorded as D12_1;

[0279] In step 9, the region D12_1 is merged into region C52 to obtain region C52a. The agricultural land feature change detection results between the remote sensing images P2t0 and P2T are obtained (i.e., the agricultural land feature change detection results after regions C32 and C52 are updated to regions C32a and C52a).

[0280] Different from determining the agricultural land feature change detection results directly based on the classification results, according to the above method provided in this embodiment, the classification results are processed through a preset merging strategy, which can further improve the accuracy of the agricultural land feature change detection results.

[0281] Furthermore, the image processing method provided by the present invention can also be applied to remote sensing water conservancy feature change detection tasks in water conservancy scenarios. For example, when performing remote sensing water conservancy feature change detection on water conservancy area Area 3, the method provided by this embodiment can be employed. A water conservancy feature type count is performed in advance on water conservancy area Area 3 to determine a water conservancy feature type set H3. For example, water conservancy feature type set H3 may include rivers, bridges, ports, houses, ships, and the like. The number of water conservancy feature types included in this water conservancy feature type set H3 is denoted as h3.

[0282] The first-level classification label Y30 for water conservancy area Area3 can include variations between any two of the h3 types of water conservancy features in the water conservancy feature set H3. For example, the first-level classification label Y30 can include "river-bridge," "bridge-river," "river-port," "port-house," and so on. In other words, the first-level classification label Y30 for water conservancy area Area3 can include n3 = h3 × (h3 - 1) variations.

[0283] However, in real-world applications, only a few changes in the first-level classification labels Y30 are prevalent in the water conservancy area Area3, while other changes rarely or not at all occur. For example, changes such as "river-to-bridge" and "bridge-to-river" are prevalent in the water conservancy area Area3, while changes such as "ship-to-bridge" and "port-to-ship" rarely occur in the water conservancy area Area3. As a result, the various changes in the first-level classification labels Y30 are fragmented, disorganized, and extremely unbalanced. This makes it difficult for the water conservancy feature change detection model to learn from changes with little or no data, thus affecting the accuracy of the detection results.

[0284] Based on the first-level classification label Y30 in the water conservancy area Area3, a derivation operation is performed to obtain three levels of classification labels from coarse to fine (equivalent to the above-mentioned multi-level classification labels).

[0285] Specifically, the n3 variations in the first-level classification labels Y30 can be used as classification labels at the finest level, which is recorded as N classifications. That is, N classifications include n3=h3×(h3-1) classification labels.

[0286] Specifically, based on whether each water feature in the water feature type set H3 is newly added or removed, the second level of multi-level classification can be performed, denoted as M classification. That is, M classification includes m3 = h3 × 3 classification labels, representing the newly added and removed labels of the h3 types of water features.

[0287] Specifically, based on whether the h3 types of water conservancy features within Area 3 have changed, we can perform the third level of multi-level classification, the coarsest level, which is called binary classification. In this binary classification, Area 3 is divided into two categories: changed areas and unchanged areas, meaning only three classification labels are included.

[0288] For the water conservancy feature change detection task in Area 3, a multi-layer decoder containing three sub-decoders can be used. Training the multi-layer decoder can include the following method steps:

[0289] In the first step, the first sub-decoder in the three-layer decoder is trained using the classification labels in the N categories;

[0290] In the second step, the classification labels in the M classification are used to train the second sub-decoder in the three-layer decoder;

[0291] In the third step, the classification labels in the two-class classification are used to train the third sub-decoder in the three-layer decoder.

[0292] Two remote sensing images corresponding to water conservancy area Area3 are obtained: P3t0 (the remote sensing image at time t0, equivalent to the first remote sensing image) and P3T (the remote sensing image at time T, equivalent to the second remote sensing image). Using a preset encoder to extract features from remote sensing images P3t0 and P3T, unclassified features Ch13 of water conservancy area Area3 are obtained. These unclassified features include the water conservancy feature Ch13t0 at time t0 and the water conservancy feature Ch13T at time T.

[0293] Inputting the unclassified feature Ch13 of the water conservancy area Area3 into decoder 40 yields three classification results. Sub-decoder 401 identifies the region C13 that has changed between remote sensing images P3t0 and P3T, and the region C03 that has not changed. The covered region classified by sub-decoder 402 is denoted as C33, while the uncovered region is denoted as C23. The covered region classified by sub-decoder 403 is denoted as C53, while the uncovered region is denoted as C43.

[0294] According to the three classification results obtained by the decoder 40, the water conservancy feature change detection result between the remote sensing image P3t0 (equivalent to the first image) and the remote sensing image P3T (equivalent to the second image) can be obtained. The specific method steps can be as follows:

[0295] The first step is to determine the area that belongs to area C13 and does not belong to area C33 (equivalent to the first difference result mentioned above), and record it as area D03, that is, D03 = C13-C33;

[0296] The second step is to set a threshold x13, select the second largest value C33x23 among the predicted values ​​of water conservancy and landform changes corresponding to area C33, and compare the second largest value C33x23 with the threshold x13;

[0297] In the third step, when the second largest value C33x23 is greater than the threshold x13, a partial area is selected from the area D03 according to the preset merging strategy, and is recorded as D03_1;

[0298] Step 4: Merge the region D03_1 into region C33 to obtain region C33a (equivalent to the above merged result);

[0299] Step 5: Determine the area that belongs to area C33a and does not belong to area C53 (equivalent to the third difference result mentioned above), and record it as area D13, that is, D13 = C33a - C53;

[0300] Step 6: Obtain the water conservancy feature change detection label set corresponding to area C33a, recorded as Y31 (equivalent to the sub-label set of the second-level classification label above);

[0301] Step 7: Select the second largest value C53x33 among the water conservancy feature change prediction values ​​corresponding to region C53, determine the water conservancy feature change detection label tx3 corresponding to the second largest value C53x33, and determine the subordinate relationship between the water conservancy feature change detection label tx3 and the water conservancy feature change detection label set Y31;

[0302] In the eighth step, when the water conservancy feature change detection label tx3 belongs to the water conservancy feature change detection label set Y31, a partial area is selected from the area D13 according to the preset merging strategy, and is recorded as D13_1;

[0303] In the ninth step, the region D13_1 is merged into the region C53 to obtain the region C53a. The water conservancy feature change detection result between the remote sensing image P3t0 and the remote sensing image P3T is obtained (i.e., the water conservancy feature change detection result after the regions C33 and C53 are updated to the regions C33a and C53a).

[0304] Different from determining the water conservancy feature change detection result directly based on the classification result, according to the above method provided by this embodiment, the classification result is processed through a preset merging strategy, which can further improve the accuracy of the water conservancy feature change detection result.

[0305] One embodiment of the present invention further provides an image processing method, which is run on a cloud server. Figure 8 is a flow chart of another image processing method according to an embodiment of the present invention. Figure 8 As shown, the image processing method includes:

[0306] Step S802, receiving a first image and a second image within a target area from a client;

[0307] Step S804: performing multi-level object classification on the first image and the second image to obtain a plurality of classification results, and merging the plurality of classification results to determine an object change detection result between the first image and the second image, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step;

[0308] Step S806: Return the ground feature change detection result to the client.

[0309] Optionally, Figure 9 is a schematic diagram of image processing on a cloud server according to an embodiment of the present invention. Figure 9 As shown, the client uploads the first image and the second image within the target area to the cloud server; the cloud server performs multi-level object classification on the first image and the second image to obtain multiple classification results, and merges the multiple classification results to determine the object change detection result between the first image and the second image, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step.

[0310] The cloud server then provides feedback to the client regarding the object change detection results between the first and second images. Ultimately, the object change detection results are presented to the user via the client's graphical user interface. The optional methods for displaying object change detection results in the graphical user interface have been described in the previous embodiments and will not be repeated here.

[0311] It should be noted that the above-mentioned image processing method provided by the embodiment of the present invention can be applicable to, but not limited to, practical application scenarios of land object classification, practical application scenarios of change detection, practical application scenarios of remote sensing image classification, and practical application scenarios of land object recognition. By interacting between the SaaS server and the client, the land object change detection model is used to detect the above-mentioned first image and the second image, and the returned land object change detection results are provided to the user through the client.

[0312] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0313] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0314] Example 2

[0315] According to an embodiment of the present invention, there is also provided an embodiment of a device for implementing the above-mentioned image processing method. Figure 10 is a structural block diagram of an image processing device according to an embodiment of the present invention. Figure 10 As shown, the device includes:

[0316] The acquisition module 1101 is used to acquire the first image and the second image within the target area; the classification module 1102 is used to perform multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; the merging module 1103 is used to merge the multiple classification results to determine the object change detection result between the first image and the second image.

[0317] It should be noted that the acquisition module 1101, classification module 1102, and merging module 1103 correspond to steps S202 to S206 in Example 1. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0318] In an embodiment of the present invention, an acquisition module is used to acquire a first image and a second image within the target area, and a training module is used to perform multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step. Then, a merging module is used to merge the multiple classification results to determine the object change detection result between the first image and the second image. Thus, the purpose of optimizing the object change detection model through image-based multi-level object classification is achieved, thereby achieving the technical effect of improving the accuracy of the object change detection results, and thus solving the technical problem of poor detection accuracy of the object change detection model in the related art, which is the processing method of directly learning the finest-grained single-level classification label by the object change detection model.

[0319] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Example 1 and will not be repeated here.

[0320] Example 3

[0321] According to an embodiment of the present invention, an embodiment of an electronic device is also provided. The electronic device can be any computing device in a computing device group. The electronic device includes: a processor and a memory, wherein:

[0322] The memory is connected to the above-mentioned processor and is used to provide the above-mentioned processor with instructions for processing the following processing steps: obtaining a first image and a second image within the target area; performing multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; merging the multiple classification results to determine the object change detection result between the first image and the second image.

[0323] In an embodiment of the present invention, a first image and a second image within the target area are acquired, and a multi-level feature classification method is used to perform multi-level feature classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of feature classification in the multi-level feature classification, and the types of feature changes contained in each level of feature classification in the multi-level feature classification increase step by step. The multiple classification results are then merged to determine the feature change detection result between the first image and the second image. Thus, the purpose of optimizing the feature change detection model through image-based multi-level feature classification is achieved, thereby achieving the technical effect of improving the accuracy of the feature change detection results, and thus solving the technical problem of poor detection accuracy of the feature change detection model in the related art, which is the processing method of directly learning the finest-grained single-level classification label by the feature change detection model.

[0324] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Example 1 and will not be repeated here.

[0325] Example 4

[0326] The embodiment of the present invention can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0327] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0328] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method: obtaining the first image and the second image within the target area; performing multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; merging the multiple classification results to determine the object change detection result between the first image and the second image.

[0329] Optionally, Figure 11 is a structural block diagram of another computer terminal according to an embodiment of the present invention, such as Figure 11 As shown, the computer terminal may include: one or more (only one is shown in the figure) processors 122 , a memory 124 , and a peripheral interface 126 .

[0330] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and device in the embodiments of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, corporate intranet, local area network, mobile communication network and combinations thereof.

[0331] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain the first image and the second image within the target area; perform multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; merge the multiple classification results to determine the object change detection result between the first image and the second image.

[0332] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtaining a first-level classification label, wherein the first-level classification label is used to indicate the changes in features between every two different types of features among multiple types of features; establishing a second-level classification label corresponding to each type of feature among multiple types of features based on the first-level classification label, wherein the second-level classification label is used to indicate the feature change category corresponding to each type of feature; establishing a third-level classification label corresponding to each type of feature among multiple types of features based on the second-level classification label, wherein the third-level classification label is used to indicate whether each type of feature among multiple types of features has changed; and constructing multi-level classification labels corresponding to multi-level feature classification using the first-level classification label, the second-level classification label and the third-level classification label.

[0333] Optionally, the processor may also execute the following program code: extract features from the first image and the second image through an encoder to obtain features to be classified; obtain a target decoder corresponding to multi-level classification labels, and perform multi-level object classification on the features to be classified based on the target decoder to obtain multiple classification results, wherein the target decoder is a multi-level decoder, and the decoder of each level includes: a fusion device and a classifier, the fusion device is used to fuse the features obtained at the current level and the features obtained at each level before the current level, and the classifier is used to obtain the classification result corresponding to the current level.

[0334] Optionally, the processor may also execute the program code of the following steps: in the fusion unit of the first layer decoder, performing feature fusion on the first feature to be classified to obtain a second feature to be classified, and in the classifier of the first layer decoder, performing object classification on the second feature to be classified to obtain a first classification result, wherein the first classification result is used to determine the area where there is a change and the area where there is no change between the first image and the second image; in the fusion unit of the second layer decoder, performing feature fusion on the second feature to be classified to obtain a third feature to be classified, and in the classifier of the second layer decoder, performing object classification on the third feature to be classified to obtain a second classification result, wherein the second classification result is used to determine the object change category to which the area where there is a change between the first image and the second image belongs; in the fusion unit of the third layer decoder, performing feature fusion on the third feature to be classified to obtain a fourth feature to be classified, and in the classifier of the third layer decoder, performing object classification on the fourth feature to be classified to obtain a third classification result, wherein the third classification result is used to determine the change category corresponding to the object change category between the first image and the second image.

[0335] Optionally, the processor may also execute the following steps of program code: determining a first difference result based on the first classification result and the second classification result, merging the first difference result and the second classification result in response to the second classification result satisfying the first preset condition, and determining the ground object change detection result between the first image and the second image, wherein the first preset condition is determined by the predicted value corresponding to the second classification result and the second-level classification label; determining a second difference result based on the second classification result and the third classification result, merging the second difference result and the third classification result in response to the third classification result satisfying the second preset condition, and determining the ground object change detection result between the first image and the second image, wherein the second preset condition is determined by the second classification result, the third classification result, the second ... The predicted value corresponding to the hierarchical classification label and the third hierarchical classification label is determined; a first difference result is determined based on the first classification result and the second classification result; in response to the second classification result satisfying the first preset condition, the first difference result and the second classification result are merged to obtain a merged result; a third difference result is determined based on the merged result and the third classification result; in response to the third classification result satisfying the second preset condition, the third difference result and the merged result are merged to determine the result of the ground object change detection between the first image and the second image, wherein the first preset condition is determined by the predicted value corresponding to the second classification result and the second hierarchical classification label, and the second preset condition is determined by the predicted value corresponding to the second classification result, the third classification result, the second hierarchical classification label and the third hierarchical classification label.

[0336] Optionally, the processor may also execute the program code of the following steps: displaying a plurality of candidate multi-level classification label templates in a graphical user interface, wherein each multi-level classification label template corresponds to a different category of ground object change detection scenarios; determining a target multi-level classification label template from a plurality of candidate objects in response to a first touch operation on the graphical user interface; and displaying a ground object change detection result of the target object between the first image and the second image in the graphical user interface in response to a second touch operation on the multi-level classification label template.

[0337] Optionally, the processor may also execute the program code of the following steps: in response to a first editing operation on the ground feature change detection result, adjusting the display range of the ground feature change detection result to obtain a first adjustment result, and / or, in response to a second editing operation on the ground feature change detection result, adjusting the display color of the ground feature change detection result to obtain a second adjustment result, and / or, in response to a sliding operation on the ground feature change detection result, checking the image changes of the ground feature change detection result between the first image and the second image; and updating the ground feature change detection result based on the first adjustment result and / or the second adjustment result.

[0338] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtaining a first urban planning remote sensing image and a second urban planning remote sensing image within the target area; performing multi-level land object classification on the first urban planning remote sensing image and the second urban planning remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of land object classification in the multi-level land object classification, and the types of land object changes contained in each level of land object classification in the multi-level land object classification increase step by step; merging the multiple classification results to determine the urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0339] Optionally, the processor may also execute program code for the following steps: displaying multiple urban planning areas in a graphical user interface; determining a target urban planning area from multiple urban planning areas in response to a first touch operation applied to the graphical user interface; and displaying in the graphical user interface the results of urban planning change detection of the target urban planning area between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0340] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtaining the first natural resource remote sensing image and the second natural resource remote sensing image within the target area; performing multi-level land object classification on the first natural resource remote sensing image and the second natural resource remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of land object classification in the multi-level land object classification, and the types of land object changes contained in each level of land object classification in the multi-level land object classification increase step by step; merging the multiple classification results to determine the natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image.

[0341] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving the first image and the second image within the target area from the client; performing multi-level object classification on the first image and the second image to obtain multiple classification results, and merging the multiple classification results to determine the object change detection result between the first image and the second image, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; and returning the object change detection result to the client.

[0342] In an embodiment of the present invention, a first image and a second image within the target area are acquired, and a multi-level feature classification method is used to perform multi-level feature classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of feature classification in the multi-level feature classification, and the types of feature changes contained in each level of feature classification in the multi-level feature classification increase step by step. The multiple classification results are then merged to determine the feature change detection result between the first image and the second image. Thus, the purpose of optimizing the feature change detection model through image-based multi-level feature classification is achieved, thereby achieving the technical effect of improving the accuracy of the feature change detection results, and thus solving the technical problem of poor detection accuracy of the feature change detection model in the related art, which is the processing method of directly learning the finest-grained single-level classification label by the feature change detection model.

[0343] It can be understood by those skilled in the art that Figure 11 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 11 It does not limit the structure of the above electronic device. For example, the computer terminal may also include Figure 11 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 11 Different configurations shown.

[0344] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0345] According to an embodiment of the present invention, an embodiment of a computer-readable storage medium is further provided. Optionally, in this embodiment, the computer-readable storage medium can be used to store program codes executed by the image processing method provided in the first embodiment.

[0346] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0347] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining a first image and a second image within the target area; performing multi-level object classification on the first image and the second image to obtain multiple classification results, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; merging the multiple classification results to determine the object change detection result between the first image and the second image.

[0348] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining a first-level classification label, wherein the first-level classification label is used to represent the feature changes occurring between every two different types of features in a plurality of types of features; establishing a second-level classification label corresponding to each type of feature in a plurality of types of features based on the first-level classification label, wherein the second-level classification label is used to represent the feature change category corresponding to each type of feature; establishing a third-level classification label corresponding to each type of feature in a plurality of types of features based on the second-level classification label, wherein the third-level classification label is used to indicate whether each type of feature in a plurality of types of features has changed; and constructing multi-level classification labels corresponding to multi-level feature classification using the first-level classification label, the second-level classification label and the third-level classification label.

[0349] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: extracting features from the first image and the second image through an encoder to obtain features to be classified; obtaining target decoders corresponding to multi-level classification labels, and performing multi-level object classification on the features to be classified based on the target decoders to obtain multiple classification results, wherein the target decoder is a multi-level decoder, and the decoder of each level includes: a fusion unit and a classifier, the fusion unit is used to fuse the features obtained at the current level and the features obtained at each level before the current level, and the classifier is used to obtain the classification results corresponding to the current level.

[0350] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: in the fusion unit of the first layer decoder, performing feature fusion on the first feature to be classified to obtain a second feature to be classified, and in the classifier of the first layer decoder, performing object classification on the second feature to be classified to obtain a first classification result, wherein the first classification result is used to determine the area where there is a change and the area where there is no change between the first image and the second image; in the fusion unit of the second layer decoder, performing feature fusion on the second feature to be classified to obtain a third feature to be classified, and in the classifier of the second layer decoder, performing object classification on the third feature to be classified to obtain a second classification result, wherein the second classification result is used to determine the object change category to which the area where there is a change between the first image and the second image belongs; in the fusion unit of the third layer decoder, performing feature fusion on the third feature to be classified to obtain a fourth feature to be classified, and in the classifier of the third layer decoder, performing object classification on the fourth feature to be classified to obtain a third classification result, wherein the third classification result is used to determine the change category corresponding to the object change category between the first image and the second image.

[0351] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: determining a first difference result based on the first classification result and the second classification result, merging the first difference result and the second classification result in response to the second classification result satisfying a first preset condition, and determining a result of detecting a change in land object between the first image and the second image, wherein the first preset condition is determined by the predicted value corresponding to the second classification result and the second-level classification label; determining a second difference result based on the second classification result and the third classification result, merging the second difference result and the third classification result in response to the third classification result satisfying a second preset condition, and determining a result of detecting a change in land object between the first image and the second image, wherein the second preset condition is determined by the second classification result, the third classification result and the second preset condition. The predicted values ​​corresponding to the three classification results, the second-level classification labels and the third-level classification labels are determined; a first difference result is determined based on the first classification result and the second classification result; in response to the second classification result satisfying the first preset condition, the first difference result and the second classification result are merged to obtain a merged result; a third difference result is determined based on the merged result and the third classification result; in response to the third classification result satisfying the second preset condition, the third difference result and the merged result are merged to determine the ground object change detection result between the first image and the second image, wherein the first preset condition is determined by the predicted values ​​corresponding to the second classification result and the second-level classification label, and the second preset condition is determined by the predicted values ​​corresponding to the second classification result, the third classification result, the second-level classification label and the third-level classification label.

[0352] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: displaying multiple candidate multi-level classification label templates in a graphical user interface, wherein each multi-level classification label template corresponds to a different category of ground object change detection scenarios; in response to a first touch operation applied to the graphical user interface, determining a target multi-level classification label template from multiple candidate objects; in response to a second touch operation applied to the multi-level classification label template, displaying the ground object change detection result of the target object between the first image and the second image in the graphical user interface.

[0353] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: in response to a first editing operation on the land feature change detection result, adjusting the display range of the land feature change detection result to obtain a first adjustment result, and / or, in response to a second editing operation on the land feature change detection result, adjusting the display color of the land feature change detection result to obtain a second adjustment result, and / or, in response to a sliding operation on the land feature change detection result, checking the image changes of the land feature change detection result between the first image and the second image; and updating the land feature change detection result based on the first adjustment result and / or the second adjustment result.

[0354] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining a first urban planning remote sensing image and a second urban planning remote sensing image within the target area; performing multi-level land object classification on the first urban planning remote sensing image and the second urban planning remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of land object classification in the multi-level land object classification, and the types of land object changes contained in each level of land object classification in the multi-level land object classification increase step by step; merging the multiple classification results to determine the urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0355] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: displaying multiple urban planning areas in a graphical user interface; determining a target urban planning area from multiple urban planning areas in response to a first touch operation applied to the graphical user interface; and displaying in the graphical user interface the urban planning change detection results of the target urban planning area between the first urban planning remote sensing image and the second urban planning remote sensing image.

[0356] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining a first natural resource remote sensing image and a second natural resource remote sensing image within the target area; performing multi-level land feature classification on the first natural resource remote sensing image and the second natural resource remote sensing image to obtain multiple classification results, wherein the multiple classification results correspond to each level of land feature classification in the multi-level land feature classification, and the types of land feature changes contained in each level of land feature classification in the multi-level land feature classification increase step by step; merging the multiple classification results to determine the natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image.

[0357] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: receiving a first image and a second image within a target area from a client; performing multi-level object classification on the first image and the second image to obtain multiple classification results, and merging the multiple classification results to determine an object change detection result between the first image and the second image, wherein the multiple classification results correspond to each level of object classification in the multi-level object classification, and the types of object changes contained in each level of object classification in the multi-level object classification increase step by step; and returning the object change detection result to the client.

[0358] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0359] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0360] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.

[0361] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0362] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0363] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0364] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire a first image and a second image within the target area; Performing multi-level object classification on the first image and the second image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step; Merging the multiple classification results to determine a ground object change detection result between the first image and the second image; The image processing method also includes: constructing a multi-level classification label corresponding to the multi-level feature classification, wherein the multi-level classification label includes a first-level classification label, a second-level classification label and a third-level classification label, the first-level classification label is used to represent the feature changes occurring between every two different types of features in a plurality of types of features, the second-level classification label is used to represent the feature change category corresponding to each type of feature, and the third-level classification label is used to indicate whether each type of feature in the plurality of types of features has changed, and the multi-level classification label includes three levels of classification labels and the correspondence between the three levels of classification labels.

2. The image processing method according to claim 1, wherein: The image processing method further includes: Get the first-level classification label; Establishing a second-level classification label corresponding to each type of ground feature in the multiple types of ground features based on the first-level classification label; A third-level classification label corresponding to each type of ground feature in the multiple types of ground features is established based on the second-level classification label.

3. The image processing method according to claim 2, wherein: Performing multi-level object classification on the first image and the second image to obtain the multiple classification results includes: Performing feature extraction on the first image and the second image by an encoder to obtain features to be classified; Obtain a target decoder corresponding to the multi-level classification label, and perform multi-level object classification on the features to be classified based on the target decoder to obtain the multiple classification results, wherein the target decoder is a multi-level decoder, and the decoder of each level includes: a fusion device and a classifier, the fusion device is used to fuse the features obtained at the current level and the features obtained at each level before the current level, and the classifier is used to obtain the classification result corresponding to the current level.

4. The image processing method according to claim 3, wherein: The target decoder includes: a first layer decoder, a second layer decoder and a third layer decoder, the features to be classified include: a first feature to be classified, a second feature to be classified, a third feature to be classified and a fourth feature to be classified, and the features to be classified are subjected to multi-level object classification based on the target decoder, and the multiple classification results obtained include: In the fusion unit of the first layer decoder, feature fusion is performed on the first feature to be classified to obtain a second feature to be classified, and in the classifier of the first layer decoder, ground object classification is performed on the second feature to be classified to obtain a first classification result, wherein the first classification result is used to determine an area where changes occur and an area where no changes occur between the first image and the second image; In the fusion unit of the second layer decoder, feature fusion is performed on the second feature to be classified to obtain a third feature to be classified, and in the classifier of the second layer decoder, ground object classification is performed on the third feature to be classified to obtain a second classification result, wherein the second classification result is used to determine a ground object change category to which an area having a change between the first image and the second image belongs; In the fusion device of the third layer decoder, the third feature to be classified is subjected to feature fusion to obtain a fourth feature to be classified, and in the classifier of the third layer decoder, the fourth feature to be classified is subjected to ground object classification to obtain a third classification result, wherein the third classification result is used to determine the change category corresponding to the ground object change category between the first image and the second image.

5. The image processing method according to claim 4, characterized in that The multiple classification results are combined to determine that a ground object change detection result between the first image and the second image includes one of the following: determining a first difference result based on the first classification result and the second classification result, and merging the first difference result and the second classification result in response to the second classification result satisfying a first preset condition to determine a ground object change detection result between the first image and the second image, wherein the first preset condition is determined by a predicted value corresponding to the second classification result and the second-level classification label; Determining a second difference result based on the second classification result and the third classification result, and in response to the third classification result satisfying a second preset condition, merging the second difference result and the third classification result to determine a ground object change detection result between the first image and the second image, wherein the second preset condition is determined by the second classification result, the third classification result, and the predicted values ​​corresponding to the second-level classification label and the third-level classification label; A first difference result is determined based on the first classification result and the second classification result. In response to the second classification result satisfying a first preset condition, the first difference result and the second classification result are merged to obtain a merged result. A third difference result is determined based on the merged result and the third classification result. In response to the third classification result satisfying a second preset condition, the third difference result and the merged result are merged to determine the ground object change detection result between the first image and the second image, wherein the first preset condition is determined by the predicted value corresponding to the second classification result and the second-level classification label, and the second preset condition is determined by the predicted value corresponding to the second classification result, the third classification result, the second-level classification label and the third-level classification label.

6. The image processing method according to claim 1, wherein: A graphical user interface is provided by a terminal device, wherein the content displayed by the graphical user interface at least partially includes a ground object change detection scene, and the image processing method further includes: Displaying a plurality of candidate multi-level classification label templates in the graphical user interface, wherein each multi-level classification label template corresponds to a different category of ground feature change detection scenario; In response to a first touch operation on the graphical user interface, determining a target multi-level classification label template from a plurality of candidate objects; In response to a second touch operation applied to the multi-level classification label template, a ground object change detection result of a target object between the first image and the second image is displayed in the graphical user interface.

7. The image processing method according to claim 6, characterized in that: The image processing method further includes: In response to a first editing operation on the ground feature change detection result, adjusting a display range of the ground feature change detection result to obtain a first adjustment result; and / or, in response to a second editing operation on the ground feature change detection result, adjusting a display color of the ground feature change detection result to obtain a second adjustment result; and / or, in response to a sliding operation on the ground feature change detection result, viewing an image change of the ground feature change detection result between the first image and the second image; The ground feature change detection result is updated based on the first adjustment result and / or the second adjustment result.

8. An image processing method, characterized in that: include: Acquire a first urban planning remote sensing image and a second urban planning remote sensing image within the target area; Performing multi-level object classification on the first urban planning remote sensing image and the second urban planning remote sensing image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step; Merging the multiple classification results to determine an urban planning change detection result between the first urban planning remote sensing image and the second urban planning remote sensing image; The image processing method also includes: constructing a multi-level classification label corresponding to the multi-level feature classification, wherein the multi-level classification label includes a first-level classification label, a second-level classification label and a third-level classification label, the first-level classification label is used to represent the feature changes occurring between every two different types of features in a plurality of types of features, the second-level classification label is used to represent the feature change category corresponding to each type of feature, and the third-level classification label is used to indicate whether each type of feature in the plurality of types of features has changed, and the multi-level classification label includes three levels of classification labels and the correspondence between the three levels of classification labels.

9. The image processing method according to claim 8, characterized in that: A graphical user interface is provided by a terminal device, wherein the content displayed by the graphical user interface at least partially includes an urban planning change detection scene, and the image processing method further includes: displaying a plurality of urban planning areas within the graphical user interface; In response to a first touch operation applied to the graphical user interface, determining a target urban planning area from the plurality of urban planning areas; In response to a second touch operation applied to the target urban planning area, an urban planning change detection result of the target urban planning area between the first urban planning remote sensing image and the second urban planning remote sensing image is displayed in the graphical user interface.

10. An image processing method, characterized in that: include: Acquire a first natural resource remote sensing image and a second natural resource remote sensing image within the target area; Performing multi-level object classification on the first natural resource remote sensing image and the second natural resource remote sensing image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step; Merging the multiple classification results to determine a natural resource change detection result between the first natural resource remote sensing image and the second natural resource remote sensing image; The image processing method also includes: constructing a multi-level classification label corresponding to the multi-level feature classification, wherein the multi-level classification label includes a first-level classification label, a second-level classification label and a third-level classification label, the first-level classification label is used to represent the feature changes occurring between every two different types of features in a plurality of types of features, the second-level classification label is used to represent the feature change category corresponding to each type of feature, and the third-level classification label is used to indicate whether each type of feature in the plurality of types of features has changed, and the multi-level classification label includes three levels of classification labels and the correspondence between the three levels of classification labels.

11. An image processing method, characterized in that: include: receiving a first image and a second image within a target area from a client; Performing multi-level object classification on the first image and the second image to obtain a plurality of classification results, and merging the plurality of classification results to determine an object change detection result between the first image and the second image, wherein the plurality of classification results respectively correspond to each level of object classification in the multi-level object classification, and the types of object changes included in each level of object classification in the multi-level object classification increase step by step; Returning the ground feature change detection result to the client; The image processing method also includes: constructing a multi-level classification label corresponding to the multi-level feature classification, wherein the multi-level classification label includes a first-level classification label, a second-level classification label and a third-level classification label, the first-level classification label is used to represent the feature changes occurring between every two different types of features in a plurality of types of features, the second-level classification label is used to represent the feature change category corresponding to each type of feature, and the third-level classification label is used to indicate whether each type of feature in the plurality of types of features has changed, and the multi-level classification label includes three levels of classification labels and the correspondence between the three levels of classification labels.

12. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 11.

13. An image processing system, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Step 1: Acquire a first image and a second image within the target area; Step 2: performing multi-level feature classification on the first image and the second image to obtain a plurality of classification results, wherein the plurality of classification results respectively correspond to each level of feature classification in the multi-level feature classification, and the feature change types included in each level of feature classification in the multi-level feature classification increase step by step; Step 3, merging the multiple classification results to determine a ground object change detection result between the first image and the second image; The processing steps also include step 4, constructing a multi-level classification label corresponding to the multi-level feature classification, wherein the multi-level classification label includes a first-level classification label, a second-level classification label and a third-level classification label, the first-level classification label is used to represent the feature changes occurring between every two different types of features in multiple types of features, the second-level classification label is used to represent the feature change category corresponding to each type of feature, and the third-level classification label is used to indicate whether each type of feature in the multiple types of features has changed, and the multi-level classification label includes three levels of classification labels and the correspondence between the three levels of classification labels.

Images