Image processing method and storage medium

Abstract

The present invention discloses an image processing method and storage medium. The method comprises: acquiring first image data of a first target object, wherein the first image data belongs to a first data domain; converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; and performing segmentation processing on the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of the second target object, wherein the third image data belongs to the second data domain. The present invention solves the technical problem of poor segmentation results for image data in different data domains.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to an image processing method and a storage medium. Background Art

[0002] At present, the relevant technology for segmentation processing of remote sensing images is an optimization method of supervised pre-training and supervised fine-tuning training. It requires labeled data in both the target domain and the source domain, has high labor costs, and has the technical problem of poor segmentation effect of image data in different data domains.

[0003] Currently, no effective solution has been proposed to the above-mentioned problem of poor segmentation effect of image data in different data domains. Summary of the Invention

[0004] The embodiments of the present invention provide an image processing method and a storage medium to at least solve the technical problem of poor segmentation effect of image data in different data domains.

[0005] According to one aspect of an embodiment of the present invention, an image processing method is provided. The method may include: acquiring first image data of a first target object, wherein the first image data belongs to a first data domain; converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; and performing segmentation processing on the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of the second target object, wherein the third image data belongs to the second data domain.

[0006] According to one aspect of an embodiment of the present invention, another image processing method is also provided. The method may include: responding to a first input instruction on an operation interface, inputting first image data of a first target object, wherein the first image data belongs to a first data domain; responding to a segmentation processing instruction on the operation interface, displaying a target segmentation result on the operation interface, wherein the target segmentation result is obtained by segmenting second image data of the first target object based on a target segmentation model, the second image data is converted from the first image data, the second image data belongs to a second data domain, and the target segmentation model is trained based on third image data of the second target object, and the third image data belongs to the second data domain.

[0007] According to one aspect of an embodiment of the present invention, another image processing method is also provided. The method may include: acquiring first remote sensing image data of a first area object, wherein the first remote sensing image data belongs to a first data domain; converting the first remote sensing image data into second remote sensing image data of the first area object, wherein the second remote sensing image data belongs to a second data domain; and performing segmentation processing on the second remote sensing image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third remote sensing image data of the second area object, wherein the third remote sensing image data belongs to the second data domain.

[0008] An embodiment of the present invention further provides an image processing device. The image processing device includes: a first acquisition unit for acquiring first image data of a first target object, wherein the first image data belongs to a first data domain; a first conversion unit for converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; and a first segmentation unit for performing segmentation processing on the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of the second target object, wherein the third image data belongs to the second data domain.

[0009] An embodiment of the present invention further provides an image processing device from a human-computer interaction perspective. The image processing device includes: an input unit for responding to a first input instruction on an operation interface and inputting first image data of a first target object, wherein the first image data belongs to a first data domain; and a first display unit for responding to a segmentation processing instruction on the operation interface and displaying a target segmentation result on the operation interface, wherein the target segmentation result is obtained by segmenting second image data of the first target object based on a target segmentation model, wherein the second image data is converted from the first image data and belongs to a second data domain, and the target segmentation model is trained based on third image data of the second target object and the third image data belongs to the second data domain.

[0010] An embodiment of the present invention also provides an image processing device for the scenario of remote sensing image style transfer. The image processing device includes: a second acquisition unit for acquiring first remote sensing image data of a first regional object, wherein the first remote sensing image data belongs to a first data domain; a second conversion unit for converting the first remote sensing image data into second remote sensing image data of the first regional object, wherein the second remote sensing image data belongs to a second data domain; and a second segmentation unit for performing segmentation processing on the second remote sensing image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third remote sensing image data of the second regional object, wherein the third remote sensing image data belongs to the second data domain.

[0011] An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored program, wherein when the program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute the image processing method of the embodiment of the present invention.

[0012] An embodiment of the present invention further provides a processor for running a program, wherein the image processing method of the embodiment of the present invention is executed when the program is run.

[0013] An embodiment of the present invention also provides an image processing system. The system may include: a processor; and a memory connected to the processor and configured to provide the processor with instructions for performing the following processing steps: acquiring first image data of a first target object, wherein the first image data belongs to a first data domain; converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; and performing segmentation processing on the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of the second target object, wherein the third image data belongs to the second data domain.

[0014] In an embodiment of the present invention, an unsupervised image segmentation method is adopted. By migrating the unlabeled first image data of the first data domain into the second image data of the second data domain, and then sending the second image data to the segmentation model for the second data domain for segmentation processing, the purpose of improving the segmentation effect of the first image data is achieved, thereby achieving the purpose of improving the effect of segmenting image data of different data domains, and further solving the technical problem of poor segmentation effect of image data of different data domains. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0016] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present disclosure;

[0017] Figure 2 is a flowchart of an image processing method according to an embodiment of the present invention;

[0018] Figure 3 is a flowchart of an image processing method provided from the human-computer interaction side according to an embodiment of the present invention;

[0019] Figure 4 is a flowchart of an image processing method provided in a scenario of style migration from a remote sensing image according to an embodiment of the present invention;

[0020] Figure 5 is a schematic diagram of a remote sensing segmentation domain adaptation method based on image generation according to an embodiment of the present invention;

[0021] Figure 6 is a schematic diagram of a processing process of a G-Net network according to an embodiment of the present invention;

[0022] Figure 7 is a schematic diagram of an image processing device according to an embodiment of the present invention;

[0023] Figure 8 is a schematic diagram of a data processing device provided from the human-computer interaction side according to an embodiment of the present invention;

[0024] Figure 9 is a schematic diagram of a data processing device provided in a scenario of style migration from remote sensing images according to an embodiment of the present invention;

[0025] Figure 10 It is a structural block diagram of a computer terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretations:

[0029] Cycle-consistent adversarial networks, used to automatically transform one type of image into another;

[0030] Generative networks, an unsupervised learning model;

[0031] Image segmentation is the technology and process of dividing an image into several specific regions with unique properties and extracting the target of interest.

[0032] Example 1

[0033] According to an embodiment of the present invention, an embodiment of a method for image processing is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (illustrated as 102a, 102b, ..., 102n) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0035] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0036] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned image processing method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0038] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0039] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 1This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the aforementioned computer device (or mobile device).

[0040] Figure 2 FIG. 1 is a flow chart of an image processing method according to an embodiment of the present invention. Figure 2 As shown, the image processing method may include the following steps:

[0041] Step S202: Acquire first image data of a first target object, wherein the first image data belongs to a first data domain.

[0042] In the technical solution provided in step S202 of the present invention, the first image data corresponding to the first target object to be acquired can be further determined by acquiring image data in the data domain. Alternatively, the first image data of the first target object can be acquired by a drone camera.

[0043] In this embodiment, the first data domain may be various morphological domains, such as domestic high-resolution images, Google satellite images, and Sentinel images.

[0044] In this embodiment, the first data domain may be a source domain, indicating a data domain to be migrated where the first image is located. Optionally, the first data domain only contains images but no labeled data.

[0045] In this embodiment, the first image data of the first target object can be further obtained by acquiring image data in the data domain. For example, high-resolution image data can be acquired through a drone shooting device. When it is determined that the first target object is Beijing, high-resolution image data of Beijing is further acquired.

[0046] Step S204 : converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain.

[0047] In the technical solution provided in the above step S204 of the present invention, after obtaining the first image data of the first target object, the first image data needs to be converted into the second image data of the first target object. For example, after obtaining the high-resolution image data of Beijing through the above-mentioned drone shooting equipment, the high-resolution image data of Beijing needs to be converted into the second image data of Beijing. The second image data can be data with a second data domain style.

[0048] In this embodiment, the second data domain may be various morphological domains, such as domestic high-resolution images, Google satellite images, and Sentinel images.

[0049] In this embodiment, the first data domain may be a target domain, indicating the target data domain to which the first image is to be migrated. Optionally, there is labeled data in the first data domain, and supervised segmentation training can be performed.

[0050] In the above embodiment, the first image data is converted into second image data of the first target object, wherein the second image data belongs to the second data domain. For example, the first image data of the source domain B is converted into image data of the target domain A, and the second image data can be image data of the first target object in the style of the target domain A.

[0051] In the above embodiment, the above second image data belongs to the second data domain. In order to realize the conversion of the first image data with the style of the source domain B into the image data with the style of the target domain A, the first image data is optionally converted into the second image data of the first target object. For example, the second data domain is Google satellite image data, and the first image data is high-resolution image data of Beijing. Then, the high-resolution image data of Beijing can be converted into Google satellite image data of Beijing.

[0052] Optionally, the above-mentioned conversion of the first image data into the second image data of the first target object can be achieved through a conversion model, such as a generator network (Generate Networks, referred to as G-Net).

[0053] In the above embodiment, the first image data can be converted into second image data of the first target object through the generator network G-Net, wherein the G-Net can convert the input random noise into a realistic picture by continuously learning the probability distribution of real data in the training set, that is, the more similar the generated picture is to the picture in the training set, the better.

[0054] Step S206 , performing segmentation processing on the second image data based on the target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0055] In the technical solution provided in the above step S206 of the present invention, after the first image data is converted into the second image data of the first target object, in order to obtain a segmentation result, it is necessary to further perform segmentation processing on the second image data.

[0056] In the above embodiment, the second image data is segmented. Optionally, the second image data is segmented using a target segmentation model to obtain a target segmentation result.

[0057] In this embodiment, optionally, the third image data belongs to the second data domain, and the third image data is labeled data.

[0058] In the above embodiment, the target segmentation model can be obtained by training the third image data based on the second target object. For example, the third image data belongs to the second data domain, and the third image data is labeled. The target segmentation model is obtained by training the labeled third image data. The target segmentation model can perform good segmentation processing on images with the style of the second data domain.

[0059] In the above embodiment, the target segmentation model can be a deep convolutional encoder-decoder architecture (Segmantic Segmentation, abbreviated as SegNet) for image segmentation. SegNet achieves image segmentation by classifying each pixel in the image and identifying the category of each pixel. The upsampling of SegNet is reverse maximum pooling. When downsampling, SegNet will record the position of the maximum pooling pointer and fill the rest with 0.

[0060] In the above embodiment, the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain. For example, the second data domain is the target domain A, and SegNet is trained based on the third image data having the style of the target domain A, and then the second image data is segmented based on the SegNet obtained by the above method to obtain the segmentation result.

[0061] Through the above steps S202 to S206 of the present application, the first image data of the first target object is obtained, wherein the first image data belongs to the first data domain; the first image data is converted into the second image data of the first target object, wherein the second image data belongs to the second data domain; the second image data is segmented based on the target segmentation model to obtain the target segmentation result, wherein the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain. In other words, the first image data of the unlabeled first data domain is migrated to the second image data of the second data domain, and then the second image data is sent to the segmentation model for the second data domain for segmentation processing. This is an unsupervised image segmentation method that can improve the segmentation effect of the first image data and solve the technical problem of poor segmentation effect of image data in different data domains.

[0062] The above method of this embodiment is further introduced below.

[0063] As an optional implementation, step S206, converting the first image data into second image data of the target object, includes: converting the first image data into second image data based on a sub-data conversion model, wherein the data conversion model to which the sub-data conversion model belongs is used to convert the first image data and the third image data into each other.

[0064] In the above embodiment, the first image data is converted into the second image data based on the sub-data conversion model. Optionally, the sub-data conversion model is a generative network model G-Net, and the first image data can be converted into the second image data based on the generative network model.

[0065] In the above embodiment, the data conversion model to which the sub-data conversion model belongs is used to convert the first image data and the third image data into each other. Optionally, the above data conversion model is a cycle-consistent adversarial network (Cycle-Consistent Adversarial Networks, referred to as Cycle GAN for short). The cycle-consistent adversarial network can convert information from one representation form to another. For example, when a given image is given, it can be blurred, colored, its clarity can be improved, or missing gaps can be filled.

[0066] In the above embodiment, the data conversion model is used to convert the first image data and the third image data into each other. Optionally, the third image data whose data domain is the target domain A is converted into the first image data whose data domain is the source domain B through the data conversion model.

[0067] In the above embodiment, the third image data whose data domain is the target domain A is converted into the first image data whose data domain is the source domain B through the data conversion model. Alternatively, the first image data whose data domain is the source domain B is converted into the third image data whose data domain is the target domain A through the data conversion model. For example, a high-resolution image of Beijing whose source domain is the high-resolution image data domain is converted into a Google satellite image of Xinjiang whose source domain is the Google satellite map data domain through the data conversion model. In short, the data conversion model can convert images belonging to different data domains into each other.

[0068] In the above embodiment, the sub-data conversion model is one of the components of the data conversion model. The sub-data conversion model can be used to convert the first image data into the second image data. For example, the sub-data conversion model extracts the content of the first image and converts the first image data belonging to the Google satellite image data domain into the second image data belonging to the high-resolution image data domain, thereby converting the first image data into the second image data of the target object.

[0069] As an optional implementation, converting the first image data into the second image data based on the sub-conversion model includes: obtaining high-frequency image information and low-frequency image information of the first image data based on the sub-conversion model; and generating the second image data based on the high-frequency image information and the low-frequency image information.

[0070] In this embodiment, high-frequency image information and low-frequency image information of the first image data are obtained based on the sub-transformation model. In related applications, due to the influence of light and fog, remote sensing images at the same location and time often have different expressions. To address this problem, a high-frequency and low-frequency reconstruction G-Net network is used to replace the original direct output G-Net network, and the high-frequency image information and low-frequency image information of the first image data are obtained through the sub-transformation model G-Net network.

[0071] In this embodiment, the second image data is generated based on the high-frequency image information and the low-frequency image information. For example, in order to convert the first image data of the first target having the style of the source domain B into the second image having the style of the target domain A, the high-frequency image information and the low-frequency image information of the first image data can be obtained through the sub-conversion model G-Net network, and then the second image data having the style of the target domain A can be generated based on the high-frequency image information and the low-frequency image information.

[0072] In this embodiment, the frequency of the image can be an indicator of the degree of grayscale value change, which is the gradient of grayscale in the plane space. The high-frequency image information can be the image data of the part where the grayscale changes very quickly, that is, the frequency of change is high. The low-frequency image can be the grayscale that changes slowly. The content within the edge is low frequency, and the content within the edge is most of the image information.

[0073] As an optional implementation, generating the second image data based on the high-frequency image information and the low-frequency image information includes: reconstructing the high-frequency image information and the low-frequency image information to obtain the second image data.

[0074] In this embodiment, the high-frequency image information and the low-frequency image information are reconstructed to obtain the second image data. The high-frequency image information and the low-frequency image information can be reconstructed through a sub-transformation model G-Net network to obtain the second image data.

[0075] In this embodiment, the sub-conversion model G-Net network is a component of the conversion model Cycle-GAN. The standard Cycle-GAN is a GAN model with a cyclic structure, which includes two parts: the generator network G-Net and the discriminator D-Net. The mutual conversion between different data domains is realized through a cycle. The high-frequency and low-frequency reconstruction G-Net network can reconstruct the high-frequency image information and the low-frequency image information to obtain the second image data.

[0076] As an optional embodiment, converting the first image data into the second image data based on the sub-conversion model includes: eliminating the sub-image data in the first image data based on the sub-conversion model to obtain the second image data, wherein the sub-image data is image data generated by lighting information and / or cloud information.

[0077] In this embodiment, sub-image data in the first image data is eliminated based on the sub-conversion model to obtain the second image data. For example, when the first image data is high-resolution image data, since the map data acquired by the drone shooting device includes high-resolution image data with light reflection and cloud occlusion, when converting the first image data of the first target with the source domain high-resolution image style into the second image data with the target domain Google satellite image style, the sub-image data with light reflection and cloud occlusion in the first image data needs to be removed to achieve a better image migration effect.

[0078] In this embodiment, the sub-image data is image data generated by illumination information and / or cloud information. For example, the sub-image data is image data of brightness and cloud in the first image data.

[0079] As an optional implementation, the first image data is image data that is not labeled with a target label.

[0080] In this embodiment, the first image data is image data that is not labeled with a target label. For example, when the target segmentation model is trained based on the third image data of the second target object, the third image data is labeled image data having the style of target data domain A. After the target segmentation model is trained with the third image data having the style of target data domain A, it can perform segmentation processing on unlabeled images having the style of target data domain A.

[0081] In this embodiment, the target label may be manually annotated data.

[0082] In this embodiment, when performing image segmentation processing, the first image data to be migrated needs to be manually labeled and then the segmentation model is trained to obtain the segmentation result. However, the present application converts the first image data of the first target with the style of source domain B into second image data with the style of target domain A, and then obtains the segmentation result through a pre-trained segmentation model. The entire processing process does not require manual labeling.

[0083] As an optional implementation, the method further includes: displaying a classification result of the first target object on the operation interface based on the target segmentation result; and / or displaying a detection result of the first target object on the operation interface based on the target segmentation result.

[0084] In this embodiment, the classification result of the first target object is displayed on the operation interface based on the target segmentation result. For example, after the second image is segmented by the trained target segmentation model SegNet, each pixel in the second image is classified to achieve pixel-level classification, and then the classification result of the first target object is displayed on the operation interface. The classification result can be a ground object classification result.

[0085] In this embodiment, the detection result of the first target object is displayed on the operation interface based on the target segmentation result. For example, after the second image is segmented by the trained target segmentation model SegNet, target instances of a specific class in the digital image are detected (or identified), and the target instances include mountains, rivers, and buildings in remote sensing images.

[0086] As an optional implementation, the target segmentation result includes multiple sub-segmentation results, and the method also includes: determining indication information for each sub-segmentation result, wherein the indication information is used to indicate the type of each sub-segmentation result; and displaying each sub-segmentation result separately on the operation interface according to the indication information.

[0087] In this embodiment, indication information of each sub-segmentation result is determined, wherein the indication information is used to indicate the type of each sub-segmentation result. For example, the target segmentation model SegNet performs upsampling and convolution at the decoder, and finally sends each pixel to the classifier, which outputs the classification result. The indication information is used to indicate the type of the classification result.

[0088] In this embodiment, each sub-segmentation result is displayed on the operation interface according to the indication information. For example, the indication information is used to represent the type of the classification result, and then each word segmentation result is determined and displayed on the operation interface.

[0089] As an optional implementation manner, the first target object is a target area, and the method further includes: displaying each sub-segmentation result in the target area on the operation interface according to the area corresponding to each sub-segmentation result.

[0090] In this embodiment, the interface displays each sub-segmentation result according to the area corresponding to each sub-segmentation result in the target area. For example, if the first target object is Beijing, the operation interface displays buildings and streets according to the area corresponding to each sub-segmentation result in the target area Beijing. The area corresponding to the sub-segmentation result can be buildings and streets, etc.

[0091] As an optional embodiment, the method also includes: responding to a selection operation instruction acting on multiple sub-segmentation results, selecting a target sub-segmentation result from the multiple sub-segmentation results; responding to an editing operation instruction acting on the target sub-segmentation result, performing an editing operation on the target sub-segmentation result, obtaining an editing result, and displaying the editing result on the operation interface.

[0092] In this embodiment, in response to a selection operation instruction acting on multiple sub-segmentation results, a target sub-segmentation result is selected from the multiple sub-segmentation results. For example, each sub-segmentation result, such as a building and a street, is displayed on an operation interface, and a selection operation can be performed on the multiple sub-segmentation results. At this time, in response to a selection operation instruction acting on the multiple sub-segmentation results, a target sub-segmentation result is selected from the multiple sub-segmentation results.

[0093] In this embodiment, in response to an editing operation instruction acting on the target sub-segmentation result, an editing operation is performed on the target sub-segmentation result to obtain an editing result, and the editing result is displayed on the operation interface. For example, after selecting multiple target sub-segmentation results on the operation interface, an editing operation can also be performed on the target sub-segmentation results to obtain an editing result.

[0094] As an optional implementation, the method further includes: acquiring adjustment information, wherein the adjustment information includes information used to characterize the accuracy of the target segmentation result; and adjusting the target segmentation model based on the adjustment information.

[0095] In this embodiment, adjustment information is obtained, wherein the adjustment information includes information for characterizing the accuracy of the target segmentation result. For example, after the image is segmented by the target segmentation model SegNet, the adjustment information can be used to characterize the accuracy of the target segmentation result. The adjustment information can be parameter information for adjusting the target segmentation model.

[0096] In this embodiment, the target segmentation model is adjusted based on the adjustment information. For example, the adjustment information may be the upsampling multiple of the target segmentation model. Fusing high-level features with low-level features can significantly improve the classification effect of pixel points. By adjusting the upsampling multiple of the fusion result, the target segmentation model is adjusted based on the adjustment information.

[0097] The embodiment of the present invention also provides another image processing method from the human-computer interaction side.

[0098] Figure 3 FIG. 1 is a flow chart of an image processing method provided from the human-computer interaction side according to an embodiment of the present invention. Figure 3 As shown, the method may include the following steps:

[0099] Step S302, in response to a first input instruction on the operation interface, inputting first image data of a first target object, wherein the first image data belongs to a first data domain;

[0100] In the technical solution provided in the above step S302 of the present invention, the operation interface can be a human-computer interaction interface on the front-end client. The user can trigger the operation interface to generate an input operation instruction, which is used to input the first target image, thereby responding to the input operation instruction and obtaining the first target image.

[0101] In this embodiment, the first input instruction may be inputting the first image data of the first target object. For example, by issuing an instruction to input a high-resolution image of Beijing on the operation interface, the high-resolution image of Beijing is inputted in response to the instruction.

[0102] In this embodiment, the first image data of the first target object may be unlabeled data, which is converted into second image data having the style of the target domain through the sub-conversion model, and then the second image data is segmented through the target segmentation model.

[0103] Step S304, in response to the segmentation processing instruction on the operation interface, the target segmentation result is displayed on the operation interface, wherein the target segmentation result is obtained by segmenting the second image data of the first target object based on the target segmentation model, the second image data is converted from the first image data, the second image data belongs to the second data domain, and the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0104] In the technical solution provided in the above step S304 of the present invention, the user can trigger the operation interface to generate a segmentation processing instruction, which is used to instruct the target segmentation model to perform segmentation processing on the second image data to obtain a segmentation result.

[0105] In this embodiment, in response to the segmentation processing instruction applied on the operation interface, the target segmentation result is displayed on the operation interface. For example, the segmentation processing instruction for the image data is input on the operation interface, and the target segmentation result is obtained after the second image is segmented and processed by the target segmentation model SegNet.

[0106] In this embodiment, the target segmentation result is obtained by performing segmentation processing on the second image data of the first target object based on the target segmentation model.

[0107] In this embodiment, the second image data is converted from the first image data, and the second image data belongs to the second data domain. For example, the first image data is converted into the second image data through the sub-conversion model G-Net network. The first image data belongs to the source domain B, and the second image data belongs to the target domain A, thereby realizing the style change of the first image.

[0108] In this embodiment, the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain. For example, the target segmentation model SegNet is trained by the third image data of the second target object, and the third image data of the second target object is labeled data and is trained in a supervised manner. The target segmentation model SegNet is used to segment and process pictures with the style of the second target object.

[0109] The embodiment of the present invention also provides another data processing method for the scenario of style migration of remote sensing images.

[0110] Figure 4 FIG. 1 is a flow chart of a data processing method provided in the context of style migration from remote sensing images according to an embodiment of the present invention. Figure 4 As shown, the method may include the following steps:

[0111] Step S402: Acquire first remote sensing image data of a first area object, wherein the first remote sensing image data belongs to a first data domain.

[0112] In the technical solution provided in the above step S402 of the present invention, optionally, the data domain can be acquired through a drone shooting device, and the image data in the data domain can be acquired according to relevant requirements. After determining that the acquired target object is the first target object, the first image data of the first target object is acquired.

[0113] In this embodiment, the first image data of the first target object can be further obtained by acquiring the image data in the data domain. For example, a high-resolution image of the first target object can be acquired by an unmanned aerial vehicle (UAV) shooting device. If it is determined that a high-resolution image of Beijing City needs to be acquired, the first data domain can be the high-resolution image data of the UAV shooting device, the first target object can be Beijing City, and the first image data can be the high-resolution image of Beijing City acquired by the UAV shooting device, thereby achieving the acquisition of the first image data of the first target object.

[0114] Step S404: converting the first remote sensing image data into second remote sensing image data of the first area object, wherein the second remote sensing image data belongs to the second data domain.

[0115] In the technical solution provided in the above step S404 of the present invention, after obtaining the first image data of the first target object, it is necessary to convert the first image data into the second image data of the first target object. For example, after obtaining the high-resolution image data of Beijing by the drone shooting equipment, it is necessary to convert the high-resolution image data of Beijing into the second image data of Beijing. The second image data can be data belonging to Google satellite map.

[0116] In the above embodiment, the first image data is converted into second image data of the first target object, wherein the second image data belongs to the second data domain. For example, the first image data with the source domain B is converted into image data with the target domain A. The second image data can be image data of the first target object with the style of the target domain A.

[0117] In the above embodiment, the above second image data belongs to the second data domain. In order to realize the conversion of the first image data with the style of the source domain B into the image data with the style of the target domain A, the first image data is optionally converted into the second image data of the first target object. For example, the second data domain is Google satellite image data, and the first image data is high-resolution image data of Beijing. Then, the high-resolution image data of Beijing can be converted into Google satellite image data of Beijing.

[0118] Optionally, the above-mentioned conversion of the first image data into the second image data of the first target object can be achieved through a conversion model, such as a generator network G-Net.

[0119] In the above embodiment, the first image data is converted into second image data of the first target object by generating a network model, wherein G-Net can convert the input random noise into a realistic picture by continuously learning the probability distribution of real data in the training set, that is, the more similar the generated picture is to the picture in the training set, the better.

[0120] Step S406 , performing segmentation processing on the second remote sensing image data based on the target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third remote sensing image data of the second regional object, and the third remote sensing image data belongs to the second data domain.

[0121] In the technical solution provided in the above step S406 of the present invention, after the first image data is converted into the second image data of the first target object, in order to obtain a segmentation result, it is necessary to further perform segmentation processing on the second image data.

[0122] In the above embodiment, the second image data is segmented. Optionally, the second image data is segmented using a target segmentation model to obtain a target segmentation result.

[0123] In this embodiment, the second data domain is a target domain, such as domestic high-resolution images, Google satellite images, Sentinel images, etc.

[0124] In this embodiment, the third remote sensing image data belongs to the second data domain, and the third remote sensing image data is labeled data.

[0125] In the above embodiment, the target segmentation model can be obtained by training the third image data based on the second target object. For example, the target segmentation model is trained with the image data of the second data domain so that the target segmentation model can perform segmentation processing on the image of the second data domain.

[0126] In the above embodiment, the target segmentation model can be a deep convolutional encoder-decoder architecture (Segmantic Segmentation, abbreviated as SegNet) for image segmentation. SegNet achieves image segmentation by classifying each pixel in the image and identifying the category of each pixel. The upsampling of SegNet is reverse maximum pooling. When downsampling, SegNet will record the position of the maximum pooling pointer and fill the rest with 0.

[0127] In the above embodiment, the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain. For example, the second data domain is the target domain A, and SegNet is trained based on the third image data having the style of the target domain A, and then the second image data is segmented based on the SegNet obtained by the above method to obtain the segmentation result.

[0128] Example 2

[0129] The preferred implementation of the above method of this embodiment is further introduced below.

[0130] Remote sensing imagery has been widely studied and applied in numerous fields, including land resources, water conservancy monitoring, and ecological assessment. In recent years, with the increase in remote sensing data from satellites and drones, the demand for automated and intelligent remote sensing image analysis has also gradually increased. Differences in imagery from different satellites and drones result in significant differences in remote sensing imagery across various morphological domains, such as the domestic high-resolution imagery series, Google satellite imagery, and Sentinel imagery series. Eliminating these differences in remote sensing data domains and improving automated analysis are pressing challenges in remote sensing imagery that need to be addressed.

[0131] In related technologies, the initialization model trained in the source domain is migrated and applied to the semantic segmentation task of remote sensing images, namely the target domain, to improve the reusability of the deep learning network model. This proposal is also a classic supervised pre-training and supervised fine-tuning training strategy, which has the problem of requiring supervised labeled data and high labor costs.

[0132] In response to the problem of poor segmentation of data from different domains in the remote sensing field, that is, the domain generalization problem, this embodiment proposes a remote sensing segmentation domain adaptation method based on image generation. Through unsupervised training, Cycle-GAN is used to perform domain migration to perform migration transformation between different domain data, thereby eliminating data differences between different domains, saving the cost of manual labeling of different domain data, and improving the segmentation effect of remote sensing images on different domain data. This embodiment migrates the unlabeled first image data of the first data domain into the second image data of the second data domain, and then sends the second image data to the segmentation model for the second data domain for segmentation processing to obtain the segmentation result. It is further described in detail below.

[0133] In recent years, with the increasing availability of remote sensing data from satellites and drones, the demand for automated and intelligent analysis of remote sensing imagery has also grown. Differences in imagery from different satellites and drones result in significant differences in remote sensing imagery across various morphological domains, such as the Chinese high-resolution imagery series, Google satellite imagery, and Sentinel imagery. Eliminating these differences in remote sensing data domains and improving automated analysis are pressing challenges in remote sensing imagery that need to be addressed.

[0134] In related technologies, there are two remote sensing images from different domains: domain A (source domain, such as Xinjiang, corresponding to Google Maps) and domain B (target domain, such as Beijing, corresponding to high-resolution images). Domain A has labeled data and can perform supervised segmentation training; domain B only has images and no labeled data. The segmentation model SegANet trained in domain A is directly applied to domain B. Due to the large differences between domains A and B, the segmentation effect of domain B is very poor.

[0135] In this application, the mutual migration generation between the two domains A and B is achieved through unsupervised non-image pair image migration technology, thereby eliminating the problem of poor segmentation effect caused by domain differences.

[0136] Figure 5 is a schematic diagram of a remote sensing segmentation domain adaptive method based on image generation according to an embodiment of the present invention, such as Figure 5 As shown, the method includes the following steps:

[0137] S1, acquiring image data of data domain B and image data of data domain A, and migrating the image data of data domain B into data domain B2 in the style of data domain A.

[0138] In this embodiment, the image data of data domain B is unlabeled data, and the image data of data A is labeled image data. The image data of data domain B is converted into data domain B2 with the style of data domain A through the generation network G-Net, that is, the image data of data domain B is converted into data domain B2 with the style of data domain A through migration.

[0139] S2 uses the high-frequency and low-frequency reconstruction G-Net network to reconstruct the cycle-consistent adversarial network Cycle-GAN.

[0140] In this embodiment, in the process of converting image data with data domain B and image data with data domain A by the cycle-consistency adversarial network Cycle-GAN, the predicted image is first preprocessed into low-frequency and high-frequency parts. The reconstructed G-Net network structure has two structural branches, which respectively predict the high-frequency and low-frequency information of the image. Finally, the predicted high-frequency and low-frequency information are reconstructed back to the original image, thereby ensuring that the details of the high-frequency part are realistic and eliminating the brightness and cloud images.

[0141] S3, use the image data of data domain A to train the segmentation model SegANet, and input the data domain B2 migrated to the style of data domain A into the segmentation model SegANet.

[0142] In this embodiment, the image data of data domain A is labeled image data. The segmentation model SegANet is trained in a supervised manner so that it can segment data images with the style of data domain A. By sending data domain B2 into the segmentation model SegANet, the segmentation effect of data domain B can be improved, and the experimental effect can be improved in scenarios such as land feature classification and change detection.

[0143] Through the above steps S1 to S3 of the present application, the image data of data domain B and the image data of data domain A are obtained, and the image data of data domain B is migrated into data domain B2 in the style of data domain A; the cycle consistency adversarial network Cycle-GAN is reconstructed using a high-frequency and low-frequency reconstruction type G-Net network; the segmentation model SegANet is trained using the image data of data domain A, and the data domain B2 migrated to the style of data domain A is input into the segmentation model SegANet to obtain a segmentation result. Through an unsupervised image segmentation method, the segmentation effect of the first image data is improved, the technical problem of poor segmentation effect of image data in different data domains is solved, and the technical problem of improving the effect of segmentation of image data in different data domains is achieved.

[0144] In the above Figure 5In the image generation-based remote sensing segmentation domain adaptation method shown in FIG, the cycle consistency adversarial network Cycle-GAN includes a generator network G-Net and a discriminator network, which realizes the mutual conversion between different data domains through a cycle.

[0145] The generator network G-Net learns the distribution of data in an unsupervised manner and simulates existing data to generate pictures, texts, etc. of the same type. In this application, the generator network G-Net processes remote sensing images of various morphological domains. Since remote sensing images are affected by light and fog, remote sensing images at the same location and time often have different forms of expression. To address this problem, a high-frequency and low-frequency reconstruction G-Net network is used to replace the original direct output G-Net network. By processing the image with the reconstructed G-Net network, the details of the high-frequency part of the image are ensured to be realistic, and the images of bright spots and fog are eliminated.

[0146] Figure 6 Schematic diagram of a processing process of a G-Net network according to an embodiment of the present invention, Figure 6 As shown, the process includes the following:

[0147] The predicted image is first preprocessed into low-frequency and high-frequency parts. The reconstructed G-Net network structure has two structural branches, which predict the high-frequency and low-frequency information of the image respectively. Finally, the predicted high-frequency and low-frequency information are reconstructed back into the original image, thereby ensuring the realistic details of the high-frequency part and eliminating the influence of brightness and fog.

[0148] After the remote sensing images of various morphological domains are processed by the above-mentioned high-frequency and low-frequency reconstruction generator network G-Net, the influence of brightness and fog is eliminated, and the domain migration is carried out by using the cycle consistency network to carry out migration transformation between different data domains, while improving the segmentation effect of remote sensing images on different domain data, thereby solving the technical problem of poor segmentation effect of image data in different data domains.

[0149] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0150] Through the description of the above embodiments, those skilled in the art can clearly understand that the image processing method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0151] Example 3

[0152] According to an embodiment of the present invention, there is also provided a method for implementing the above Figure 2 An image processing device according to the image processing method shown.

[0153] Figure 7 FIG. 1 is a schematic diagram of an image processing device according to an embodiment of the present invention. Figure 7 As shown, the image processing device 70 may include: a first acquiring unit 71 , a first converting unit 72 , and a first segmenting unit 73 .

[0154] The first acquiring unit 71 is configured to acquire first image data of a first target object, wherein the first image data belongs to a first data domain.

[0155] The first conversion unit 72 is configured to convert the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain.

[0156] The first segmentation unit 73 is configured to segment the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0157] It should be noted that the first acquisition unit 71, the first conversion unit 72, and the first reconstruction and segmentation unit 73 correspond to steps S202 to S206 in Example 1. The examples and application scenarios implemented by these three units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above units, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0158] According to an embodiment of the present invention, there is also provided a method for implementing the above Figure 3 An image processing device according to the image processing method shown.

[0159] Figure 8 Schematic diagram of a data processing device provided from the human-computer interaction side according to an embodiment of the present disclosure. Figure 8 As shown, the data processing device 80 may include: an input unit 81 and a first display unit 82 .

[0160] The input unit 81 is configured to respond to a first input instruction applied to the operation interface and input first image data of a first target object, wherein the first image data belongs to a first data domain.

[0161] The first display unit 82 is used to respond to the segmentation processing instruction acting on the operation interface and display the target segmentation result on the operation interface, wherein the target segmentation result is obtained by segmenting the second image data of the first target object based on the target segmentation model, the second image data is converted from the first image data, and the second image data belongs to the second data domain. The target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0162] It should be noted that the input unit 81 and the first display unit 82 correspond to steps S302 to S304 in Example 3. The examples and application scenarios implemented by the two units and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned units, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0163] According to an embodiment of the present invention, there is also provided a method for implementing the above Figure 4 A data processing device according to the image processing method shown.

[0164] Figure 9 Schematic diagram of a data processing device provided in the context of style migration from remote sensing images according to an embodiment of the present disclosure. Figure 9 As shown, the image processing device 90 may include: a second acquiring unit 91 , a second converting unit 92 and a second segmenting unit 93 .

[0165] The second acquiring unit 91 is configured to acquire first remote sensing image data of a first area object, wherein the first remote sensing image data belongs to a first data domain.

[0166] The second conversion unit 92 is configured to convert the first remote sensing image data into second remote sensing image data of the first area object, wherein the second remote sensing image data belongs to the second data domain.

[0167] The second segmentation unit 93 is used to segment the second remote sensing image data based on the target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third remote sensing image data of the second area object, and the third remote sensing image data belongs to the second data domain.

[0168] It should be noted that the second acquisition unit 91, second conversion unit 92, and second segmentation unit 93 described above correspond to steps S402 to S406 in Example 2. The examples and application scenarios implemented by these three units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above units, as part of the apparatus, can be run in the computer terminal 10 provided in Example 1.

[0169] In the data processing device of this embodiment, by migrating the unlabeled first image data of the first data domain into the second image data of the second data domain, and then sending the second image data to the segmentation model for the second data domain for segmentation processing, the purpose of improving the segmentation effect of the first image data is achieved, thereby achieving the technical effect of improving the effect of segmenting image data in different data domains, and further solving the technical problem of poor segmentation effect of image data in different data domains.

[0170] Example 4

[0171] An embodiment of the present invention may provide an image processing system, which may include a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0172] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0173] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the vulnerability detection method of the application: obtaining first image data of the first target object, wherein the first image data belongs to the first data domain; converting the first image data into second image data of the first target object, wherein the second image data belongs to the second data domain; performing segmentation processing on the second image data based on the target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0174] Optionally, Figure 10 1 is a block diagram of a computer terminal according to an embodiment of the present invention. Figure 10As shown, the computer terminal A may include: one or more (only one is shown in the figure): a processor 1002 , a memory 1004 , and a transmission device 1006 .

[0175] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and apparatus in the embodiments of the present invention. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the aforementioned data processing method. The memory may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and such remote memory may be connected to the computer terminal A via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and apparatus in the embodiments of the present invention. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the aforementioned system vulnerability attack detection method. The memory may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and such remote memory may be connected to the computer terminal A via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain first image data of the first target object, wherein the first image data belongs to the first data domain; convert the first image data into second image data of the first target object, wherein the second image data belongs to the second data domain; segment the second image data based on the target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0177] Optionally, the processor may further execute program code of the following steps: converting the first image data into the second image data based on the sub-data conversion model, wherein the data conversion model to which the sub-data conversion model belongs is used to convert the first image data and the third image data into each other.

[0178] Optionally, the processor may further execute program code of the following steps: acquiring high-frequency image information and low-frequency image information of the first image data based on the sub-transformation model; and generating second image data based on the high-frequency image information and the low-frequency image information.

[0179] Optionally, the processor may further execute program code of the following steps: reconstructing the high-frequency image information and the low-frequency image information to obtain second image data.

[0180] Optionally, the processor may further execute program code of the following steps: eliminating sub-image data in the first image data based on the sub-transformation model to obtain second image data, wherein the sub-image data is image data generated by illumination information and / or cloud information.

[0181] Optionally, the processor may further execute program code for the following steps: displaying a classification result of the first target object on the operation interface based on the target segmentation result; and / or displaying a detection result of the first target object on the operation interface based on the target segmentation result.

[0182] Optionally, the processor may further execute program code of the following steps: determining indication information of each sub-segmentation result, wherein the indication information is used to indicate the type of each sub-segmentation result; and displaying each sub-segmentation result on the operation interface according to the indication information.

[0183] Optionally, the processor may further execute program code of the following steps: displaying each sub-segmentation result on the operation interface according to the area corresponding to each sub-segmentation result in the target area.

[0184] Optionally, the processor may also execute the program code of the following steps: responding to a selection operation instruction acting on multiple sub-segmentation results, selecting a target sub-segmentation result from the multiple sub-segmentation results; responding to an editing operation instruction acting on the target sub-segmentation result, performing an editing operation on the target sub-segmentation result, obtaining an editing result, and displaying the editing result on the operation interface.

[0185] Optionally, the processor may further execute program code of the following steps: obtaining adjustment information, wherein the adjustment information includes information used to characterize the accuracy of the target segmentation result; and adjusting the target segmentation model based on the adjustment information.

[0186] As an optional example, the above-mentioned processor can also execute the program code of the following steps: responding to a first input instruction on the operation interface, inputting the first image data of the first target object, wherein the first image data belongs to the first data domain; responding to a segmentation processing instruction on the above-mentioned operation interface, displaying the target segmentation result on the operation interface, wherein the target segmentation result is obtained by segmenting the second image data of the first target object based on the target segmentation model, the second image data is converted from the first image data, the second image data belongs to the second data domain, and the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0187] As an optional example, the above-mentioned processor can also execute the program code of the following steps: obtaining first remote sensing image data of a first area object, wherein the first remote sensing image data belongs to a first data domain; converting the first remote sensing image data into second remote sensing image data of the first area object, wherein the second remote sensing image data belongs to a second data domain; performing segmentation processing on the second remote sensing image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third remote sensing image data of a second area object, and the third remote sensing image data belongs to the second data domain.

[0188] An embodiment of the present invention provides an image processing solution. The solution comprises acquiring first image data of a first target object, wherein the first image data belongs to a first data domain; converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; and performing segmentation processing on the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of the second target object, wherein the third image data belongs to the second data domain. This improves the effect of segmenting images in different data domains and solves the technical problem of poor segmentation effect of image data in different data domains.

[0189] It can be understood by those skilled in the art that Figure 10 The structure shown is for illustration only, and the computer terminal A may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 10 It does not limit the structure of the above-mentioned computer terminal A. For example, the computer terminal A may also include Figure 10 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 10 Different configurations shown.

[0190] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0191] Example 5

[0192] The embodiment of the present invention further provides a computer-readable storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image processing method provided in the first embodiment.

[0193] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0194] Optionally, in this embodiment, the above-mentioned computer-readable storage medium is configured to store program code for performing the following steps: obtaining first image data of a first target object, wherein the first image data belongs to a first data domain; converting the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; performing segmentation processing on the second image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of the second target object, and the third image data belongs to the second data domain.

[0195] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: converting the first image data into the second image data based on the sub-data conversion model, wherein the data conversion model to which the sub-data conversion model belongs is used to convert the first image data and the third image data into each other.

[0196] Optionally, the computer-readable storage medium is further configured to store program code for executing the following steps: acquiring high-frequency image information and low-frequency image information of the first image data based on the sub-transformation model; and generating second image data based on the high-frequency image information and the low-frequency image information.

[0197] Optionally, the computer-readable storage medium is further configured to store program codes for executing the following steps: reconstructing the high-frequency image information and the low-frequency image information to obtain second image data.

[0198] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: eliminating sub-image data in the first image data based on the sub-transformation model to obtain second image data, wherein the sub-image data is image data generated by lighting information and / or cloud information.

[0199] Optionally, the computer-readable storage medium is also configured to store program code for performing the following steps: displaying the classification result of the first target object on the operation interface based on the target segmentation result; and / or displaying the detection result of the first target object on the operation interface based on the target segmentation result.

[0200] Optionally, the computer-readable storage medium is further configured to store program code for executing the following steps: determining indication information for each sub-segmentation result, wherein the indication information is used to indicate the type of each sub-segmentation result; and displaying each sub-segmentation result on the operation interface according to the indication information.

[0201] Optionally, the computer-readable storage medium is further configured to store program codes for executing the following steps: displaying each sub-segmentation result on the operation interface according to the area corresponding to each sub-segmentation result in the target area.

[0202] Optionally, the computer-readable storage medium is also configured to store program codes for executing the following steps: responding to a selection operation instruction acting on multiple sub-segmentation results, selecting a target sub-segmentation result from the multiple sub-segmentation results; responding to an editing operation instruction acting on the target sub-segmentation result, performing an editing operation on the target sub-segmentation result, obtaining an editing result, and displaying the editing result on the operation interface.

[0203] Optionally, the computer-readable storage medium is further configured to store program code for executing the following steps: obtaining adjustment information, wherein the adjustment information includes information for characterizing the accuracy of the target segmentation result; and adjusting the target segmentation model based on the adjustment information.

[0204] As an optional example, the computer-readable storage medium is also configured to store program code for performing the following steps: in response to a first input instruction acting on an operation interface, inputting first image data of a first target object, wherein the first image data belongs to a first data domain; in response to a segmentation processing instruction acting on the operation interface, displaying a target segmentation result on the operation interface, wherein the target segmentation result is obtained by segmenting the second image data of the first target object based on a target segmentation model, the second image data is converted from the first image data, the second image data belongs to the second data domain, and the target segmentation model is trained based on the third image data of the second target object, and the third image data belongs to the second data domain.

[0205] As an optional example, the computer-readable storage medium is also configured to store program code for executing the following steps: obtaining first remote sensing image data of a first area object, wherein the first remote sensing image data belongs to a first data domain; converting the first remote sensing image data into second remote sensing image data of the first area object, wherein the second remote sensing image data belongs to a second data domain; performing segmentation processing on the second remote sensing image data based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third remote sensing image data of a second area object, and the third remote sensing image data belongs to the second data domain.

[0206] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0207] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0208] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0209] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0210] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0211] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0212] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire first image data of a first target object, wherein the first image data belongs to a first data domain and is image data not labeled with a target label; converting the high-frequency image information and the low-frequency image information of the first image data into second image data of the first target object, wherein the second image data belongs to a second data domain; The second image data is segmented based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on third image data of a second target object, the third image data belongs to the second data domain, and is the labeled image data.

2. The method according to claim 1, characterized in that Converting the high-frequency image information and the low-frequency image information of the first image data into second image data of the first target object includes: Based on a sub-data conversion model, high-frequency image information and low-frequency image information of the first image data are converted into the second image data, wherein the data conversion model to which the sub-data conversion model belongs is used to convert the first image data and the third image data into each other.

3. The method according to claim 2, characterized in that Converting the high-frequency image information and the low-frequency image information of the first image data into the second image data based on the sub-data conversion model includes: acquiring high-frequency image information and low-frequency image information of the first image data based on the sub-data conversion model; The second image data is generated based on the high-frequency image information and the low-frequency image information.

4. The method according to claim 3, characterized in that Generating the second image data based on the high-frequency image information and the low-frequency image information includes: The high-frequency image information and the low-frequency image information are reconstructed to obtain the second image data.

5. The method according to claim 2, characterized in that Converting the high-frequency image information and the low-frequency image information of the first image data into the second image data based on the sub-data conversion model includes: Sub-image data in the first image data is eliminated based on the sub-data conversion model to obtain the second image data, wherein the sub-image data is image data generated by lighting information and / or cloud information.

6. The method according to claim 1, characterized in that The method further comprises: Displaying the classification result of the first target object on an operation interface based on the target segmentation result; and / or The detection result of the first target object is displayed on the operation interface based on the target segmentation result.

7. The method according to claim 1, characterized in that The target segmentation result includes a plurality of sub-segmentation results, and the method further includes: Determining indication information of each of the sub-segmentation results, wherein the indication information is used to indicate a type of each of the sub-segmentation results; Each of the sub-segmentation results is displayed separately on the operation interface according to the instruction information.

8. The method according to claim 7, characterized in that The first target object is a target area, and the method further includes: Each of the sub-segmentation results is displayed on the operation interface according to the area corresponding to each of the sub-segmentation results in the target area.

9. The method according to claim 7, characterized in that The method further comprises: In response to a selection operation instruction acting on the plurality of sub-segmentation results, select a target sub-segmentation result from the plurality of sub-segmentation results; In response to an editing operation instruction acting on the target sub-segmentation result, an editing operation is performed on the target sub-segmentation result to obtain an editing result, and the editing result is displayed on the operation interface.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Acquiring adjustment information, wherein the adjustment information includes information used to characterize the accuracy of the target segmentation result; The target segmentation model is adjusted based on the adjustment information.

11. An image processing method, characterized in that: include: In response to a first input instruction applied to the operation interface, first image data of a first target object is input, wherein the first image data belongs to a first data domain and is image data not labeled with a target label; In response to a segmentation processing instruction applied on the operation interface, a target segmentation result is displayed on the operation interface, wherein the target segmentation result is obtained by segmenting the second image data of the first target object based on a target segmentation model, the second image data is obtained by converting high-frequency image information and low-frequency image information of the first image data, the second image data belongs to a second data domain, and the target segmentation model is obtained by training based on the third image data of the second target object, the third image data belongs to the second data domain, and is the labeled image data.

12. An image processing method, characterized in that: include: Acquire first remote sensing image data of a first area object, wherein the first remote sensing image data belongs to a first data domain and is remote sensing image data not labeled with a target label; converting the high-frequency image information and the low-frequency image information of the first remote sensing image data into second remote sensing image data of the first area object, wherein the second remote sensing image data belongs to a second data domain; The second remote sensing image data is segmented based on a target segmentation model to obtain a target segmentation result, wherein the target segmentation model is trained based on the third remote sensing image data of the second area object, the third remote sensing image data belongs to the second data domain, and is the labeled remote sensing image data.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed by a processor, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

Images