Image processing method, storage medium and computer terminal

Abstract

The present application discloses an image processing method, a storage medium and a computer terminal. The method comprises: acquiring a first image and a second image, wherein the first image and the second image are images acquired from the same area at different times; performing change detection on the first image and the second image to determine an initial spot, wherein the initial spot is used to characterize the area where the image changes in the first image and the second image; performing image recognition on the first image and the second image to determine the target type corresponding to the initial spot; filtering the initial spot based on the target type to obtain a target spot, wherein the type corresponding to the target spot is a preset type. The present application solves the technical problem of low accuracy of image recognition in the related art.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, a storage medium and a computer terminal. Background Art

[0002] At present, the cultivated land change detection algorithm based on deep learning algorithm usually adopts the same paradigm as the general change detection algorithm, and only predicts the cultivated land change area of ​​concern. For different scenarios, such as returning farmland to forest, abandoned farmland, excavation of farmland, and restoration of farmland, it is necessary to build the required data set for training the model in a targeted manner. However, the number of samples used to build the data set is small, so it is difficult to obtain a model with high accuracy, and thus it is difficult to improve the accuracy of image recognition.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide an image processing method, a storage medium, and a computer terminal to at least solve the technical problem of low accuracy of image recognition in the related art.

[0005] According to one aspect of an embodiment of the present application, there is provided an image processing method, including: acquiring a first image and a second image, wherein the first image and the second image are images acquired from the same area at different times; performing change detection on the first image and the second image to determine an initial spot, wherein the initial spot is used to characterize an area where changes have occurred in the first image and the second image; performing image recognition on the first image and the second image to determine a target type corresponding to the initial spot; filtering the initial spot based on the target type to obtain a target spot, wherein the type corresponding to the target spot is a preset type.

[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, including: acquiring a first cultivated land image and a second cultivated land image, wherein the first cultivated land image and the second cultivated land image are images acquired from the same cultivated land area at different times; performing change detection on the first cultivated land image and the second cultivated land image to determine initial cultivated land patches, wherein the initial cultivated land patches are used to characterize areas where cultivated land images in the first cultivated land image and the second cultivated land image have changed; performing image recognition on the first cultivated land image and the second cultivated land image to determine target types corresponding to the initial cultivated land patches; filtering the initial cultivated land patches based on the target types to obtain target cultivated land patches, wherein the type corresponding to the target cultivated land patches is the cultivated land type.

[0007] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the image processing method in any one of the above embodiments.

[0008] According to another aspect of an embodiment of the present application, a computer terminal is further provided, including: a processor and a memory, wherein the processor is used to run a program stored in the memory, wherein the image processing method in any one of the above embodiments is executed when the program is run.

[0009] Through the above steps, first, the first image and the second image are acquired, wherein the first image and the second image are images acquired by capturing the same area at different times; the first image and the second image are subjected to change detection to determine the initial spots, wherein the initial spots are used to characterize the areas where the images in the first image and the second image have changed; the first image and the second image are subjected to image recognition to obtain the target type corresponding to the initial spots, and the initial spots are filtered based on the target type to obtain the target spots, wherein the type corresponding to the target spots is a preset type, thereby achieving the purpose of improving the image recognition accuracy. It is easy to notice that the initial spots can be classified so as to filter the initial spots by the target type to obtain the target spots, and the preset type of target spots can be retained. Since the regional information is integrated into the recognition process, the accuracy of image recognition can be further improved, so as to at least solve the technical problem of low accuracy of image recognition in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present application;

[0012] Figure 2 is a flowchart of an image processing method according to an embodiment of the present application;

[0013] Figure 3 is a schematic diagram of an optional interactive interface according to an embodiment of the present application;

[0014] Figure 4 is a schematic diagram of an optional interactive interface according to an embodiment of the present application;

[0015] Figure 5 is a schematic diagram of a general binary classification change detection network according to an embodiment of the present application;

[0016] Figure 6 is a schematic structural diagram of a spot classifier according to an embodiment of the present application;

[0017] Figure 7 is a flowchart of another image processing method according to an embodiment of the present application;

[0018] Figure 8 is a flowchart of another image processing method according to an embodiment of the present application;

[0019] Fig. 9 is a schematic diagram of an image processing device according to an embodiment of the present application;

[0020] Fig.10 is a schematic diagram of another image processing device according to an embodiment of the present application;

[0021] Fig.11 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] With the development of satellite and airborne sensors, it is becoming increasingly convenient to obtain remote sensing, aerial images and video data, and these data have been widely used in urban planning, agriculture and forestry, etc. Therefore, it is increasingly important to analyze the information in remote sensing and aerial images. However, labeling these data requires a lot of human resources, which greatly increases the cost of using remote sensing data. In addition, when remote sensing data is applied to different scenarios, due to different task requirements, the labeling rules for data are also different, which makes it extremely difficult to reuse existing labeled data. Cultivated land change detection is an important task in resource change monitoring. At present, manual visual interpretation is generally used. With the rapid increase in the amount of remote sensing data, traditional manual calibration methods are difficult to support the explosive growth of tasks and demand workloads.

[0025] General change detection algorithms based on deep learning are widely used in remote sensing image change detection tasks to predict the changed areas between the input images before and after. Such algorithms usually use a twin network structure, compare and interact the features of the two images, and output the changed areas that do not contain the ground object category. General change detection datasets contain more types of images and more comprehensive data than specific change scene datasets, such as farmland change detection datasets. Therefore, general change detection models are usually used as pre-trained models for specific types of change detection tasks.

[0026] Existing farmland change detection algorithms based on deep learning algorithms usually adopt the same paradigm as general change detection algorithms, and only predict the farmland change areas of interest. For different scenarios, such as returning farmland to forest, abandoned farmland, excavation of farmland, and restoration of farmland, it is necessary to build the required data sets for specific scenarios to train the model. With the increasing number of remote sensing interpretation tasks, this method of redesigning algorithms for specific scenarios has seriously hindered the efficiency of image interpretation. In addition, in scenarios containing multiple types of farmland changes, it is difficult to predict the type of farmland change for each patch.

[0027] In order to solve the above problems, the present application provides an image processing method, which can reuse the general change detection model in various scenes to reduce the detection cost, and for scenes containing multiple types of cultivated land changes, can accurately predict the cultivated land change type of each map block.

[0028] Example 1

[0029] According to an embodiment of the present application, an embodiment of an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuits may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). The data processing circuits may be used as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the image processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned image processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0035] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the above-described computer device (or mobile device).

[0036] Under the above operating environment, this application provides Figure 2 The image processing method shown. Figure 2 is a flow chart of an image processing method according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0037] Step S202: Acquire a first image and a second image.

[0038] The first image and the second image are images acquired from the same area at different times.

[0039] The first image and the second image mentioned above may be remote sensing images, aerial images taken by a drone, radar images, or images acquired by a camera, but are not limited thereto.

[0040] The first image and the second image may be images acquired at different times from the same area, that is, the first image and the second image both contain the same area. It should be noted that the acquisition positions of the first image and the second image may be the same or different. In order to improve the accuracy of image recognition, the first image and the second image may be acquired at the same acquisition position at different time points, but this is not limited to this.

[0041] The acquisition time of the first image may be earlier than the acquisition time of the second image. The acquisition time of the first image may also be later than the acquisition time of the second image. Specifically, it can be set according to actual conditions.

[0042] The above-mentioned image processing method can be applied to the process of detecting the change of the coverage type of the ground object, wherein the change of the coverage type of the ground object includes the mutual transformation between various types such as groundbreaking, greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, arable land without crops, arable land with crops, woodland, water area, hardened open space, green space, natural bare land, and sports field. By detecting the first image and the second image, it can be detected whether the coverage type of the ground object has changed. Furthermore, it can also be detected according to the type change that the user is concerned about. If the user is concerned about the change of the cultivated land type to the greenhouse type, then when the cultivated land type is changed to the greenhouse type, the user can be reminded that the coverage type of the ground object has changed.

[0043] In an optional embodiment, the first image and the second image may be taken by a satellite or a drone and transmitted to a server via a network, processed by the server, and the first image and the second image may be displayed to a user at the same time, such as Figure 3 As shown, the first image can be input into the early image acquisition frame, and the second image can be input into the late image acquisition frame; in another optional embodiment, the first image and the second image can be taken by a satellite or a drone, and actively uploaded to the server by the user and processed by the server. Figure 4 As shown, the user can upload the first image and the second image to the server by clicking the "Upload Early Image" button and the "Upload Later Image" button in the interactive interface, or by dragging the first image and the second image into two dotted boxes respectively. Moreover, the images uploaded by the user can be displayed in the early image acquisition box and the late image acquisition box for the user to view or modify. The server here can be a server deployed locally or a server deployed in the cloud.

[0044] Step S204: Perform change detection on the first image and the second image to determine an initial image spot.

[0045] The initial image spots are used to represent regions where images in the first image and the second image have changed.

[0046] In an optional embodiment, a universal change detection model can be used to perform universal change detection on the first image and the second image in order to determine the initial patch corresponding to the area where the change occurs. It should be noted that the universal change refers to the change in the type of ground coverage that occurs, which usually includes the conversion between multiple categories such as groundbreaking, greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, arable land without crops, arable land with crops, woodland, water area, hardened open space, green space, natural bare land, and sports fields. Since it covers most types of ground changes, that is, its data coverage is wide and the data volume is large, the universal change detection model can be the currently used twin structure-based change detection network. Specifically, the twin structure-based change detection network can be an efficient net+cascade bifpn (universal binary classification change detection network) network structure.

[0047] In another optional embodiment, you can first build Figure 5 The general binary classification change detection network shown in the figure uses efficient-b0 (compound scaling network) as the backbone network to extract features corresponding to the first image and features corresponding to the second image. The features corresponding to the first image and the features corresponding to the second image can be spliced ​​and input into the cascadebifpn (cascade feature layer) structure for feature interaction to obtain target features. The target features can be input into the decoder to obtain the final prediction result, that is, the initial image patch corresponding to the image change area in the first image and the second image.

[0048] In another optional embodiment, in the process of training the above-mentioned general change detection model, the training data can be divided into 14,932 pairs of before and after images (the above-mentioned first image and second image) as training sets, 3,494 pairs of before and after images as test sets, the image size is 1024*1024, and the annotations in the change areas in all images are carried out according to the mutual conversion rules between multiple categories such as groundbreaking, greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, non-crop arable land, crop arable land, forest land, water area, hardened vacant land, green space, natural bare land, and sports field. Use an initial learning rate of 0.005, a batch size of 32, and 500,000 iterations to obtain a general change detection model with high accuracy.

[0049] In another optional embodiment, since a single patch obtained by general change detection may contain multiple types of objects, and the patches in the patch classification data set are patches that only contain a single type of objects, there is a huge difference in the patch distribution between the training patches and the test scene. Therefore, the patches obtained by the general change can be pre-segmented in an unsupervised manner to ensure that each patch after pre-segmentation contains only a single type of object as much as possible. Pre-segmentation methods include grid methods, superpixel methods, and model methods based on deep learning. The superpixel method that takes into account both performance and speed can be selected as the pre-segmentation method.

[0050] Furthermore, the superpixel method used may be seeds (superpixel segmentation algorithm), and the specific pre-segmentation process may be to divide the original large image spot into a series of small areas composed of pixels with adjacent positions and similar characteristics such as color, brightness, texture, etc. Since most of these small areas retain effective information for further image segmentation and generally do not destroy the boundary information of objects in the image, these small areas can be used as the above-mentioned initial image spots for subsequent image recognition.

[0051] Step S206: performing image recognition on the first image and the second image to determine the target type corresponding to the initial image spot.

[0052] In an optional embodiment, the area where the initial spot is located in the first image can be identified to determine the type of the initial spot in the first image, and the area where the initial spot is located in the second image can be identified to determine the type of the initial spot in the second image.

[0053] Specifically, the residual convolutional neural network (ResNeSt50) can be used to perform image recognition on the first image and the second image to obtain the target type corresponding to the initial spot. Specifically, a mask information fusion branch can be added on the basis of the residual convolutional neural network (ResNeSt50), and the size of the mask can be adjusted according to the characteristics of the output initial spot, so that the mask size can be aligned with the characteristics corresponding to the initial spot to obtain mask information, and a single convolution operation can be used to expand the single-channel mask information to obtain mask features, and then the mask features are added to the features of the initial spot and input into the subsequent network to achieve the purpose of combining the mask information to recognize the image and improve the recognition accuracy.

[0054] Step S208, filtering the initial spots based on the target type to obtain the target spots.

[0055] Among them, the type corresponding to the target spot is the preset type.

[0056] The above preset types are used to indicate the preset change types of the spots, for example, the preset types of spots before and after the changes, which are mainly used to filter out the change types of spots that the user needs to pay attention to. Exemplarily, the preset types can be one or more change types such as cultivated land-cultivated land, cultivated land-others, others-cultivated land, water area-water area, water area-others, others-water area, etc.

[0057] The above-mentioned preset types can also represent the types that the user is interested in. If the target type contains an initial spot of a preset type, the initial spot of the preset type can be directly retained as the target spot. Exemplarily, the preset types can include one or more types such as cultivated land, water area, greenhouse, etc.

[0058] In an optional embodiment, the preset type may be the same type before and after the patch changes, such as cultivated land-cultivated land, water area-water area, etc. If the target type is cultivated land-cultivated land, it means that the target type is the same as the preset type, and the user needs to pay attention to the initial patch of the same type before and after. At this time, the initial patch can be retained as the target patch. If the initial patch type is cultivated land-other, it means that the target type is different from the preset type. At this time, it means that the initial patch is not the initial patch of the same type before and after that the user needs to pay attention to. At this time, the initial patch can be filtered and the remaining patches can be retained as target patches.

[0059] In another optional embodiment, the preset type can be different types before and after the patch changes, such as cultivated land-others, other-cultivated land, etc. If the target type is cultivated land-cultivated land, it means that the target type is different from the preset type, and the user does not need to pay attention to the initial patches of different types before and after. At this time, the initial patches can be filtered and the remaining patches can be retained as target patches; if the target type is cultivated land-others, or other-cultivated land, it means that the target type is the same as the preset type, and the user needs to pay attention to the initial patches of the same type before and after. At this time, the initial patches can be retained as target patches.

[0060] In another optional embodiment, the preset type may be cultivated land or water area. If the target type of the initial spot is cultivated land, it means that the target type of the initial spot is the same as the preset type. At this time, the initial spot can be determined as the target spot and the target spot can be retained.

[0061] Through the above steps, first, the first image and the second image are acquired, wherein the first image and the second image are images acquired by capturing the same area at different times; the first image and the second image are subjected to change detection to determine the initial spots, wherein the initial spots are used to characterize the areas where the images in the first image and the second image have changed; the first image and the second image are subjected to image recognition to obtain the target type corresponding to the initial spots, and the initial spots are filtered based on the target type to obtain the target spots, wherein the type corresponding to the target spots is a preset type, thereby achieving the purpose of improving the image recognition accuracy. It is easy to notice that the initial spots can be classified so as to filter the initial spots by the target type to obtain the target spots, and the preset type of target spots can be retained. Since the regional information is integrated into the recognition process, the accuracy of image recognition can be further improved, so as to at least solve the technical problem of low accuracy of image recognition in the related art.

[0062] Optionally, the target type includes: a first subtype of the initial spot in the first image, and a second subtype of the initial spot in the second image, wherein performing image recognition on the first image and the second image to determine the target type corresponding to the initial spot includes: determining first mask information and second mask information corresponding to the initial spot; performing image recognition on the first mask information and the first image using a spot classifier to obtain a first subtype; performing image recognition on the second mask information and the second image using a spot classifier to obtain a second subtype.

[0063] The first mask information is used to indicate whether each first pixel point in the first image is located within the initial image spot, and the second mask information is used to indicate whether each second pixel point in the second image is located within the initial image spot.

[0064] The above-mentioned first mask information and second mask information are mainly used to describe the regional information of the initial image spot in the first image and the second image, wherein the regional information can be represented in the form of pixels. Specifically, the regional information can be described by the numerical value of the pixels. For example, the mask information may include the numerical value corresponding to each pixel point in the remote sensing image, which may be 0 or 1. Therefore, the area composed of all pixels with a value of 1 can be determined as the area of ​​the initial image spot.

[0065] The first pixel point and the second pixel point mentioned above may be each pixel point in the image. The first pixel point and the second pixel point located in the initial image spot may be represented by 1, and otherwise represented by 0.

[0066] In an optional embodiment, the first mask information can be determined according to the value of each first pixel in the first image, wherein the first mask information may include a first pixel whose value is 1; the second mask information can be determined according to the value of each second pixel in the second image, wherein the second mask information may include a second pixel whose value is 1.

[0067] In an optional embodiment, after determining the initial image spot, first mask information corresponding to the changed area in the first image can be determined based on the initial image spot, so that the area of ​​the initial image spot in the first image can be determined by the first mask information; second mask information corresponding to the changed area in the second image can be determined based on the initial image spot, so that the area of ​​the initial image spot in the second image can be determined by the second mask information.

[0068] By acquiring the first mask information and the second mask information mentioned above, the image information and the region information corresponding to the initial image spot can be fused, and the target type of the initial image spot can be identified based on the fused information to improve the recognition accuracy.

[0069] In an optional embodiment, image recognition can be performed on the first image and the second image based on the first mask information and the second mask information, respectively, to obtain the target type corresponding to the initial spot, wherein the target type includes: a first subtype of the initial spot in the first image, and a second subtype of the initial spot in the second image.

[0070] Specifically, the size of the mask can be adjusted according to the features of the output initial spot in the first image to obtain the above-mentioned first mask information, the first mask information can be feature processed to obtain the first mask feature, the first mask feature can be added to the features of the initial spot on the first image, and input to the subsequent network for image recognition, so as to identify the first subtype of the initial spot in the first image. The size of the mask can be adjusted according to the features of the output initial spot in the second image to obtain the above-mentioned second mask information, the second mask information can be feature processed to obtain the second mask feature, the second mask feature can be added to the features of the initial spot on the second image, and input to the subsequent network for image recognition, so as to identify the second subtype of the initial spot in the second image.

[0071] In another optional embodiment, in order to solve the problem that the initial image spots with a small area are seriously disturbed by the background, the circumscribed rectangle of the image spots can be used in the feature map of the whole image to deduct the features of the image spot area. Considering the problem that small-area images are too small in size in deep features, a feature pyramid structure can be introduced to determine the level of features to be used according to the size of the image spots, that is, for small-area images, the image spot features can be deducted from the shallow features, and for large-area images, the image spot features can be deducted from the deep features, to ensure that the features of the tiny images can have a suitable size in the low-level features, and because of the top-down feature transfer method of the feature pyramid, it can be ensured that the low-level features also contain deep semantic information.

[0072] In the application scenario of transportation, the first image may be an image of a parking space that does not contain a vehicle and is collected in the early stage, and the second image may be an image of a parking space that contains a vehicle and is collected in the later stage. Change detection may be performed on the first image and the second image to determine the initial spot corresponding to the vehicle change area. The area information corresponding to the initial spot in the first image, i.e., the first mask information mentioned above, may be determined. The area information corresponding to the initial spot in the second image, i.e., the second mask information mentioned above, may be determined. The first image and the second image are identified according to the first mask information and the second mask information respectively, and the type of the initial spot in the first image is that there is no vehicle in the parking space, and the type of the initial spot in the second image is that there is a vehicle in the parking space.

[0073] In agricultural and forestry application scenarios, the first image may be a crop-free farmland scene collected in the early stage, and the second image may be a crop-containing farmland scene collected in the later stage. Change detection may be performed on the first image and the second image to determine the initial spot corresponding to the area where changes occur in the farmland scene. The regional information corresponding to the initial spot in the first image, i.e., the first mask information mentioned above, may be determined. The regional information corresponding to the initial spot in the second image, i.e., the second mask information mentioned above, may be determined. Image recognition is performed on the first image and the second image according to the first mask information and the second mask information respectively, and the type of the initial spot in the first image is crop-free farmland, and the type of the initial spot in the second image is crop-containing farmland.

[0074] In the urban planning application scenario, the first image may be a building image corresponding to a building under construction collected in the early stage, and the second image may be a building image corresponding to a completed building collected in the later stage. Change detection may be performed on the first image and the second image to determine the initial spot corresponding to the changed area in the building image. The area information corresponding to the initial spot in the first image, i.e., the first mask information mentioned above, may be determined. The area information corresponding to the initial spot in the second image, i.e., the second mask information mentioned above, may be determined. Image recognition is performed on the first image and the second image according to the first mask information and the second mask information respectively, and the type of the initial spot in the first image is a building under construction, and the type of the initial spot in the second image is a completed building.

[0075] The above-mentioned spot classifier is used to classify the attributes of the initial spots.

[0076] In an optional embodiment, a spot classifier can be used to fuse the first mask information and the image information corresponding to the initial spots in the first image, and image recognition can be performed based on the fused information to obtain a first subtype; a spot classifier can be used to fuse the second mask information and the image information corresponding to the initial spots in the second image, and image recognition can be performed based on the fused information to obtain a second subtype.

[0077] In another optional embodiment, the attributes of the initial spots can be annotated for data sets of multiple categories, such as groundbreaking, greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, arable land without crops, arable land with crops, woodland, water area, hardened open space, green space, natural bare land, and sports fields. This can cover all test scenarios as much as possible while reducing the magnitude of the annotation task. After completing the attribute annotation of all initial spots, the annotated initial spots can be divided into training sets and test sets for training the above-mentioned spot classifier, thereby improving the accuracy of the spot classifier.

[0078] Furthermore, the spots in the binary change detection dataset are annotated according to multiple categories such as groundbreaking, greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, non-crop arable land, crop arable land, woodland, water area, hardened open space, green space, natural bare land, and sports field. And the geometric center of each initial spot is used to cut a 400*400 image and the corresponding mask information from the corresponding image to construct the spot classification dataset, and finally 2,230,032 groups of initial spot training sets and 179,864 pairs of initial spot test sets are obtained. Since the mask information corresponding to the initial spot is added in the process of constructing the training data, the higher the accuracy of the spot classifier obtained through training.

[0079] In yet another optional embodiment, the stochastic gradient descent method may be used to train the spot classifier using a training data set, wherein the initial learning rate may be 0.1, the batch size may be 512, and a total of 50 batches may be trained.

[0080] In the above-mentioned embodiment of the present application, the image spot classifier includes at least: a mask information input module, an image input module, a feature extraction module and an output module, wherein the feature extraction module is respectively connected to the mask information input module and the image input module, and the output module is connected to the feature extraction module.

[0081] The above-mentioned mask information input module can be used to input mask information corresponding to an image, the above-mentioned image input module can be used to input an image that needs to be identified, the feature extraction module can be connected to the mask information input module and the image input module respectively, the feature extraction module can perform convolution operations on the input mask information and the input image respectively to obtain mask features and image features, the mask features and the image features can be input into a target residual block of a plurality of residual blocks, the image features are input into the first residual block of a plurality of residual blocks, each residual block is used to extract features from the input features, and the output module is used to classify the input features to obtain classification results.

[0082] In the above embodiment of the present application, using a spot classifier to perform image recognition on the first mask information and the first image to obtain the first subtype, or using the spot classifier to perform image recognition on the second mask information and the second image to obtain the second subtype includes: using the mask information input module to perform a convolution operation on the first mask information to obtain a first mask feature, or using the mask information input module to perform a convolution operation on the second mask information to obtain a second mask information; using the image input module to perform a convolution operation on the first image to obtain a first image feature, or using the image input module to perform a convolution operation on the second image to obtain a second image feature; using the feature extraction module to extract features from the first mask feature and the first image feature to obtain a first spot feature, or using the feature extraction module to extract features from the second mask feature and the second image feature to obtain a second spot feature; using the output module to classify the first spot feature to obtain the first subtype, or using the output module to classify the second spot feature to obtain the second subtype.

[0083] In an optional embodiment, the mask information input module can be used to perform a convolution operation on the first mask information to obtain a first mask feature, or the mask information input module can be used to perform a convolution operation on the second mask information to obtain a second mask feature, so that the first mask feature and the second mask feature can be applied to the image recognition process of the subsequent network.

[0084] Furthermore, the first image feature can be extracted based on the region information recorded in the first mask feature in the feature extraction module, so as to extract the first spot feature corresponding to the region where the spot is located, or the second image feature can be extracted based on the region information recorded in the second mask feature in the feature extraction module, so as to extract the second spot feature corresponding to the region where the spot is located. The first spot feature corresponding to the first image can be classified using the output module to obtain the first subtype of the initial spot in the first image, and the second spot feature corresponding to the second image can be radically classified using the output module to obtain the second subtype of the initial spot in the second image.

[0085] In the above embodiment of the present application, the feature extraction module includes: multiple residual blocks connected in sequence, wherein the image input module is connected to the first residual block among the multiple residual blocks, and the mask information input module is connected to the target residual block among the multiple residual blocks.

[0086] The target residual block is used to extract the image features using the regional information of the initial spot in the mask feature, so that the features input to the output module are more accurate, and the classification results obtained by the output module are more accurate.

[0087] In an optional embodiment, the image features obtained by the feature extraction module can be input into the first residual block among multiple residual blocks, and the first residual block can be used to extract the image features, and the output features can be input into the second residual block to output the features corresponding to the initial image spots. The mask size can be aligned with the features output by the second residual block using the nearest neighbor method, and then the aligned mask information can be input into the mask information input module. In the mask information input module, the single-channel mask information is expanded to 512 dimensions to obtain the mask features, and then the mask features are added to the features output by the second residual block and input into subsequent residual blocks. This can achieve the integration of the mask area information into the classification process of the initial image spots.

[0088] In the above-mentioned embodiment of the present application, the target residual block is used to determine the target spot feature in the image feature by using the mask feature, and input the target spot feature into the next residual block.

[0089] The above-mentioned target residual block is used to extract the image features by using the mask features, so that the obtained target spot features are more accurate, and the determined target spot features can be input into the next residual block.

[0090] In an optional embodiment, in order to solve the problem that the initial image spot with a small area is seriously disturbed by the background, the circumscribed rectangle of the initial image spot can be used in the feature map of the whole image to extract the features of the image spot area. Specifically, the regional information of the initial image spot in the mask feature can be used to extract the image features. Considering the problem that the size of a small area image spot is too small in the deep feature, a feature pyramid structure can be introduced to determine the level of the feature to be used according to the size of the image spot. The specific level calculation method is as follows:

[0091]

[0092] Among them, S is the diagonal length of the initial image spot, and L is the level of the calculated feature map.

[0093] That is, the smaller spots can be deducted from the shallow features, and the larger spots can be deducted from the deep features. This can ensure that the features of the smaller spots have a suitable size on the low-level features, and the top-down feature transfer of the feature pyramid can ensure that the low-level features also contain deep semantic information.

[0094] In the above embodiment of the present application, the spot classifier also includes: multiple fusion modules and cropping modules connected in sequence, the multiple fusion modules are connected to the multiple residual blocks in a one-to-one correspondence, the cropping module is connected to the multiple fusion modules, the output module is connected to the cropping module, and the number of the multiple fusion modules is determined based on the size of the initial spot.

[0095] Optionally, the method also includes: using multiple fusion modules to fuse the input features to obtain multiple fused features; using the cropping module to crop the multiple fused features to obtain cropped features; using the output module to classify the cropped features to obtain the classification results.

[0096] The number of the above-mentioned multiple fusion modules is determined based on the size of the initial image spot. Each fusion module is used to fuse the input features. The multiple fusion features output by the multiple fusion modules are input to the cropping module. The cropping module is used to crop the multiple fusion features to obtain the cropped features. The output module is used to classify the cropped features to obtain the classification results. The larger the size of the initial image spot, the more corresponding fusion modules there are, and the smaller the size of the initial image spot, the smaller the corresponding fusion modules there are.

[0097] The above-mentioned fusion module is used to fuse the image features and the mask features, and the above-mentioned cropping module is used to crop multiple fused features so that the cropped features contain features of a single category, which is convenient for subsequent classification of the cropped features.

[0098] In an optional embodiment, features can be cropped using superpixels to divide the features corresponding to the original larger initial image spots into a series of features corresponding to regions with adjacent positions and similar features such as color, brightness, and texture, that is, the above-mentioned cropped features. Since the cropped features are features of a single category, the accuracy of the classification results can be improved by classifying the cropped features through an output module.

[0099] like Figure 6 The above is a structural diagram of a spot classifier, where P1, P2, P3, and P4 are multiple fusion modules connected in sequence, and the multiple modules are connected to the multiple residual blocks in a one-to-one correspondence. The size of the features output by each residual block can be different. The multiple fusion modules are used to fuse the features output by other fusion modules with the features output by the residual blocks, and output fused features. After obtaining the fused features, the cropping module can be used to crop the multiple types of features contained in the fused features so that the cropped features contain one type. Finally, the output module is used to classify the cropped features to obtain the classification result of the feature.

[0100] In another optional embodiment, a feature pyramid structure can be introduced to determine the level of features according to the size of the initial image spot. For smaller images, the image spot features can be deducted from the shallow features, and for larger images, the image spot features can be deducted from the deep features, so as to ensure that the features of the tiny images have a suitable size on the low-level features, and the feature pyramid's top-down feature transfer can ensure that the low-level features also contain deep semantic information. That is, the larger the initial image spot, the more levels it corresponds to, and the smaller the initial image spot, the fewer levels it corresponds to.

[0101] In the above embodiment of the present application, the initial image spots are filtered based on the target type to obtain the target image spots, including: determining whether the target type is the same as the preset type, and when the target type is different from the preset type, filtering the initial image spots to obtain the target image spots; when the target type is the same as the preset type, retaining the initial image spots to obtain image spots.

[0102] The preset type may include a first sub-preset type and a second sub-preset type.

[0103] In an optional embodiment, it can be determined whether the target type is the same as the preset type. When the target type is different from the preset type, it means that the area change type corresponding to the initial spot is not the change type that the user is concerned about. At this time, the initial spot can be filtered. When the target type is the same as the preset type, it means that the area change type corresponding to the initial spot is the type that the user is concerned about. At this time, the initial spot can be retained and determined as the target spot to remind the user that the area corresponding to the target spot has changed.

[0104] For example, in agricultural and forestry application scenarios, for the task of detecting cultivated land changes, the initial image patches corresponding to pixels in the change area whose attributes have changed from cultivated land to other and other to cultivated land can be retained, and the cultivated land change type can be output based on the cultivated land change area. That is, the change types belonging to cultivated land-other or other-cultivated land are the change types that users are concerned about, and the above two change types can be set as preset types.

[0105] It should be noted that since the final rule filtering step will only cause the detected spots to decrease but not increase, in order to ensure a sufficient recall rate, a lower confidence threshold is needed to output the initial spots that need to be retained, so that the model used to detect image changes has a sufficiently high recall rate, which is convenient for subsequent training of the model, thereby further improving the accuracy of the model.

[0106] In the above embodiment of the present application, before determining whether the target type is the same as the preset type, the method also includes: outputting multiple types; receiving a type selected from the multiple types; and determining that the selected type is the preset type.

[0107] The above-mentioned multiple types may be types to be determined as needing to be filtered. The above-mentioned multiple types may be pre-set types, and the above-mentioned multiple types may also be types that may not need to be paid attention to and are pre-determined according to the current detection task. Before determining the preset types, the types that may not need to be paid attention to during this detection process may be determined according to the detection tasks to be completed, that is, the above-mentioned multiple types. The user can select the preset types that do not actually need to be paid attention to from the multiple types.

[0108] In an optional embodiment, before determining whether the target type is the same as the preset type, multiple types can be output to the user's client to facilitate the user to determine the type that does not need to be paid attention to from the multiple types, and then receive the user's feedback information, and determine that the selected type is the above-mentioned preset type based on the user's feedback information.

[0109] Exemplarily, the preset type may be a type with the same previous and subsequent types, such as cultivated land-cultivated land, water area-water area, etc. If the target type is cultivated land-cultivated land, it means that the target type is the same as the preset type, and the user does not need to pay attention to the initial spots with the same previous and subsequent types. In this case, the initial spots can be filtered; if the initial spots are cultivated land-others, it means that the target type is different from the preset type. In this case, the initial spots are spots that the user needs to pay attention to, and the initial spots can be retained.

[0110] In another optional embodiment, the above-mentioned preset type can also be a type that the user needs to pay attention to. For example, the user only pays attention to the change from cultivated land to other or the change from other to cultivated land. At this time, the preset type can be set to cultivated land-other or other-cultivated land. When the target type is the same as the preset type, since the target type is the type that the user needs to pay attention to, the initial map spot can be retained to remind the user that changes that need attention have occurred in the area; if the target type is different from the preset type, for example, the target type is a change from water area to open space or from open space to water area, it means that although the area has changed, it is not a change that the user needs to pay attention to. At this time, the initial map spot can be filtered.

[0111] In the above embodiment of the present application, after performing image recognition on the first image and the second image to determine the target type corresponding to the initial image spot, the method also includes: outputting the first image, the second image and the target type; receiving a type feedback result corresponding to the target type, wherein the type feedback result is used to determine whether the target type is correct.

[0112] In an optional embodiment, the first image, the second image and the target type may be output so that the user can determine whether the target type is accurate based on the first image and the second image. If it is accurate, the spot classifier may be trained based on the type feedback result to improve the classification accuracy of the spot classifier. If it is inaccurate, the user may modify the target type and provide feedback on the modified type in the type feedback result so that the spot classifier may be trained based on the type feedback result to improve the classification accuracy of the spot classifier.

[0113] In the above-mentioned embodiment of the present application, outputting the first image, the second image and the target type includes: determining a first display mode corresponding to the first subtype and a second display mode corresponding to the second subtype; marking the initial image spot in the first image according to the first display mode; and marking the initial image spot in the second image according to the second display mode.

[0114] The first display mode and the second display mode may be the same or different. The first display mode and the second display mode may be distinguished in color or texture.

[0115] In an optional embodiment, the first display mode corresponding to the first subtype may be to set the color of the initial spot to yellow, and the second display mode corresponding to the second subtype may be to set the color of the initial spot to green. The first display mode corresponding to the first subtype may be to fill and display the initial spot with a solid line, and the display mode corresponding to the second subtype may be to fill and display the initial spot with a dotted line.

[0116] In the above embodiment of the present application, after performing change detection on the first image and the second image to determine the initial pattern spot, the method also includes: outputting the first image, the second image and the initial pattern spot; receiving a pattern spot feedback result corresponding to the initial pattern spot, wherein the pattern spot feedback result is obtained by modifying the initial pattern spot based on the first image and the second image; and determining the first mask information and the second mask information corresponding to the pattern spot feedback result.

[0117] In an optional embodiment, the first image, the second image, and the initial image spot can be output, so that the user can adjust the coverage area of ​​the initial image spot according to the first image and the second image, so that the initial image spot can cover the changed area in the first image and the second image, and can also avoid the initial image spot from covering the unchanged area, thereby improving the accuracy of the initial image spot. After the initial image spot is modified to obtain the image spot feedback result, the first mask information and the second mask information can be determined according to the modified initial image spot in the image spot feedback result, so that subsequent image recognition can be performed according to the first mask information and the second mask information, which is conducive to improving the accuracy of subsequent image recognition.

[0118] In another optional embodiment, a model for detecting changes between the first image and the second image may be trained based on the spot feedback result to improve the accuracy of the model so that the initial spots obtained using the model are more accurate.

[0119] In the above embodiment of the present application, after determining the first mask information and the second mask information corresponding to the initial image spot, the method also includes: outputting the first image, the second image, the initial image spot, the first mask information and the second mask information; receiving a first mask feedback result corresponding to the first mask information, and a second mask feedback result corresponding to the second mask information, wherein the first mask feedback result is obtained by modifying the first mask information based on the first image and the initial image spot, and the second mask feedback result is obtained by modifying the second mask information based on the second image and the initial image spot; performing image recognition on the first image and the second image based on the first mask feedback result and the second mask feedback result, respectively, to obtain the target type.

[0120] In an optional embodiment, the first image, the second image, the initial image spot, the first mask information and the second mask information can be output so that the user can determine whether the first mask information and the second mask information are accurate. When the accuracy of the first mask information and the second mask information is low, the user can modify the first mask information according to the area corresponding to the initial image spot in the first image to obtain a first mask feedback result, and can modify the second mask information according to the area corresponding to the initial image spot in the second image to obtain a second mask feedback result, wherein the first mask feedback result includes the modified first mask information, and image recognition can be performed on the first image according to the modified first mask information to make the obtained first subtype more accurate, and image recognition can be performed on the second image according to the modified second mask information to make the obtained second subtype more accurate.

[0121] Combine the following Figure 7 A preferred embodiment of the present application is described in detail. The method can be executed by a computer terminal or a server, such as Figure 7 As shown, the method may include the following steps:

[0122] Step S701, training a general change detection model, wherein the general change detection model is used to recognize the first image and the second image;

[0123] The above-mentioned universal change detection model is mainly used to detect the universal changes between the first image and the second image, so as to determine the initial image spots corresponding to the universal change areas.

[0124] Optionally, first build Figure 5 The binary change detection network shown uses efficient-b0 as the backbone network to extract features corresponding to the first image and features corresponding to the second image. The features corresponding to the first image and the features corresponding to the second image can be concatenated and input into the cascade bifpn structure for feature interaction to obtain target features. The target features can be input into the decoder to obtain the final prediction result, that is, the initial image patch corresponding to the image change area in the first image and the second image.

[0125] Optionally, training can be performed on the task data to divide 14932 pairs of first images and second images as training sets, and 3494 pairs of first images and second images as test sets, wherein the image size can be 1024*1024, and the annotations in the initial image areas of all images are carried out according to the mutual conversion rules between multiple categories such as groundbreaking, greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, non-crop arable land, crop arable land, forest land, water area, hardened open space, green space, natural bare land, and sports field. Specifically, an initial learning rate of 0.005, a batch size of 32, and 500,000 iterations can be used.

[0126] Step S702, classifying the initial image spots to obtain initial image spots with class labels;

[0127] Optionally, the above-mentioned initial spots can be labeled with attributes according to multiple categories such as greenhouses under construction, buildings under construction, ground film, greenhouses, buildings, uncultivated land, cultivated land with crops, woodland, water area, hardened open space, green space, natural bare land, and sports fields, and a 400*400 image and a corresponding spot mask are cut from the first image and the second image at the geometric center of each spot to construct a data set for spot classification, and finally 2,230,032 groups of training spots and 179,864 pairs of test spots can be obtained.

[0128] Step S703, building a deep neural network that can fuse the mask area information and image information corresponding to the initial image spot.

[0129] Optional, such as Figure 6 As shown in the figure, a mask information fusion branch is added to the residual convolutional neural network. First, the nearest neighbor method can be used to align the mask size with the features output by the second residual block. Then, a single-channel mask is expanded to 512 dimensions using a convolution operation to obtain the mask features. The mask features are then added to the image features and input into the subsequent network, achieving the purpose of integrating the mask area information into the classification network. In addition, in order to solve the problem that small-area spots are seriously disturbed by the background, the circumscribed rectangle of the initial spot can be used in the feature map of the journey to deduct the features of the spot area. Considering the problem that small-area spots are too small in size in deep features, a feature pyramid structure can be introduced to determine the level of features to be used based on the size of the initial spot.

[0130] Step S704, using the initial image spots with category labels to train the deep neural network;

[0131] Optionally, the stochastic gradient descent method may be used to train the spot classifier according to the spot classification training data of the component in step S702, wherein the output learning rate is 0.1, the batch size is 512, and a total of 50 batches may be trained.

[0132] In step S705, the first image and the second image may be input into a universal change detection model to obtain an initial image patch that generates universal changes.

[0133] Step S706, pre-segmenting the above initial image spots to obtain pre-segmented initial image spots.

[0134] Step S707, using a spot classifier to classify the attributes of the initial spots after pre-segmentation;

[0135] Step S708, filtering the spots according to the categories required by the scene to obtain spots corresponding to the change types that the user is concerned about in the initial spots.

[0136] Optionally, for the cultivated land change detection task, pixels in the change area whose attributes change from cultivated land to other and from other to cultivated land can be retained. The predicted cultivated land change area can be used to output detailed land feature change types.

[0137] The effect of this application can be further illustrated by the following experiments:

[0138] 1) Experimental conditions

[0139] This experiment uses a task scenario dataset, adopts the pytorch deep learning framework, and the GPU configuration is NVIDIA Tesla P100.

[0140] 2) Experimental content

[0141] The data source of this experiment is 14,932 pairs of training data and 3,494 pairs of test data. The image size is 1024*1024, which is used to train and test the general binary change detection model. 2,230,032 sets of training patches and 179,864 sets of test patches were cut out from them for training and testing the patch classification model. We use the intersection over union (IoU) as the evaluation indicator. When evaluating the performance of cultivated land changes, only the data containing the cultivated land change type is tested.

[0142] 3) Comparison method

[0143] In order to verify the feasibility of the proposed method, we can compare it with the manually designed farmland change detection model:

[0144] In the general change detection dataset with completed category annotation, the labeled patches are filtered according to the rules of cultivated land change to obtain the training set and test set for training the cultivated land change detection model. For fair comparison, the general change model can be used as a pre-training model for the cultivated land change model, and the model can be fine-tuned on the cultivated land change dataset.

[0145] 4) Experimental results

[0146] The average classification accuracy (meanOA) of five-fold cross validation can be used as the evaluation indicator. The specific results are shown in the following table:

[0147] Method IoU Recall Precision Farmland change detection model 0.459 0.650 0.610 Our Method 0.448 0.645 0.591

[0148] As can be seen from the table, the above method can be close to the proprietary model for cultivated land change detection in performance. More importantly, the method used in this application is not specially optimized for the task of cultivated land change, and can obtain the change type of each cultivated land change map, and can be flexibly extended to other cultivated land-related changes or change detection tasks between other land objects.

[0149] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

[0150] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0152] Example 2

[0153] According to an embodiment of the present application, an image processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0154] Figure 8 is a flowchart of an image processing method according to an embodiment of the present application, such as Figure 8 As shown, the method may include the following steps:

[0155] Step S802: acquiring a first cultivated land image and a second cultivated land image.

[0156] The first farmland image and the second farmland image are images acquired from the same farmland area at different times.

[0157] Step S804: performing change detection on the first cultivated land image and the second cultivated land image to determine initial cultivated land patches.

[0158] The initial cultivated land image patch is used to represent the area where the cultivated land image in the first cultivated land image and the second cultivated land image changes.

[0159] Step S806: performing image recognition on the first cultivated land image and the second cultivated land image to determine the target type corresponding to the initial cultivated land patch.

[0160] Step S808, filtering the initial farmland map patches based on the target type to obtain target farmland map patches, wherein the type corresponding to the target farmland map patches is the farmland type.

[0161] The above-mentioned initial cultivated land map patches may be areas that have changed in the first cultivated land image and the second cultivated land image. Specifically, it may be that a greenhouse built on the cultivated land has caused the cultivated land image to change. The above-mentioned cultivated land type may be cultivated land-greenhouse or greenhouse-cultivated land type, that is, the user needs to pay attention to whether a greenhouse is built on the cultivated land or whether the existing greenhouse on the cultivated land is normal. The initial cultivated land map patches may be filtered according to the types of greenhouses and cultivated land to obtain target cultivated land map patches containing greenhouse changes to cultivated land or target cultivated land map patches containing cultivated land changes to greenhouses. The initial cultivated land map patches may be filtered by the target type to obtain the target cultivated land map patches, so that the user can pay attention to the type changes of greenhouses and cultivated land in the cultivated land.

[0162] In an embodiment of the present application, the target type includes: a first subtype of the initial cultivated land patch in the first cultivated land image, and a second subtype of the initial cultivated land patch in the second cultivated land image, wherein cultivated land image recognition is performed on the first cultivated land image and the second cultivated land image, and determining the target type corresponding to the initial cultivated land patch includes: determining the first mask information and the second mask information corresponding to the initial cultivated land patch; performing cultivated land image recognition on the first mask information and the first cultivated land image using a patch classifier to obtain the first subtype; performing cultivated land image recognition on the second mask information and the second cultivated land image using a patch classifier to obtain the second subtype.

[0163] In an embodiment of the present application, the image spot classifier includes at least: a mask information input module, a cultivated land image input module, a feature extraction module and an output module, wherein the feature extraction module is respectively connected to the mask information input module and the cultivated land image input module, and the output module is connected to the feature extraction module.

[0164] In an embodiment of the present application, a first mask information and a first cultivated land image are identified by a pattern classifier to obtain a first subtype, or a second mask information and a second cultivated land image are identified by a pattern classifier to obtain a second subtype, including: using a mask information input module to perform a convolution operation on the first mask information to obtain a first mask feature, or using the mask information input module to perform a convolution operation on the second mask information to obtain a second mask feature; using a cultivated land image input module to perform a convolution operation on the first cultivated land image to obtain a first cultivated land image feature, or using the cultivated land image input module to perform a convolution operation on the second cultivated land image to obtain a second cultivated land image feature; using a feature extraction module to extract features from the first mask feature and the first cultivated land image feature to obtain a first cultivated land pattern feature, or using a feature extraction module to extract features from the second mask feature and the second cultivated land image feature to obtain a second cultivated land pattern feature; using an output module to classify the first cultivated land pattern feature to obtain a first subtype, or using the output module to classify the second cultivated land pattern feature to obtain a second subtype.

[0165] In an embodiment of the present application, the feature extraction module includes: multiple residual blocks connected in sequence, wherein the farmland image input module is connected to the first residual block among the multiple residual blocks, and the mask information input module is connected to the target residual block among the multiple residual blocks.

[0166] In the embodiment of the present application, the target residual block is used to determine the target cultivated land patch features in the cultivated land image features using the mask features, and input the target cultivated land patch features into the next residual block.

[0167] In an embodiment of the present application, the spot classifier also includes: multiple fusion modules and cropping modules connected in sequence, the multiple fusion modules are connected to the multiple residual blocks in a one-to-one correspondence, the cropping module is connected to the multiple fusion modules, the output module is connected to the cropping module, and the number of the multiple fusion modules is determined based on the size of the initial cultivated land spot.

[0168] In an embodiment of the present application, the method also includes: using multiple fusion modules to fuse input features to obtain multiple fused features; using a cropping module to crop multiple fused features to obtain cropped features; and using an output module to classify the cropped features to obtain classification results.

[0169] In an embodiment of the present application, the initial cultivated land map patches are filtered based on the target type to obtain the target cultivated land map patches, including: determining whether the target type is the same as the cultivated land type; when the target type is different from the cultivated land type, the initial cultivated land map patches are filtered to obtain the target cultivated land map patches; when the target type is the same as the cultivated land type, the initial cultivated land map patches are retained to obtain the target cultivated land map patches.

[0170] In an embodiment of the present application, before determining whether the target type is the same as the cultivated land type, the method further includes: outputting multiple types; receiving a type selected from the multiple types; and determining that the selected type is the cultivated land type.

[0171] In an embodiment of the present application, after performing cultivated land image recognition on the first cultivated land image and the second cultivated land image to determine the target type corresponding to the initial cultivated land map patch, the method also includes: outputting the first cultivated land image, the second cultivated land image and the target type; receiving a type feedback result corresponding to the target type, wherein the type feedback result is used to determine whether the target type is correct.

[0172] In an embodiment of the present application, outputting a first cultivated land image, a second cultivated land image and a target type includes: determining a first display mode corresponding to a first subtype and a second display mode corresponding to a second subtype; marking initial cultivated land patches in the first cultivated land image according to the first display mode; and marking initial cultivated land patches in the second cultivated land image according to the second display mode.

[0173] In an embodiment of the present application, after performing change detection on the first cultivated land image and the second cultivated land image and determining the initial cultivated land patch, the method also includes: outputting the first cultivated land image, the second cultivated land image and the initial cultivated land patch; receiving cultivated land patch feedback results corresponding to the initial cultivated land patch, wherein the cultivated land patch feedback results are obtained by modifying the initial cultivated land patch based on the first cultivated land image and the second cultivated land image; and determining the first mask information and the second mask information corresponding to the cultivated land patch feedback results.

[0174] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0175] Example 3

[0176] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig. 9 As shown, the device 900 includes: an acquisition module 902 , a detection module 904 , a determination module 906 , and a filtering module 908 .

[0177] Among them, the acquisition module is used to acquire the first image and the second image, wherein the first image and the second image are images obtained by collecting the same area at different times; the detection module is used to perform change detection on the first image and the second image, and determine the initial spot, wherein the initial spot is used to characterize the area where the image changes in the first image and the second image; the determination module is used to perform image recognition on the first image and the second image, and determine the target type corresponding to the initial spot; the recognition module is used to filter the initial spot based on the target type to obtain the target spot, wherein the type corresponding to the target spot is a preset type.

[0178] It should be noted that the acquisition module 902, the detection module 904, the determination module 906, and the filtering module 908 correspond to steps S202 to S208 in Example 1, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0179] In the above embodiments of the present application, the determination module includes: a first determination unit and an identification unit.

[0180] Among them, the first determination unit is used to determine the first mask information and the second mask information corresponding to the initial pattern spot; the recognition unit is used to use the pattern spot classifier to perform image recognition on the first mask information and the first image to obtain the first subtype; the recognition unit is also used to use the pattern spot classifier to perform image recognition on the second mask information and the second image to obtain the second subtype.

[0181] In the above embodiments of the present application, the image spot classifier includes at least: a mask information input module, an image input module, a feature extraction module and an output module, wherein the feature extraction module is respectively connected to the mask information input module and the image input module, and the output module is connected to the feature extraction module.

[0182] In the above embodiments of the present application, the recognition unit is used to use the mask information input module to perform a convolution operation on the first mask information to obtain the first mask feature, or use the mask information input module to perform a convolution operation on the second mask information to obtain the second mask feature; use the image input module to perform a convolution operation on the first image to obtain the first image feature, or use the image input module to perform a convolution operation on the second image to obtain the second image feature; use the feature extraction module to extract features from the first mask feature and the first image feature to obtain the first spot feature, or use the feature extraction module to extract features from the second mask feature and the second image feature to obtain the second spot feature; use the output module to classify the first spot feature to obtain the first subtype, or use the output module to classify the second spot feature to obtain the second subtype.

[0183] In the above embodiment of the present application, the feature extraction module includes: multiple residual blocks connected in sequence, wherein the image input module is connected to the first residual block among the multiple residual blocks, and the mask information input module is connected to the target residual block among the multiple residual blocks.

[0184] In the above embodiment of the present application, the spot classifier also includes: multiple fusion modules and cropping modules connected in sequence, the multiple fusion modules are connected to the multiple residual blocks in a one-to-one correspondence, the cropping module is connected to the multiple fusion modules, the output module is connected to the cropping module, and the number of the multiple fusion modules is determined based on the size of the initial spot.

[0185] In the above embodiments of the present application, the device further includes: a fusion module, a cropping module, and a classification module.

[0186] Among them, the fusion module is used to use multiple fusion modules to fuse the input features to obtain multiple fused features; the cropping module is used to use the cropping module to crop multiple fused features to obtain cropped features; the classification module is used to classify the cropped features using the output module to obtain classification results.

[0187] In the above embodiment of the present application, the filtering module includes: a second determining unit, a filtering unit, and a retaining unit.

[0188] Among them, the second determination unit is used to determine whether the target type is the same as the preset type; the filtering unit is used to filter the initial image spot to obtain the target image spot when the target type is different from the preset type; the retaining unit is used to retain the initial image spot to obtain the target image spot when the target type is the same as the preset type.

[0189] In the above embodiments of the present application, the device further includes: an output module and a receiving module.

[0190] Among them, the output module is used to output multiple types; the receiving module is used to receive a type selected from the multiple types; and the determination module is also used to determine that the selected type is a preset type.

[0191] In the above embodiments of the present application, the output module is also used to output the first image, the second image and the target type; the receiving module is also used to receive a type feedback result corresponding to the target type, wherein the type feedback result is used to determine whether the target type is correct.

[0192] In the above embodiment of the present application, the output module includes: a third determination unit and a marking unit.

[0193] Among them, the third determination unit is used to determine the first display mode corresponding to the first subtype and the second display mode corresponding to the second subtype; the marking unit is used to mark the initial image spot in the first image according to the first display mode; the marking unit is also used to mark the initial image spot in the second image according to the second display mode.

[0194] In the above embodiments of the present application, the output module is also used to output the first image, the second image and the initial image spot; the receiving module is also used to receive the image spot feedback result corresponding to the initial image spot, wherein the image spot feedback result is obtained by modifying the initial image spot based on the first image and the second image; the determination module is also used to determine the first mask information and the second mask information corresponding to the image spot feedback result.

[0195] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0196] Example 4

[0197] According to an embodiment of the present application, an image processing device for implementing the above-mentioned image processing method is also provided. Fig.10 As shown, the device 1000 includes: an acquisition module 1002 , a detection module 1004 , a determination module 1006 , and a filtering module 1008 .

[0198] Among them, the acquisition module is used to acquire the first cultivated land image and the second cultivated land image, wherein the first cultivated land image and the second cultivated land image are images obtained by collecting the same cultivated land area at different times; the detection module is used to detect changes in the first cultivated land image and the second cultivated land image, and determine the initial cultivated land map patches, wherein the initial cultivated land map patches are used to characterize the areas where the cultivated land images in the first cultivated land image and the second cultivated land image have changed; the determination module is used to perform image recognition on the first cultivated land image and the second cultivated land image, and determine the target type corresponding to the initial cultivated land map patches; the filtering module is used to filter the initial cultivated land map patches based on the target type to obtain the target cultivated land map patches, wherein the type corresponding to the target cultivated land map patches is the cultivated land type.

[0199] It should be noted that the acquisition module 1002, the detection module 1004, the determination module 1006, and the filtering module 1008 correspond to steps S802 to S808 of Example 2, and the examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 2. It should be noted that the above-mentioned modules as part of the device can be run in the computer terminal 10 provided in Example 1.

[0200] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 2 as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 2.

[0201] Example 5

[0202] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0203] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0204] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method: acquiring a first image and a second image, wherein the first image and the second image are images acquired from the same area at different times; performing change detection on the first image and the second image to determine an initial spot, wherein the initial spot is used to characterize an area where the images in the first image and the second image have changed; performing image recognition on the first image and the second image to determine a target type corresponding to the initial spot; filtering the initial spot based on the target type to obtain a target spot, wherein the type corresponding to the target spot is a preset type.

[0205] Optionally, Fig.11 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Fig.11 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors and a memory.

[0206] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the image processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0207] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: acquire a first image and a second image, wherein the first image and the second image are images acquired from the same area at different times; perform change detection on the first image and the second image to determine an initial spot, wherein the initial spot is used to characterize an area where the images in the first image and the second image have changed; perform image recognition on the first image and the second image to determine a target type corresponding to the initial spot; filter the initial spot based on the target type to obtain a target spot, wherein the type corresponding to the target spot is a preset type.

[0208] Optionally, the processor may also execute the program code of the following steps: determining the first mask information and the second mask information corresponding to the initial spot; performing image recognition on the first mask information and the first image using the spot classifier to obtain a first subtype; performing image recognition on the second mask information and the second image using the spot classifier to obtain a second subtype.

[0209] Optionally, the processor may also execute the program code of the following steps: the spot classifier includes at least: a mask information input module, an image input module, a feature extraction module and an output module, wherein the feature extraction module is respectively connected to the mask information input module and the image input module, and the output module is connected to the feature extraction module.

[0210] Optionally, the processor may also execute the program code of the following steps: using the mask information input module to perform a convolution operation on the first mask information to obtain a first mask feature, or using the mask information input module to perform a convolution operation on the second mask information to obtain a second mask feature; using the image input module to perform a convolution operation on the first image to obtain a first image feature, or using the image input module to perform a convolution operation on the second image to obtain a second image feature; using the feature extraction module to extract features from the first mask feature and the first image feature to obtain a first pattern feature, or using the feature extraction module to extract features from the second mask feature and the second image feature to obtain a second pattern feature; using the output module to classify the first pattern feature to obtain a first subtype, or using the output module to classify the second pattern feature to obtain a second subtype.

[0211] Optionally, the above-mentioned processor can also execute the program code of the following steps: the feature extraction module includes: multiple residual blocks connected in sequence, wherein the image input module is connected to the first residual block among the multiple residual blocks, and the mask information input module is connected to the target residual block among the multiple residual blocks.

[0212] Optionally, the processor may also execute the program code of the following steps: the target residual block is used to determine the target spot feature in the image feature by using the mask feature, and input the target spot feature to the next residual block.

[0213] Optionally, the processor may also execute the program code of the following steps: the spot classifier also includes: a plurality of fusion modules and cropping modules connected in sequence, the plurality of fusion modules are connected one-to-one with the plurality of residual blocks, the cropping module is connected with the plurality of fusion modules, the output module is connected with the cropping module, and the number of the plurality of fusion modules is determined based on the size of the initial spot.

[0214] Optionally, the processor may also execute the following program code: using multiple fusion modules to fuse input features to obtain multiple fused features; using a cropping module to crop multiple fused features to obtain cropped features; and using an output module to classify the cropped features to obtain classification results.

[0215] Optionally, the processor may also execute the following program code: determining whether the target type is the same as a preset type; when the target type is different from the preset type, filtering the initial image spots to obtain target image spots; when the target type is the same as the preset type, retaining the initial image spots to obtain target image spots.

[0216] Optionally, the processor may also execute program code of the following steps: outputting multiple types; receiving a type selected from the multiple types; and determining that the selected type is a preset type.

[0217] Optionally, the processor may also execute program code of the following steps: outputting a first image, a second image and a target type; and receiving a type feedback result corresponding to the target type, wherein the type feedback result is used to determine whether the target type is correct.

[0218] Optionally, the processor may also execute program code of the following steps: determining a first display mode corresponding to the first subtype and a second display mode corresponding to the second subtype; marking the initial image spot in the first image according to the first display mode; and marking the initial image spot in the second image according to the second display mode.

[0219] Optionally, the processor may also execute the program code of the following steps: outputting a first image, a second image and an initial image spot; receiving an image spot feedback result corresponding to the initial image spot, wherein the image spot feedback result is obtained by modifying the initial image spot based on the first image and the second image; and determining the first mask information and the second mask information corresponding to the image spot feedback result.

[0220] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain a first cultivated land image and a second cultivated land image, wherein the first cultivated land image and the second cultivated land image are images collected from the same cultivated land area at different times; perform change detection on the first cultivated land image and the second cultivated land image to determine an initial cultivated land map patch, wherein the initial cultivated land map patch is used to characterize an area where the cultivated land image has changed in the first cultivated land image and the second cultivated land image; perform image recognition on the first cultivated land image and the second cultivated land image to determine a target type corresponding to the initial cultivated land map patch; filter the initial cultivated land map patch based on the target type to obtain a target cultivated land map patch, wherein the type corresponding to the target cultivated land map patch is the cultivated land type.

[0221] It can be understood by those skilled in the art that Fig.11 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.11 It does not limit the structure of the above electronic device. For example, the computer terminal A may also include Fig.11 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.11 Different configurations are shown.

[0222] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0223] Example 6

[0224] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image processing method provided by the above embodiment.

[0225] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0226] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: acquiring a first image and a second image, wherein the first image and the second image are images acquired by capturing the same area at different times; performing change detection on the first image and the second image to determine an initial spot, wherein the initial spot is used to characterize an area where image changes occur in the first image and the second image; performing image recognition on the first image and the second image to determine a target type corresponding to the initial spot; filtering the initial spot based on the target type to obtain a target spot, wherein the type corresponding to the target spot is a preset type.

[0227] Optionally, the storage medium is also configured to store program codes for executing the following steps: determining the first mask information and the second mask information corresponding to the initial pattern spot; performing image recognition on the first mask information and the first image using a pattern spot classifier to obtain a first subtype; performing image recognition on the second mask information and the second image using a pattern spot classifier to obtain a second subtype.

[0228] Optionally, the above-mentioned storage medium is also configured to store program codes for executing the following steps: mask information input module, image input module, feature extraction module and output module, wherein the feature extraction module is respectively connected to the mask information input module and the image input module, and the output module is connected to the feature extraction module.

[0229] Optionally, the storage medium is also configured to store program codes for executing the following steps: using a mask information input module to perform a convolution operation on the first mask information to obtain a first mask feature, or using the mask information input module to perform a convolution operation on the second mask information to obtain a second mask feature; using an image input module to perform a convolution operation on the first image to obtain a first image feature, or using the image input module to perform a convolution operation on the second image to obtain a second image feature; using a feature extraction module to extract features from the first mask feature and the first image feature to obtain a first pattern feature, or using a feature extraction module to extract features from the second mask feature and the second image feature to obtain a second pattern feature; using an output module to classify the first pattern feature to obtain a first subtype, or using the output module to classify the second pattern feature to obtain a second subtype.

[0230] Optionally, the above-mentioned storage medium is also configured to store program code for executing the following steps: the feature extraction module includes: a plurality of residual blocks connected in sequence, wherein the image input module is connected to the first residual block among the plurality of residual blocks, and the mask information input module is connected to the target residual block among the plurality of residual blocks.

[0231] Optionally, the storage medium is further configured to store program codes for executing the following steps: the target residual block is used to determine target spot features in the image features using mask features, and input the target spot features into the next residual block.

[0232] Optionally, the storage medium is also configured to store program code for executing the following steps: the spot classifier also includes: multiple fusion modules and cropping modules connected in sequence, the multiple fusion modules are connected to the multiple residual blocks in a one-to-one correspondence, the cropping module is connected to the multiple fusion modules, the output module is connected to the cropping module, and the number of the multiple fusion modules is determined based on the size of the initial spot.

[0233] Optionally, the storage medium is also configured to store program codes for executing the following steps: using multiple fusion modules to fuse input features to obtain multiple fused features; using a cropping module to crop multiple fused features to obtain cropped features; using an output module to classify the cropped features to obtain classification results.

[0234] Optionally, the above-mentioned storage medium is also configured to store program codes for executing the following steps: determining whether the target type is the same as the preset type; when the target type is different from the preset type, filtering the initial image spots to obtain target image spots; when the target type is the same as the preset type, retaining the initial image spots to obtain target image spots.

[0235] Optionally, the storage medium is further configured to store program codes for executing the following steps: outputting multiple types; receiving a type selected from the multiple types; and determining that the selected type is a preset type.

[0236] Optionally, the storage medium is also configured to store program codes for executing the following steps: outputting a first image, a second image and a target type; receiving a type feedback result corresponding to the target type, wherein the type feedback result is used to determine whether the target type is correct.

[0237] Optionally, the storage medium is also configured to store program codes for executing the following steps: determining a first display mode corresponding to the first subtype and a second display mode corresponding to the second subtype; marking the initial image spot in the first image according to the first display mode; and marking the initial image spot in the second image according to the second display mode.

[0238] Optionally, the storage medium is also configured to store program codes for executing the following steps: outputting a first image, a second image and an initial image spot; receiving an image spot feedback result corresponding to the initial image spot, wherein the image spot feedback result is obtained by modifying the initial image spot based on the first image and the second image; and determining first mask information and second mask information corresponding to the image spot feedback result.

[0239] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps: acquiring a first cultivated land image and a second cultivated land image, wherein the first cultivated land image and the second cultivated land image are images acquired from the same cultivated land area at different times; performing change detection on the first cultivated land image and the second cultivated land image to determine initial cultivated land patches, wherein the initial cultivated land patches are used to characterize areas where the cultivated land images in the first cultivated land image and the second cultivated land image have changed; performing image recognition on the first cultivated land image and the second cultivated land image to determine a target type corresponding to the initial cultivated land patches; filtering the initial cultivated land patches based on the target type to obtain target cultivated land patches, wherein the type corresponding to the target cultivated land patches is the cultivated land type.

[0240] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0241] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0242] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0243] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0244] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0245] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0246] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a first image and a second image, wherein the first image and the second image are images acquired from the same area at different times; Performing change detection on the first image and the second image to determine an initial image spot, wherein the initial image spot is used to characterize an area where images in the first image and the second image have changed; Using a spot classifier, image recognition is performed on the first image and the second image to determine the target type corresponding to the initial spot, wherein the spot classifier at least includes an image input module, a mask information input module and a feature extraction module, and the feature extraction module includes: a plurality of residual blocks connected in sequence, the output end of the image input module is connected to the input end of a first residual block among the plurality of residual blocks, the output end of the mask information input module is connected to the output end of a second residual block among the plurality of residual blocks, the input end of the second residual block is connected to the output end of the first residual block, the mask information input module is used to generate mask features, and the feature extraction module is used to determine the target type based on the mask features; The initial spots are filtered based on the target type to obtain target spots, wherein the type corresponding to the target spots is a preset type.

2. The method according to claim 1, characterized in that The target type includes: a first subtype of the initial spot in the first image, and a second subtype of the initial spot in the second image, wherein the first image and the second image are subjected to image recognition by using a spot classifier to determine the target type corresponding to the initial spot includes: Determine the first mask information and the second mask information corresponding to the initial image spot; Using the spot classifier to perform image recognition on the first mask information and the first image to obtain the first subtype; The spot classifier is used to perform image recognition on the second mask information and the second image to obtain the second subtype.

3. The method according to claim 2, characterized in that The pattern classifier further includes: an output module, wherein an input end of the output module is connected to an output end of the feature extraction module.

4. The method according to claim 3, characterized in that Using the spot classifier to perform image recognition on the first mask information and the first image to obtain the first subtype, or using the spot classifier to perform image recognition on the second mask information and the second image to obtain the second subtype includes: Using the mask information input module to perform a convolution operation on the first mask information to obtain a first mask feature in the mask features, or using the mask information input module to perform a convolution operation on the second mask information to obtain a second mask feature in the mask features; Using the image input module to perform a convolution operation on the first image to obtain a first image feature, or using the image input module to perform a convolution operation on the second image to obtain a second image feature; Using the feature extraction module to perform feature extraction on the first mask feature and the first image feature to obtain a first spot feature, or using the feature extraction module to perform feature extraction on the second mask feature and the second image feature to obtain a second spot feature; The first pattern feature is classified using the output module to obtain the first subtype, or the second pattern feature is classified using the output module to obtain the second subtype.

5. The method according to claim 3, characterized in that: The spot classifier also includes: a plurality of fusion modules and cropping modules connected in sequence, the plurality of fusion modules are connected to the plurality of residual blocks in a one-to-one correspondence, the cropping module is connected to the plurality of fusion modules, the output module is connected to the cropping module, and the number of the plurality of fusion modules is determined based on the size of the initial spot.

6. The method according to claim 5, characterized in that The initial spots are filtered based on the target type to obtain target spots including: Determining whether the target type is the same as the preset type; When the target type is different from the preset type, filtering the initial image spots to obtain the target image spots; In the case where the target type is the same as the preset type, the initial pattern spot is retained to obtain the target pattern spot.

7. The method according to claim 6, characterized in that Before determining whether the target type is the same as the preset type, the method further includes: Output multiple types; receiving a type selected from the plurality of types; The selected type is determined to be the preset type.

8. The method according to claim 5, characterized in that After performing image recognition on the first image and the second image to determine the target type corresponding to the initial image spot, the method further includes: outputting the first image, the second image, and the target type; A type feedback result corresponding to the target type is received, wherein the type feedback result is used to determine whether the target type is correct.

9. The method according to claim 8, characterized in that Outputting the first image, the second image, and the target type includes: Determine a first display mode corresponding to the first subtype and a second display mode corresponding to the second subtype; marking the initial image spot in the first image according to the first display mode; The initial image spot is marked in the second image according to the second display mode.

10. The method according to claim 5, characterized in that After performing change detection on the first image and the second image to determine the initial image spots, the method further includes: Outputting the first image, the second image and the initial image spot; receiving a spot feedback result corresponding to the initial spot, wherein the spot feedback result is obtained by modifying the initial spot based on the first image and the second image; Determine the first mask information and the second mask information corresponding to the spot feedback result.

11. An image processing method, characterized in that: include: Acquire a first cultivated land image and a second cultivated land image, wherein the first cultivated land image and the second cultivated land image are images acquired from the same cultivated land area at different times; Performing change detection on the first cultivated land image and the second cultivated land image to determine initial cultivated land patches, wherein the initial cultivated land patches are used to represent areas where cultivated land images in the first cultivated land image and the second cultivated land image have changed; Using a spot classifier, image recognition is performed on the first cultivated land image and the second cultivated land image to determine the target type corresponding to the initial cultivated land spot, wherein the spot classifier at least includes an image input module, a mask information input module and a feature extraction module, and the feature extraction module includes: a plurality of residual blocks connected in sequence, the output end of the image input module is connected to the input end of a first residual block among the plurality of residual blocks, the output end of the mask information input module is connected to the output end of a second residual block among the plurality of residual blocks, the input end of the second residual block is connected to the output end of the first residual block, the mask information input module is used to generate a mask feature, and the feature extraction module is used to determine the target type based on the mask feature; The initial cultivated land patches are filtered based on the target type to obtain target cultivated land patches, wherein the type corresponding to the target cultivated land patches is the cultivated land type.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 11.

13. A computer terminal, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the image processing method according to any one of claims 1 to 11 when running.

Images