Image Change Detection Method, Device, Storage Medium and Electronic Device

Abstract

The present invention provides an image change detection method, device, storage medium and electronic device. Among them, the method includes: obtaining a reference image and obtaining a target image corresponding to the reference image, where the image acquisition time of the target image is later than the image acquisition time of the reference image; calling an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively to obtain a target instance mask sub-graph set corresponding to the target image and a reference instance mask sub-graph set corresponding to the reference image; generating a difference mask sub-graph sequence based on the target instance mask sub-graph set and the reference instance mask sub-graph set; generating an image change mask graph according to the difference mask sub-graph sequence to determine an image change detection result based on the image change mask graph. The embodiments of the present invention can improve the adaptability and accuracy of image change detection while ensuring relatively low time cost and labor cost.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to an image change detection method, device, storage medium, and electronic device. Background Art

[0002] At present, with the continuous development of computer technology (such as digital image processing technology), image change detection has become an important research field, and in daily life and industrial production, image change detection has a wide range of applications, such as in fields like security monitoring, medical diagnosis, industrial quality control, intelligent transportation systems, etc.; among them, the so-called image change detection refers to taking multiple pictures of the same object or scene over a period of time, and then comparing the images taken each time to detect whether any changes exist. In related technologies, image change detection methods usually include threshold-based methods (i.e., dividing the image into different regions and setting a threshold for each region; if the pixel value of a certain region in the image exceeds the threshold, it is considered that the region has changed), feature-based methods (i.e., using machine learning technology to automatically extract the features of the image and comparing the feature differences between different images; if the feature differences between two images are large, it is considered that they have changed), statistics-based methods (i.e., using statistical techniques to calculate the change rate of the image, such as using the mean squared error (MSE) or Peak Signal-to-Noise Ratio (PSNR) to measure the degree of change of the image), and deep learning-based methods (i.e., using deep neural networks to automatically extract the features of the image and comparing the feature differences between different images to achieve end-to-end image change detection), etc. Among them, traditional non-deep learning methods (such as threshold-based methods, etc.) have relatively strict environmental requirements for business scenarios, poor adaptability, and low accuracy, and are prone to false changes resulting in false alarms; while the end-to-end image change detection method based on deep learning has high requirements for the training data, and it takes a lot of time and manpower to collect and label the data. Based on this, how to improve the adaptability and accuracy of image change detection while ensuring low time cost and manpower cost has become a research hotspot. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an image change detection method, device, storage medium, and electronic device to solve problems such as low adaptability and low accuracy in related technologies, or the need to consume a large amount of time cost and manpower cost; that is, embodiments of the present invention can improve the adaptability and accuracy of image change detection while ensuring low time cost and manpower cost.

[0004] According to one aspect of the present invention, there is provided an image change detection method, the method comprising:

[0005] Obtain a reference image and obtain a target image corresponding to the reference image, wherein the image acquisition time of the target image is later than the image acquisition time of the reference image;

[0006] Call an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, to obtain a set of target instance mask sub - graphs corresponding to the target image and a set of reference instance mask sub - graphs corresponding to the reference image. Any instance mask sub - graph corresponding to any image means: the mask of the instance corresponding to the any instance mask sub - graph under the any image;

[0007] Based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs, generate a sequence of difference mask sub - graphs, where one difference mask sub - graph is used to indicate: the difference points between two mask sub - graphs;

[0008] Generate an image change mask graph according to the sequence of difference mask sub - graphs, so as to determine an image change detection result based on the image change mask graph.

[0009] According to another aspect of the present invention, there is provided an image change detection device, the device comprising:

[0010] An acquisition unit, configured to obtain a reference image and obtain a target image corresponding to the reference image, wherein the image acquisition time of the target image is later than the image acquisition time of the reference image;

[0011] A processing unit, configured to call an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, to obtain a set of target instance mask sub - graphs corresponding to the target image and a set of reference instance mask sub - graphs corresponding to the reference image. Any instance mask sub - graph corresponding to any image means: the mask of the instance corresponding to the any instance mask sub - graph under the any image;

[0012] The processing unit is further configured to generate a sequence of difference mask sub - graphs based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs, where one difference mask sub - graph is used to indicate: the difference points between two mask sub - graphs;

[0013] The processing unit is further configured to generate an image change mask graph according to the sequence of difference mask sub - graphs, so as to determine an image change detection result based on the image change mask graph.

[0014] According to another aspect of the present invention, there is provided an electronic device, which includes a processor and a memory for storing a program. Wherein, the program includes instructions that, when executed by the processor, cause the processor to execute the above-mentioned method.

[0015] According to another aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the above-mentioned method.

[0016] In an embodiment of the present invention, after obtaining a reference image and a target image corresponding to the reference image, an image instance segmentation model can be called to perform instance segmentation on the target image and the reference image respectively, to obtain a set of target instance mask sub-images corresponding to the target image and a set of reference instance mask sub-images corresponding to the reference image. Any instance mask sub-image corresponding to any image refers to: the mask of the instance corresponding to any instance mask sub-image under any image. The image acquisition time of the target image is later than the image acquisition time of the reference image. Then, based on the set of target instance mask sub-images and the set of reference instance mask sub-images, a sequence of difference mask sub-images can be generated. One difference mask sub-image is used to indicate: the difference points between two mask sub-images; and according to the sequence of difference mask sub-images, an image change mask map can be generated, so as to determine an image change detection result based on the image change mask map. It can be seen that in the embodiment of the present invention, the reference image and the target image can be instance-segmented by the image instance segmentation model, without spending a large amount of time cost and labor cost to obtain a training data set for end-to-end model training, which can effectively save time cost and labor cost; moreover, in the embodiment of the present invention, the change situation between the reference image and the target image can be accurately represented by the image change mask map, so as to obtain a relatively accurate image change detection result, and it can be applied to image change detection in any scenario; that is to say, in the embodiment of the present invention, the adaptability and accuracy rate of image change detection can be improved while ensuring relatively low time cost and labor cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the following description of exemplary embodiments with reference to the accompanying drawings, more details, features and advantages of the present invention are disclosed. In the drawings:

[0018] Figure 1 A flowchart showing a method for image change detection according to an exemplary embodiment of the present invention is shown;

[0019] Figure 2a A schematic diagram showing an instance segmentation according to an exemplary embodiment of the present invention is shown;

[0020] Figure 2b Another schematic diagram showing an instance segmentation according to an exemplary embodiment of the present invention is shown;

[0021] Figure 3 Shows a schematic diagram of a masked sub - graph according to an exemplary embodiment of the present invention;

[0022] Figure 4 Shows a schematic flowchart of another image change detection method according to an exemplary embodiment of the present invention;

[0023] Figure 5a Shows a schematic flowchart of yet another image change detection method according to an exemplary embodiment of the present invention;

[0024] Figure 5b Shows a schematic flowchart of a deduplication process according to an exemplary embodiment of the present invention;

[0025] Figure 5c Shows a schematic flowchart of a process for determining a matching masked sub - graph according to an exemplary embodiment of the present invention;

[0026] Figure 5d Shows a schematic flowchart of a process for generating a sequence of difference masked sub - graphs according to an exemplary embodiment of the present invention;

[0027] Figure 5e Shows a schematic diagram of an image change region according to an exemplary embodiment of the present invention;

[0028] Figure 6 Shows a schematic block diagram of an image change detection device according to an exemplary embodiment of the present invention;

[0029] Figure 7 Shows a structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present invention. Detailed implementation manners

[0030] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the protection scope of the present invention.

[0031] It should be understood that the various steps recorded in the method embodiments of the present invention can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0032] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] It should be noted that the modifications of "one" and "a plurality of" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly specified in the context, it should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] It should be noted that the execution subject of the image change detection method provided in the embodiments of the present invention can be one or more electronic devices, and the present invention does not make any limitations in this regard; among them, the electronic device can be a terminal (i.e., a client) or a server. Then, when the execution subject includes multiple electronic devices and at least one terminal and at least one server are included in the multiple electronic devices, the image change detection method provided in the embodiments of the present invention can be jointly executed by the terminal and the server. Correspondingly, the terminal mentioned herein can include, but is not limited to: smart phones, tablet computers, laptop computers, desktop computers, smart watches, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, and so on. The server mentioned herein can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, and so on.

[0036] Based on the above description, an image change detection method is proposed in the embodiments of the present invention. This image change detection method can be executed by the above-mentioned electronic device (terminal or server); or, this image change detection method can be jointly executed by the terminal and the server. For the convenience of description, in the following, it will be described by taking the electronic device executing this image change detection method as an example; as Figure 1As shown, the image change detection method may include the following steps S101 - S104:

[0037] S101, obtain a reference image and obtain a target image corresponding to the reference image, where the image acquisition time of the target image is later than the image acquisition time of the reference image.

[0038] It should be noted that the image change detection method mentioned in the embodiments of the present invention can be applied to various scenarios, such as security monitoring, medical diagnosis, and intelligent transportation. Based on this, the above reference image and target image can be images in any scenario, and the present invention does not limit this. As the name implies, to perform image change detection on a certain scenario, two images (i.e., the reference image and the target image) need to be input.

[0039] In the embodiments of the present invention, when obtaining the target image corresponding to the reference image, the electronic device can obtain an initial image, where the image acquisition time of the initial image is later than the image acquisition time of the reference image; then, according to the histogram distribution of the reference image (i.e., the histogram), perform histogram matching on the initial image to obtain the target image corresponding to the reference image, so that the histogram distribution of the target image matches the histogram distribution of the reference image, and the image acquisition time of the target image is equal to the image acquisition time of the initial image. Among them, histogram matching means: transforming the histogram of an image (such as the initial image) with the histogram of a standard image (such as the reference image) as the standard, so that the histograms of the two images are the same or approximate, so that the two images have similar hues and contrasts. That is to say, the electronic device can make the target image and the reference image have a similar histogram distribution through histogram matching, so that the histogram distribution of the target image matches the histogram distribution of the reference image. It can be seen that the embodiments of the present invention can effectively avoid interference caused by factors such as changes in the illumination conditions of the environment to image change detection.

[0040] Then correspondingly, the acquisition methods of the above initial image and reference image can include but are not limited to the following several types:

[0041] The first acquisition method: The electronic device can obtain the initial download link of the initial image and use the initial download link to download the image, so as to use the image downloaded based on the initial download link as the initial image; correspondingly, the electronic device can obtain the reference download link of the reference image and use the reference download link to download the image, so as to use the image downloaded based on the reference download link as the reference image.

[0042] The second acquisition method: The electronic device stores multiple images captured in a certain scenario. In this case, the electronic device can select a first image from the multiple images and use the first image as the reference image, and select a second image from the multiple images and use the second image as the initial image, and the image acquisition time (i.e., the shooting and acquisition time) of the second image is later than the image acquisition time of the first image.

[0043] The third acquisition method: The electronic device has a shooting and acquisition component, and this shooting and acquisition component can be used to shoot and acquire images. In this case, the electronic device can shoot and acquire a certain scenario at the reference image acquisition time through the shooting and acquisition component, and shoot and acquire the scenario at the initial image acquisition time through the shooting and acquisition component, so as to use the image shot and acquired at the reference image acquisition time as the reference image, and use the image shot and acquired at the initial image acquisition time as the initial image, and the initial image acquisition time is later than the reference image acquisition time, and so on. Optionally, since the image acquisition time of the initial image is later than the image acquisition time of the reference image, the reference image can be the image shot and acquired last time, and the initial image can be the image shot and acquired currently, that is to say, the reference image acquisition time is the time of the last shooting and acquisition, and the initial image acquisition time is the time of the current shooting and acquisition.

[0044] S102, Call the image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, and obtain a set of target instance mask subgraphs corresponding to the target image and a set of reference instance mask subgraphs corresponding to the reference image. Any instance mask subgraph corresponding to an image means: the mask of the instance corresponding to any instance mask subgraph under any image.

[0045] Wherein, an instance in the image refers to an object in the image; correspondingly, when performing instance segmentation on the reference image, the electronic device can determine each reference instance in the reference image and determine the reference instance mask subgraph of each reference instance to obtain a set of reference instance mask subgraphs. The set of reference instance mask subgraphs can include the reference instance mask subgraphs of each reference instance; Exemplarily, each reference instance in the reference image can be as Figure 2a shown. Similarly, when performing instance segmentation on the target image, the electronic device can determine each target instance in the target image and determine the target instance mask subgraph of each target instance to obtain a set of target instance mask subgraphs. The set of target instance mask subgraphs can include the target instance mask subgraphs of each target instance; Exemplarily, each target instance in the target image can be as Figure 2b shown.

[0046] In an embodiment of the present invention, any mask sub - graph (i.e., mask) may include pixel identifiers of each pixel point in the image corresponding to the any mask sub - graph, and the pixel identifier may include either a value - bearing pixel identifier or a non - value - bearing pixel identifier; wherein, the value - bearing pixel identifier may be used to indicate that the pixel point has a value (i.e., to indicate that the pixel point is a pixel point constituting an instance), and the non - value - bearing pixel identifier may be used to indicate that the pixel point has no value (i.e., to indicate that the pixel point is not a pixel point constituting an instance). That is to say, the pixel point indicated by the value - bearing pixel identifier is a pixel point with a value, and the pixel point indicated by the non - value - bearing pixel identifier is a pixel point without a value. Optionally, the pixel identifier may be a digital identifier or a Boolean value identifier, etc., and the present invention does not limit this; for example, when the pixel identifier is a digital identifier, the numerical value 1 may be used as the value - bearing pixel identifier, and the numerical value 0 may be used as the non - value - bearing pixel identifier, and so on. Correspondingly, any mask graph may also include pixel identifiers of each pixel point in the image corresponding to the any mask graph. For the convenience of description, subsequent descriptions will be given taking the mask sub - graph as an example.

[0047] It should be noted that the above - mentioned image instance segmentation model may be a threshold - based image instance segmentation model, an edge - detection - based image instance segmentation model, a deep - learning - based image instance segmentation model, etc.; the present invention does not limit this. Optionally, the embodiment of the present invention may preferably use a deep - learning - based image instance segmentation model as the above - mentioned image instance segmentation model to obtain a set of target instance mask sub - graphs and a set of reference instance mask sub - graphs with higher accuracy.

[0048] S103, Generate a sequence of difference mask sub - graphs based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs. One difference mask sub - graph is used to indicate: the difference points between two mask sub - graphs.

[0049] For example, assume that the value - bearing pixel identifier is 1, the non - value - bearing pixel identifier is 0, and the difference mask sub - graph A is used to indicate the difference points between mask sub - Figure 1 and mask sub - graph 2. Then, for the position of the target pixel point in the difference mask sub - graph A, if the pixel identifier at the position of the target pixel point in mask sub - Figure 1 is the same as the pixel identifier at the position of the target pixel point in mask sub - graph 2 (such as both being value - bearing pixel identifiers or both being non - value - bearing pixel identifiers), the pixel identifier of the target pixel point position in the difference mask sub - graph A may be 0. If the pixel identifier at the position of the target pixel point in mask sub - Figure 1 is different from the pixel identifier at the position of the target pixel point in mask sub - graph 2 (such as one being a value - bearing pixel identifier and the other being a non - value - bearing pixel identifier), the pixel identifier of the target pixel point position in the difference mask sub - graph A may be 1. Based on this, the difference mask sub - graph A can use the value - bearing pixel identifier (i.e., 1) to indicate the difference points between mask sub - Figure 1 and mask sub - graph 2.

[0050] S104. Generate an image change mask map according to the difference mask sub-map sequence, and determine the image change detection result based on the image change mask map.

[0051] In the embodiment of the present invention, the image change mask map can be used to indicate the image change area between the reference image and the target image. That is to say, after obtaining the image change mask map, the specific area where the change occurs (i.e., the image change area) can be obtained, so as to obtain the image change detection result. Among them, the image change area may refer to the valid area of the image change mask map. The valid area of a mask sub-map or a mask map refers to the area composed of pixel points with values (i.e., the pixel points indicated by the pixel identification with values). Further, the valid area of a mask sub-map or a mask map refers to the smallest rectangular area composed of pixel points with values.

[0052] For example, as Figure 3 shown, the valid area of the image change mask map may be area 301. That is to say, the image change area may be area 301. Specifically, for any pixel point in the image change mask map, if the any pixel point is the pixel point indicated by the pixel identification with value (such as 1), it can be determined that the pixel point position corresponding to the any pixel point has changed, that is, the any pixel point is located in the image change area. Correspondingly, if the any pixel point is the pixel point indicated by the pixel identification without value (such as 0), it can be determined that the pixel point position corresponding to the any pixel point has not changed, that is, the any pixel point may not be located in the image change area.

[0053] In an embodiment of the present invention, after obtaining a reference image and a target image corresponding to the reference image, an image instance segmentation model can be called to perform instance segmentation on the target image and the reference image respectively, so as to obtain a set of target instance mask sub-images corresponding to the target image and a set of reference instance mask sub-images corresponding to the reference image. Any instance mask sub-image corresponding to any image refers to the mask of the instance corresponding to any instance mask sub-image under any image. The image acquisition time of the target image is later than the image acquisition time of the reference image. Then, based on the set of target instance mask sub-images and the set of reference instance mask sub-images, a sequence of difference mask sub-images can be generated. One difference mask sub-image is used to indicate the difference points between two mask sub-images. And according to the sequence of difference mask sub-images, an image change mask map can be generated, so as to determine an image change detection result based on the image change mask map. It can be seen that in the embodiment of the present invention, the reference image and the target image can be subjected to instance segmentation through the image instance segmentation model, without spending a large amount of time cost and labor cost to obtain a training data set for end-to-end model training, which can effectively save time cost and labor cost. Moreover, in the embodiment of the present invention, the change situation between the reference image and the target image can be accurately represented through the image change mask map, so as to obtain a relatively accurate image change detection result, and it can be applied to image change detection in any scenario. That is to say, in the embodiment of the present invention, the adaptability and accuracy rate of image change detection can be improved while ensuring relatively low time cost and labor cost.

[0054] Based on the above description, an embodiment of the present invention further proposes a more specific image change detection method. Correspondingly, this image change detection method can be executed by the above-mentioned electronic device (terminal or server); or, this image change detection method can be jointly executed by the terminal and the server. For the convenience of description, in the following, it is taken as an example that the electronic device executes this image change detection method for explanation; please refer to Figure 4 , this image change detection method may include the following steps S401-S406:

[0055] S401, obtain a reference image and a target image corresponding to the reference image, and the image acquisition time of the target image is later than the image acquisition time of the reference image.

[0056] S402, call an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, so as to obtain a set of target instance mask sub-images corresponding to the target image and a set of reference instance mask sub-images corresponding to the reference image. Any instance mask sub-image corresponding to any image refers to the mask of the instance corresponding to any instance mask sub-image under any image.

[0057] S403. Based on the target instance mask sub-graph set and the reference instance mask sub-graph set, determine the target mask sub-graph sequence and the reference mask sub-graph sequence, where the number of target mask sub-graphs in the target mask sub-graph sequence is greater than the number of reference mask sub-graphs in the reference mask sub-graph sequence.

[0058] In an embodiment of the present invention, the electronic device can perform deduplication processing on the target instance mask sub-graph set to obtain the deduplicated target instance mask sub-graph set; and perform deduplication processing on the reference instance mask sub-graph set to obtain the deduplicated reference instance mask sub-graph set, thereby removing the overlapping mask sub-graphs in the set, so that only a unique instance mask sub-graph corresponds to the same area in the image. In this case, the electronic device can perform deduplication processing on each instance mask sub-graph set after performing instance segmentation on the target image and the reference image respectively, so as to provide more accurate data for subsequent implementation processes, as Figure 5a shown.

[0059] Specifically, when performing deduplication processing on the target instance mask sub-graph set to obtain the deduplicated target instance mask sub-graph set, the electronic device can respectively determine the effective area of each target instance mask sub-graph in the target instance mask sub-graph set. The effective area of a mask sub-graph is used to indicate: the number of pixel points with values in the corresponding mask sub-graph, that is, the area size of the effective area in the corresponding mask sub-graph; then, sort each target instance mask sub-graph in descending order according to the effective area of each target instance mask sub-graph to obtain the target sorting result, and perform deduplication processing on the target instance mask sub-graph set according to the target sorting result and the first preset area ratio to obtain the deduplicated target instance mask sub-graph set. Among them, the first preset area ratio can be 0.5, or 0.6, etc., and the present invention does not limit this.

[0060] Based on this, when deduplicating the set of target instance mask subgraphs according to the target sorting result and the first preset area ratio to obtain the deduplicated set of target instance mask subgraphs, the electronic device can perform different ablation processes on every two target instance mask subgraphs in the target sorting result to obtain M different ablation mask subgraphs, where M is a positive integer; for any one of the M different ablation mask subgraphs, the electronic device can calculate the effective area of the any one of the different ablation mask subgraphs; if the effective area of the any one of the different ablation mask subgraphs is greater than the multiplication result of the effective area of the atomic graph corresponding to the any one of the different ablation mask subgraphs and the first preset area ratio (i.e., greater than the first preset area ratio times the effective area of the atomic graph corresponding to the any one of the different ablation mask subgraphs), then the other target instance mask subgraph except the atomic graph in the two target instance mask subgraphs corresponding to the any one of the different ablation mask subgraphs is removed from the set of target instance mask subgraphs, so as to implement deduplication of the set of target instance mask subgraphs and obtain the deduplicated set of target instance mask subgraphs, as Figure 5b shown. Among them, the atomic graph corresponding to the any one of the different ablation mask subgraphs is located before the other target instance mask subgraph corresponding to the any one of the different ablation mask subgraphs in the target sorting result.

[0061] Correspondingly, when deduplicating the set of reference instance mask subgraphs to obtain the deduplicated set of reference instance mask subgraphs, the electronic device can respectively determine the effective area of each reference instance mask subgraph in the set of reference instance mask subgraphs; sort the reference instance mask subgraphs according to the effective area of each reference instance mask subgraph from large to small to obtain a reference sorting result, and deduplicate the set of reference instance mask subgraphs according to the reference sorting result and the second preset area ratio to obtain the deduplicated set of reference instance mask subgraphs. The second preset area ratio can be 0.5, or 0.6, etc.; and the second preset area ratio can be the same as the first preset area ratio or different from the first preset area ratio, etc., and the present invention does not limit this.

[0062] Based on this, when deduplicating the set of reference instance mask subgraphs according to the reference sorting result and the second preset area ratio to obtain the deduplicated set of reference instance mask subgraphs, the electronic device can perform different ablation processes on every two reference instance mask subgraphs in the reference sorting result to obtain N different ablation mask subgraphs, where N is a positive integer; for any one of the N different ablation mask subgraphs, the electronic device can calculate the area of the effective region of the any one of the different ablation mask subgraphs; if the area of the effective region of the any one of the different ablation mask subgraphs is greater than the multiplication result between the area of the effective region of the atomic graph corresponding to the any one of the different ablation mask subgraphs and the second preset area ratio, then the other reference instance mask subgraph except the atomic graph among the two reference instance mask subgraphs corresponding to the any one of the different ablation mask subgraphs is removed from the set of reference instance mask subgraphs, so as to achieve deduplication of the set of reference instance mask subgraphs and obtain the deduplicated set of reference instance mask subgraphs. Among them, in the reference sorting result, the atomic graph is before the other reference instance mask subgraph among the two reference instance mask subgraphs corresponding to the any one of the different ablation mask subgraphs.

[0063] Optionally, for any one of the target sorting result and the reference sorting result, when performing different ablation processes on every two instance mask subgraphs in any one of the sorting results, the electronic device can traverse any one of the sorting results (i.e., the sorted set of instance mask subgraphs), and perform different ablation processes on the currently traversed instance mask subgraph and each instance mask subgraph after the currently traversed instance mask subgraph in any one of the sorting results one by one; or, if the currently traversed instance mask subgraph has not been subjected to different ablation processes with any instance mask subgraph in any one of the sorting results, then perform different ablation processes on the currently traversed instance mask subgraph and the first instance mask subgraph after the currently traversed instance mask subgraph in any one of the sorting results, and so on; the present invention does not limit this.

[0064] It should be noted that for the first instance mask subgraph and the second instance mask subgraph in any one of the sorting results (the first instance mask subgraph is before the second instance mask subgraph in any one of the sorting results), when performing different ablation processes on the first instance mask subgraph and the second instance mask subgraph, the electronic device can perform an intersection operation on the first instance mask subgraph and the second instance mask subgraph to achieve different ablation processes on the first instance mask subgraph and the second instance mask subgraph, thereby ablating the pixel points at the same pixel position with different pixel identifiers in the first instance mask subgraph and the second instance mask subgraph. Among them, the first instance mask subgraph can be any instance mask subgraph in any one of the sorting results. Optionally, the pixel identifier of the pixel points ablated in a different ablation mask subgraph can be a non-value pixel identifier.

[0065] Further, if the number of target instance mask subgraphs in the deduplicated target instance mask subgraph set is greater than or equal to the number of reference instance mask subgraphs in the deduplicated reference instance mask subgraph set, the deduplicated target instance mask subgraph set is used as the target mask subgraph sequence, and the deduplicated reference instance mask subgraph set is used as the reference mask subgraph sequence; if the number of target instance mask subgraphs is less than the number of reference instance mask subgraphs, the deduplicated reference instance mask subgraph set is used as the target mask subgraph sequence, and the deduplicated target instance mask subgraph set is used as the reference mask subgraph sequence. It can be seen that the embodiments of the present invention can use the instance mask subgraph set with a larger number of instance mask subgraphs as the target mask subgraph sequence, so as to facilitate more precise comparison between multiple mask subgraphs in the target mask subgraph sequence and the matching mask subgraphs that match them later, thereby improving the accuracy.

[0066] S404. Determine the matching mask subgraphs that match each target mask subgraph in the target mask subgraph sequence from the reference mask subgraph sequence respectively.

[0067] In a specific implementation, for any target mask subgraph in the target mask subgraph sequence, the electronic device can traverse each reference mask subgraph in the reference mask subgraph sequence and use the currently traversed reference mask subgraph as the current reference mask subgraph; then, based on any target mask subgraph and the current reference mask subgraph, determine the current transformation mask subgraph corresponding to the current reference mask subgraph under any target mask subgraph, and calculate the effective area of the current transformation mask subgraph; after traversing each reference mask subgraph in the reference mask subgraph sequence, obtain the effective areas of the transformation mask subgraphs corresponding to each reference mask subgraph under any target mask subgraph. Based on this, the electronic device can determine the matching mask subgraph that matches any target mask subgraph from the reference mask subgraph sequence based on the effective areas of the transformation mask subgraphs corresponding to each reference mask subgraph under any target mask subgraph. The effective area of the transformation mask subgraph corresponding to the matching mask subgraph that matches any target mask subgraph is less than the effective areas of the transformation mask subgraphs corresponding to other reference mask subgraphs in the reference mask subgraph sequence. In other words, the electronic device can traverse the target mask subgraph sequence and the reference mask subgraph sequence respectively to determine the matching mask subgraphs that match each target mask subgraph. Optionally, the matching mask subgraph that matches any target mask subgraph can also be referred to as the best matching mask subgraph of any target mask subgraph in the reference mask subgraph sequence. Among them, other reference mask subgraphs refer to any reference mask subgraph in the reference mask subgraph sequence except the matching mask subgraph that matches any target mask subgraph.

[0068] Optionally, when determining the current transformed mask sub-graph corresponding to the current reference mask sub-graph under any target mask sub-graph, the electronic device may perform an intersection operation on any target mask sub-graph and the current reference mask sub-graph to obtain an intersection mask sub-graph (i.e., the mask_and mask sub-graph), and perform a union operation on any target mask sub-graph and the current reference mask sub-graph to obtain a union mask sub-graph (i.e., the mask_or mask sub-graph); further, the electronic device may invert the intersection mask sub-graph (i.e., convert the pixel identification with a value to a pixel identification without a value, and convert the pixel identification without a value to a pixel identification with a value) to obtain an inverted mask sub-graph (i.e., the mask_not mask sub-graph), and perform an intersection operation on the inverted mask sub-graph and the union mask sub-graph to obtain the current transformed mask sub-graph corresponding to the current reference mask sub-graph under any target mask sub-graph (i.e., the mask_change mask sub-graph), as Figure 5c shown. Among them, the process of determining a transformed mask sub-graph may also be referred to as the same ablation processing process. That is to say, the electronic device may perform the same ablation processing on any of the above target mask sub-graph and the current reference mask sub-graph, so as to ablate the pixel points with the same pixel point positions and the same pixel identifications in any target mask sub-graph and the current reference mask sub-graph, so as to obtain the current transformed mask sub-graph; in this case, the pixel identification of the pixel points ablated in a transformed mask sub-graph may be a pixel identification without a value.

[0069] In the embodiments of the present invention, the intersection operation may refer to the AND operation. Then, when performing the intersection operation, if there is a pixel identification without a value at the same pixel point position in the two mask sub-graphs, the pixel identification without a value may be used to represent the pixel identification of the corresponding pixel point in the intersection operation result. If the pixel identifications at the same pixel point position in the two mask sub-graphs are both pixel identifications with a value, the pixel identification with a value may be used to represent the pixel identification of the corresponding pixel point in the intersection operation result. Exemplarily, assume that the pixel identification of the i-th pixel point in any target mask sub-graph is a pixel identification without a value, and the pixel identification of the i-th pixel point in the current reference mask sub-graph is a pixel identification with a value. Then, the pixel identification of the i-th pixel point in the above intersection mask sub-graph may be a pixel identification without a value; assume again that the pixel identification of the i-th pixel point in any target mask sub-graph is a pixel identification with a value, and the pixel identification of the i-th pixel point in the current reference mask sub-graph is a pixel identification with a value. Then, the pixel identification of the i-th pixel point in the above intersection mask sub-graph may be a pixel identification with a value, where i is a positive integer, and the value of i is less than or equal to the number of pixel points included in the image (such as the target image or the reference image).

[0070] Correspondingly, the union operation may refer to the OR operation. When performing the union operation, if there are value pixel identifiers at the same pixel position in two masked subgraphs, the value pixel identifier can be used to represent the pixel identifier of the corresponding pixel in the union operation result. If the pixel identifiers at the same pixel position in two masked subgraphs are both no-value pixel identifiers, the no-value pixel identifier can be used to represent the pixel identifier of the corresponding pixel in the union operation result. Exemplarily, assume that the pixel identifier of the i-th pixel in any target masked subgraph is a no-value pixel identifier, and the pixel identifier of the i-th pixel in the current reference masked subgraph is a value pixel identifier. Then, the pixel identifier of the i-th pixel in the above union masked subgraph can be a value pixel identifier. Also assume that the pixel identifier of the i-th pixel in any target masked subgraph is a no-value pixel identifier, and the pixel identifier of the i-th pixel in the current reference masked subgraph is a no-value pixel identifier. Then, the pixel identifier of the i-th pixel in the above union masked subgraph can be a no-value pixel identifier.

[0071] S405. Based on each target masked subgraph and the matching masked subgraph that matches it, generate a sequence of difference masked subgraphs. One difference masked subgraph is used to indicate the difference points between two masked subgraphs.

[0072] In a specific implementation, for any target masked subgraph in the target masked subgraph sequence, the electronic device can determine the first transformed masked subgraph corresponding to any target masked subgraph based on any target masked subgraph and the matching masked subgraph that matches it, and calculate the effective area ratio (i.e., the ratio area_ratio of the effective area of the first transformed masked subgraph to the effective area of any target masked subgraph) according to the effective area of the first transformed masked subgraph and the effective area of any target masked subgraph. If the effective area ratio is greater than or equal to the preset ratio threshold, add any target masked subgraph to the sequence of difference masked subgraphs. If the effective area ratio is less than the preset ratio threshold, determine the second transformed masked subgraph and add the second transformed masked subgraph to the sequence of difference masked subgraphs. Among them, the preset ratio threshold can be 1, or 1.1, etc. The present invention does not limit this. Optionally, in the embodiments of the present invention, the preset ratio threshold can be preferably 1, so as to avoid the number of difference points indicated by the difference masked subgraph corresponding to any target masked subgraph exceeding the number of value pixels in any target masked subgraph (i.e., the number of pixels with value pixel identifiers).

[0073] It should be understood that the process of determining the transformed mask subgraph can be the same ablation process. That is to say, the electronic device can perform the same ablation process on each target mask subgraph in the target mask subgraph sequence and the matching mask subgraph corresponding to each target mask subgraph, generate the difference mask subgraph corresponding to each target mask subgraph, and store it in the difference mask subgraph sequence. Based on this, the electronic device can perform the same ablation process on any target mask subgraph and the matching mask subgraph that matches it to obtain the first transformed mask subgraph. Specifically, when performing the same ablation process on any target mask subgraph and the matching mask subgraph that matches it to obtain the first transformed mask subgraph, the electronic device can perform an intersection operation on any target mask subgraph and the matching mask subgraph that matches it to obtain an intersection mask subgraph, and perform a union operation on any target mask subgraph and the matching mask subgraph that matches it to obtain a union mask subgraph. Further, the electronic device can invert the intersection mask subgraph to obtain an inverted mask subgraph, and perform an intersection operation on the inverted mask subgraph and the union mask subgraph to obtain the first transformed mask subgraph, as Figure 5d shown.

[0074] Further, when determining the second transformed mask subgraph, the electronic device can calculate the rectangle corresponding to any target mask subgraph according to the valid area of any target mask subgraph, and calculate the rectangle corresponding to the matching mask subgraph that matches any target mask subgraph according to the valid area of the matching mask subgraph that matches any target mask subgraph. The valid area of a mask subgraph refers to the area composed of pixel points with values. Then, based on the rectangle corresponding to any target mask subgraph and the rectangle corresponding to the matching mask subgraph that matches any target mask subgraph, the target rectangle can be determined, and any target mask subgraph and the matching mask subgraph that matches any target mask subgraph can be cropped according to the target rectangle respectively to obtain the cropped any target mask subgraph and the cropped matching mask subgraph. Based on this, the electronic device can perform the same ablation process on the cropped any target mask subgraph and the cropped matching mask subgraph to obtain the mask subgraph after the same ablation process, and use the mask subgraph after the same ablation process to determine the second transformed mask subgraph, as Figure 5d shown. Among them, the process of performing the same ablation process on the cropped any target mask subgraph and the cropped matching mask subgraph is the same as the process of performing the same ablation process on any target mask subgraph and the matching mask subgraph that matches it, and this embodiment of the present invention will not be elaborated here.

[0075] It should be understood that since the electronic devices are all cropped according to the target rectangular frame, any target mask sub - graph after cropping and the matching mask sub - graph after cropping have the same size; wherein, the target rectangular frame is the smallest rectangular frame that includes the rectangular frame corresponding to any target mask sub - graph and the rectangular frame corresponding to the matching mask sub - graph that matches any target mask sub - graph. That is to say, the target rectangular frame includes the rectangular frame corresponding to any target mask sub - graph and the rectangular frame corresponding to the matching mask sub - graph that matches any target mask sub - graph, and each boundary of the target rectangular frame coincides with one boundary of the rectangular frame corresponding to any target mask sub - graph or the rectangular frame corresponding to the matching mask sub - graph that matches any target mask sub - graph.

[0076] It should be noted that when determining the second transformed mask sub - graph from the mask sub - graphs after the same ablation process, the size of the mask sub - graph after the same ablation process can be restored according to the restored mask sub - graph (such as any target mask sub - graph or the matching mask sub - graph that matches any target mask sub - graph, etc.) to obtain the second transformed mask sub - graph, so that the size of the second transformed mask sub - graph is the same as the size of the restored mask sub - graph. The second transformed mask sub - graph includes the mask sub - graph after the same ablation process, and the pixel identifiers outside the target rectangular frame of the second transformed mask sub - graph can be the same as the pixel identifiers outside the target rectangular frame of the restored mask sub - graph, or the pixel identifiers outside the target rectangular frame of the second transformed mask sub - graph can all be no - value pixel identifiers.

[0077] S406, generate an image change mask map according to the sequence of difference mask sub - graphs, and determine the image change detection result based on the image change mask map.

[0078] In a specific implementation, the electronic device can initialize the image change mask map, traverse each difference mask sub - graph in the sequence of difference mask sub - graphs, and use the currently traversed difference mask sub - graph as the current difference mask sub - graph; if the area of the valid region of the current difference mask sub - graph is greater than or equal to the preset region area threshold, then superimpose the current difference mask sub - graph on the image change mask map; after traversing all the difference mask sub - graphs in the sequence of difference mask sub - graphs, the image change mask map is obtained. Among them, the preset region area threshold can be set according to experience or configured according to the actual business scenario, and the present invention does not limit this; it can be seen that the embodiments of the present invention can filter out the difference mask sub - graphs below the preset region area threshold from the sequence of difference mask sub - graphs according to the size of the valid region area, and then superimpose the remaining difference mask sub - graphs to generate the final image change mask map. Among them, the superimposing process can also be called the union operation process.

[0079] Among them, the image change detection result can refer to the image change region between the reference image and the target image (such as Figure 5eAs shown, it can also refer to a change identifier indicating whether the target image has changed relative to the reference image, etc.; the present invention does not limit this. It should be understood that the change identifier can be a change identifier (in this case, the image change detection result can be used to indicate that the target image has changed relative to the reference image), or it can be a non-change identifier (in this case, the image change detection result can be used to indicate that the target image has not changed relative to the reference image); optionally, the change identifier and the non-change identifier can be digital identifiers (such as 0 represents the non-change identifier, 1 represents the change identifier, etc.), or they can be letter identifiers (such as a represents the non-change identifier, b represents the change identifier, etc.), etc., and the present invention does not limit this.

[0080] In an embodiment of the present invention, after obtaining the reference image and the target image corresponding to the reference image, an image instance segmentation model can be called to perform instance segmentation on the target image and the reference image respectively, to obtain a set of target instance mask sub-images corresponding to the target image and a set of reference instance mask sub-images corresponding to the reference image, and based on the set of target instance mask sub-images and the set of reference instance mask sub-images, determine a target mask sub-image sequence and a reference mask sub-image sequence, where the number of target mask sub-images in the target mask sub-image sequence is greater than the number of reference mask sub-images in the reference mask sub-image sequence. Then, matching mask sub-images that match each target mask sub-image in the target mask sub-image sequence can be determined from the reference mask sub-image sequence respectively, and based on each target mask sub-image and the matching mask sub-images, a sequence of difference mask sub-images can be generated; based on this, an image change mask image can be generated according to the sequence of difference mask sub-images, so as to determine the image change detection result based on the image change mask image. Among them, the image instance segmentation model can be an image instance segmentation model based on deep learning; it can be seen that the embodiment of the present invention can use an image instance segmentation model based on deep learning to preprocess the image, thereby more accurately extracting all the instances included in the image, laying a reliable foundation for subsequent processing, and thus having stronger environmental adaptability and higher detection accuracy. Moreover, after using the image instance segmentation model to extract the instances in the image in the embodiment of the present invention, a more economical and practical technical means is adopted. Compared with the end-to-end detection method of deep learning, while ensuring the detection accuracy and robustness, the development time cost and labor cost can be greatly reduced.

[0081] Based on the description of the related embodiments of the above image change detection method, an embodiment of the present invention also proposes an image change detection device. The image change detection device can be a computer program (including program code) running in an electronic device; as Figure 6 shown, the image change detection device can include an acquisition unit 601 and a processing unit 602. The image change detection device can execute Figure 1 or Figure 4The image change detection method shown, that is, the image change detection device can run the above units:

[0082] An acquisition unit 601, configured to acquire a reference image and acquire a target image corresponding to the reference image, where the image acquisition time of the target image is later than the image acquisition time of the reference image;

[0083] A processing unit 602, configured to call an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, to obtain a set of target instance mask sub - graphs corresponding to the target image and a set of reference instance mask sub - graphs corresponding to the reference image. Any instance mask sub - graph corresponding to an image means: the mask of the instance corresponding to the any instance mask sub - graph under the any image;

[0084] The processing unit 602 is further configured to generate a sequence of difference mask sub - graphs based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs. One difference mask sub - graph is used to indicate: the difference points between two mask sub - graphs;

[0085] The processing unit 602 is further configured to generate an image change mask graph according to the sequence of difference mask sub - graphs, so as to determine an image change detection result based on the image change mask graph.

[0086] In one implementation, when generating the sequence of difference mask sub - graphs based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs, the processing unit 602 may specifically be configured to:

[0087] Based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs, determine a sequence of target mask sub - graphs and a sequence of reference mask sub - graphs, where the number of target mask sub - graphs in the sequence of target mask sub - graphs is greater than the number of reference mask sub - graphs in the sequence of reference mask sub - graphs;

[0088] Respectively determine matching mask sub - graphs that match each target mask sub - graph in the sequence of target mask sub - graphs from the sequence of reference mask sub - graphs;

[0089] Generate a sequence of difference mask sub - graphs based on each target mask sub - graph and the matching mask sub - graph that matches it.

[0090] In another implementation, when determining the sequence of target mask sub - graphs and the sequence of reference mask sub - graphs based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs, the processing unit 602 may specifically be configured to:

[0091] Perform a deduplication process on the set of target instance mask sub - graphs to obtain a deduplicated set of target instance mask sub - graphs;

[0092] Deduplicate the set of reference instance mask subgraphs to obtain a deduplicated set of reference instance mask subgraphs;

[0093] If the number of target instance mask subgraphs in the deduplicated set of target instance mask subgraphs is greater than or equal to the number of reference instance mask subgraphs in the deduplicated set of reference instance mask subgraphs, then use the deduplicated set of target instance mask subgraphs as the target mask subgraph sequence, and use the deduplicated set of reference instance mask subgraphs as the reference mask subgraph sequence;

[0094] If the number of target instance mask subgraphs is less than the number of reference instance mask subgraphs, then use the deduplicated set of reference instance mask subgraphs as the target mask subgraph sequence, and use the deduplicated set of target instance mask subgraphs as the reference mask subgraph sequence.

[0095] In another implementation, when the processing unit 602 deduplicates the set of target instance mask subgraphs to obtain a deduplicated set of target instance mask subgraphs, it can be specifically used for:

[0096] Respectively determine the effective area of each target instance mask subgraph in the set of target instance mask subgraphs. The effective area of a mask subgraph is used to indicate the number of pixels with values in the corresponding mask subgraph;

[0097] Sort the target instance mask subgraphs in descending order according to the effective area of each target instance mask subgraph to obtain a target sorting result, and deduplicate the set of target instance mask subgraphs according to the target sorting result and a first preset area ratio to obtain a deduplicated set of target instance mask subgraphs;

[0098] When the processing unit 602 deduplicates the set of reference instance mask subgraphs to obtain a deduplicated set of reference instance mask subgraphs, it can be specifically used for:

[0099] Respectively determine the effective area of each reference instance mask subgraph in the set of reference instance mask subgraphs;

[0100] Sort the reference instance mask subgraphs in descending order according to the effective area of each reference instance mask subgraph to obtain a reference sorting result, and deduplicate the set of reference instance mask subgraphs according to the reference sorting result and a second preset area ratio to obtain a deduplicated set of reference instance mask subgraphs.

[0101] In another implementation, when the processing unit 602 respectively determines the matching mask sub - graphs that match each target mask sub - graph in the target mask sub - graph sequence from the reference mask sub - graph sequence, it may specifically be used for:

[0102] For any target mask sub - graph in the target mask sub - graph sequence, traverse each reference mask sub - graph in the reference mask sub - graph sequence, and use the currently traversed reference mask sub - graph as the current reference mask sub - graph;

[0103] Based on the any target mask sub - graph and the current reference mask sub - graph, determine the current transformed mask sub - graph corresponding to the current reference mask sub - graph under the any target mask sub - graph, and calculate the area of the effective region of the current transformed mask sub - graph;

[0104] After traversing each reference mask sub - graph in the reference mask sub - graph sequence, obtain the areas of the effective regions of the transformed mask sub - graphs corresponding to each reference mask sub - graph under the any target mask sub - graph;

[0105] Based on the areas of the effective regions of the transformed mask sub - graphs corresponding to each reference mask sub - graph under the any target mask sub - graph, determine the matching mask sub - graph that matches the any target mask sub - graph from the reference mask sub - graph sequence. The area of the effective region of the transformed mask sub - graph corresponding to the matching mask sub - graph that matches the any target mask sub - graph is smaller than the areas of the effective regions of the transformed mask sub - graphs corresponding to other reference mask sub - graphs in the reference mask sub - graph sequence.

[0106] In another implementation, when the processing unit 602 generates a difference mask sub - graph sequence based on each target mask sub - graph and the matching mask sub - graphs that match them, it may specifically be used for:

[0107] For any target mask sub - graph in the target mask sub - graph sequence, based on the any target mask sub - graph and the matching mask sub - graph that matches it, determine the first transformed mask sub - graph corresponding to the any target mask sub - graph;

[0108] Calculate the effective region ratio according to the area of the effective region of the first transformed mask sub - graph and the area of the effective region of the any target mask sub - graph;

[0109] If the effective region ratio is greater than or equal to a preset ratio threshold, add the any target mask sub - graph to the difference mask sub - graph sequence;

[0110] If the effective region ratio is less than the preset ratio threshold, determine the second transformed mask sub - graph and add the second transformed mask sub - graph to the difference mask sub - graph sequence.

[0111] In another implementation, when determining the second transformed mask subgraph, the processing unit 602 may specifically be used for:

[0112] According to the valid region of any one of the target mask subgraphs, calculate the rectangular box corresponding to any one of the target mask subgraphs, and according to the valid region of the matching mask subgraph that matches any one of the target mask subgraphs, calculate the rectangular box corresponding to the matching mask subgraph that matches any one of the target mask subgraphs. The valid region of a mask subgraph refers to the region composed of pixel points with values;

[0113] Based on the rectangular box corresponding to any one of the target mask subgraphs and the rectangular box corresponding to the matching mask subgraph that matches any one of the target mask subgraphs, determine the target rectangular box, and respectively crop any one of the target mask subgraphs and the matching mask subgraph that matches any one of the target mask subgraphs according to the target rectangular box to obtain the cropped any one of the target mask subgraphs and the cropped matching mask subgraph;

[0114] Perform the same ablation processing on the cropped any one of the target mask subgraphs and the cropped matching mask subgraph to obtain the mask subgraph after the same ablation processing, and use the mask subgraph after the same ablation processing to determine the second transformed mask subgraph.

[0115] In another implementation, when generating the image change mask graph according to the sequence of difference mask subgraphs, the processing unit 602 may specifically be used for:

[0116] Initialize the image change mask graph, traverse each difference mask subgraph in the sequence of difference mask subgraphs, and use the currently traversed difference mask subgraph as the current difference mask subgraph;

[0117] If the area of the valid region of the current difference mask subgraph is greater than or equal to the preset region area threshold, then superimpose the current difference mask subgraph on the image change mask graph;

[0118] After traversing each difference mask subgraph in the sequence of difference mask subgraphs, obtain the image change mask graph.

[0119] In another implementation, when the acquisition unit 601 acquires the target image corresponding to the reference image, it may specifically be used for:

[0120] Acquire an initial image, and the image acquisition time of the initial image is later than the image acquisition time of the reference image;

[0121] Perform histogram matching on the initial image according to the histogram distribution of the reference image to obtain a target image corresponding to the reference image, so that the histogram distribution of the target image matches the histogram distribution of the reference image, and the image acquisition time of the target image is equal to the image acquisition time of the initial image.

[0122] According to an embodiment of the present invention, Figure 1 or Figure 4 Each step involved in the method shown can be performed by Figure 6 each unit in the image change detection device shown. For example, Figure 1 the step S101 shown in can be performed by Figure 6 the acquisition unit 601 shown in, and steps S102 - S104 can all be performed by Figure 6 the processing unit 602 shown in. Another example, Figure 4 the step S401 shown in can be performed by Figure 6 the acquisition unit 601 shown in, and steps S402 - S406 can all be performed by Figure 6 the processing unit 602 shown in, and so on.

[0123] According to another embodiment of the present invention, Figure 6 Each unit in the image change detection device shown can be respectively or all combined into one or several other units to form, or a certain one (or some) of the units can be further split into multiple smaller units in terms of function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present invention, any image change detection device can also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0124] According to another embodiment of the present invention, it can be achieved by running a computer program (including program code) that can execute each step involved in the corresponding method shown in Figure 1 or Figure 4 on a general - purpose electronic device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random - access storage medium (RAM), and a read - only storage medium (ROM), to construct an image change detection device as shown in Figure 6 and to implement the image change detection method of the embodiments of the present invention. The computer program can be recorded on a computer storage medium, for example, and loaded into the above - mentioned electronic device through the computer storage medium and run therein.

[0125] In an embodiment of the present invention, after obtaining a reference image and a target image corresponding to the reference image, an image instance segmentation model may be called to perform instance segmentation on the target image and the reference image respectively, so as to obtain a set of target instance mask sub - graphs corresponding to the target image and a set of reference instance mask sub - graphs corresponding to the reference image. Any instance mask sub - graph corresponding to an image refers to the mask of the instance corresponding to the any instance mask sub - graph under the any image. The image acquisition time of the target image is later than the image acquisition time of the reference image. Then, based on the set of target instance mask sub - graphs and the set of reference instance mask sub - graphs, a sequence of difference mask sub - graphs may be generated. A difference mask sub - graph is used to indicate the difference points between two mask sub - graphs; and an image change mask graph may be generated according to the sequence of difference mask sub - graphs, so as to determine an image change detection result based on the image change mask graph. It can be seen that in the embodiment of the present invention, the reference image and the target image can be subjected to instance segmentation by using an image instance segmentation model, without spending a large amount of time cost and labor cost to obtain a training data set for end - to - end model training, which can effectively save time cost and labor cost; moreover, in the embodiment of the present invention, the change situation between the reference image and the target image can be accurately represented by using the image change mask graph, so as to obtain a relatively accurate image change detection result, and it can be applied to image change detection in any scenario; that is to say, in the embodiment of the present invention, the adaptability and accuracy rate of image change detection can be improved while ensuring relatively low time cost and labor cost.

[0126] Based on the descriptions of the above - mentioned method embodiments and apparatus embodiments, an exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program that can be executed by the at least one processor, and when the computer program is executed by the at least one processor, it is used to cause the electronic device to execute the method according to the embodiment of the present invention.

[0127] An exemplary embodiment of the present invention further provides a non - transitory computer - readable storage medium storing a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the method according to the embodiment of the present invention.

[0128] An exemplary embodiment of the present invention further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the method according to the embodiment of the present invention.

[0129] Reference Figure 7, a structural block diagram of an electronic device 700 that can be a server or a client of the present invention will now be described. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0130] As Figure 7 shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0131] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 can be any type of device that can input information into the electronic device 700. The input unit 706 can receive input digital or character information, and generate key signal inputs related to the user settings and / or function controls of the electronic device. The output unit 707 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 708 can include but is not limited to magnetic disks, optical disks. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a BluetoothTM device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0132] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above. For example, in some embodiments, the image change detection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 can be configured to execute the image change detection method in any other suitable way (e.g., by means of firmware).

[0133] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] As used in this invention, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0136] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0137] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0138] A computer system can include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0139] Also, it should be understood that what is disclosed above are only the preferred embodiments of the present invention, and of course cannot be used to limit the scope of the rights of the present invention. Therefore, equivalent changes made in accordance with the claims of the present invention still fall within the scope covered by the present invention.

Claims

1. An image change detection method, characterized in that, Including: Obtaining a reference image and obtaining a target image corresponding to the reference image, where the image acquisition time of the target image is later than the image acquisition time of the reference image; Invoking an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, to obtain a set of target instance mask sub-images corresponding to the target image and a set of reference instance mask sub-images corresponding to the reference image. Any instance mask sub-image corresponding to an image means: the mask of the instance corresponding to the any instance mask sub-image under the any image; wherein, any mask sub-image includes pixel identifiers of each pixel point in the image corresponding to the any mask sub-image, and a pixel identifier includes either a valued pixel identifier or a non-valued pixel identifier; Generating a sequence of difference mask sub-images based on the set of target instance mask sub-images and the set of reference instance mask sub-images, where one difference mask sub-image is used to indicate: the difference points between two mask sub-images; Generating an image change mask map according to the sequence of difference mask sub-images, to determine an image change detection result based on the image change mask map; wherein, the image change mask map is used to indicate the image change area between the reference image and the target image.

2. The method according to claim 1, wherein The generating a sequence of difference mask sub-images based on the set of target instance mask sub-images and the set of reference instance mask sub-images includes: Determining a sequence of target mask sub-images and a sequence of reference mask sub-images based on the set of target instance mask sub-images and the set of reference instance mask sub-images, where the number of target mask sub-images in the sequence of target mask sub-images is greater than the number of reference mask sub-images in the sequence of reference mask sub-images; Respectively determining, from the sequence of reference mask sub-images, matching mask sub-images that match each target mask sub-image in the sequence of target mask sub-images; Generating a sequence of difference mask sub-images based on each target mask sub-image and the matching mask sub-images that match it.

3. The method according to claim 2, wherein The determining a sequence of target mask sub-images and a sequence of reference mask sub-images based on the set of target instance mask sub-images and the set of reference instance mask sub-images includes: Performing deduplication processing on the set of target instance mask sub-images to obtain a deduplicated set of target instance mask sub-images; Performing deduplication processing on the set of reference instance mask sub-images to obtain a deduplicated set of reference instance mask sub-images; If the number of target instance mask sub-images in the deduplicated set of target instance mask sub-images is greater than or equal to the number of reference instance mask sub-images in the deduplicated set of reference instance mask sub-images, then using the deduplicated set of target instance mask sub-images as the sequence of target mask sub-images and using the deduplicated set of reference instance mask sub-images as the sequence of reference mask sub-images; If the number of target instance mask sub-images is less than the number of reference instance mask sub-images, then using the deduplicated set of reference instance mask sub-images as the sequence of target mask sub-images and using the deduplicated set of target instance mask sub-images as the sequence of reference mask sub-images.

4. The method according to claim 3, characterized in that, The performing deduplication processing on the set of target instance mask sub-images to obtain a deduplicated set of target instance mask sub-images includes: Determine the effective region area of each target instance mask sub - graph in the set of target instance mask sub - graphs respectively. The effective region area of a mask sub - graph is used to indicate the number of pixels with values in the corresponding mask sub - graph. Sort the target instance mask sub - graphs according to the effective region areas of the target instance mask sub - graphs from large to small to obtain a target sorting result, and perform deduplication processing on the set of target instance mask sub - graphs according to the target sorting result and a first preset area ratio to obtain a deduplicated set of target instance mask sub - graphs. The process of performing deduplication processing on the set of reference instance mask sub - graphs to obtain a deduplicated set of reference instance mask sub - graphs includes: Determine the effective region area of each reference instance mask sub - graph in the set of reference instance mask sub - graphs respectively. Sort the reference instance mask sub - graphs according to the effective region areas of the reference instance mask sub - graphs from large to small to obtain a reference sorting result, and perform deduplication processing on the set of reference instance mask sub - graphs according to the reference sorting result and a second preset area ratio to obtain a deduplicated set of reference instance mask sub - graphs.

5. The method according to claim 2, wherein The process of respectively determining the matching mask sub - graphs that match each target mask sub - graph in the target mask sub - graph sequence from the reference mask sub - graph sequence includes: For any target mask sub - graph in the target mask sub - graph sequence, traverse each reference mask sub - graph in the reference mask sub - graph sequence and use the currently traversed reference mask sub - graph as the current reference mask sub - graph. Based on the any target mask sub - graph and the current reference mask sub - graph, determine the current transformed mask sub - graph corresponding to the current reference mask sub - graph under the any target mask sub - graph, and calculate the effective region area of the current transformed mask sub - graph. After traversing all the reference mask sub - graphs in the reference mask sub - graph sequence, obtain the effective region areas of the transformed mask sub - graphs corresponding to each reference mask sub - graph under the any target mask sub - graph. Based on the effective region areas of the transformed mask sub - graphs corresponding to each reference mask sub - graph under the any target mask sub - graph, determine the matching mask sub - graph that matches the any target mask sub - graph from the reference mask sub - graph sequence. The effective region area of the transformed mask sub - graph corresponding to the matching mask sub - graph that matches the any target mask sub - graph is smaller than the effective region areas of the transformed mask sub - graphs corresponding to other reference mask sub - graphs in the reference mask sub - graph sequence.

6. The method according to claim 2, characterized in that, The process of generating a difference mask sub - graph sequence based on the target mask sub - graphs and the matching mask sub - graphs that match them includes: For any target mask sub - graph in the target mask sub - graph sequence, determine the first transformed mask sub - graph corresponding to the any target mask sub - graph based on the any target mask sub - graph and the matching mask sub - graph that matches it. Calculate the effective region ratio according to the effective region area of the first transformed mask sub - graph and the effective region area of the any target mask sub - graph. If the effective area ratio is greater than or equal to a preset ratio threshold, add any one of the target mask subgraphs to the difference mask subgraph sequence; If the effective area ratio is less than the preset ratio threshold, determine a second transformed mask subgraph and add the second transformed mask subgraph to the difference mask subgraph sequence.

7. The method according to claim 6, wherein The determining of the second transformed mask subgraph includes: Calculate the rectangular box corresponding to any one of the target mask subgraphs according to the effective area of any one of the target mask subgraphs, and calculate the rectangular box corresponding to the matching mask subgraph that matches any one of the target mask subgraphs according to the effective area of the matching mask subgraph that matches any one of the target mask subgraphs. The effective area of a mask subgraph refers to the area composed of pixel points with values; Based on the rectangular box corresponding to any one of the target mask subgraphs and the rectangular box corresponding to the matching mask subgraph that matches any one of the target mask subgraphs, determine a target rectangular box, and crop any one of the target mask subgraphs and the matching mask subgraph that matches any one of the target mask subgraphs according to the target rectangular box respectively to obtain the cropped any one of the target mask subgraphs and the cropped matching mask subgraph; Perform the same ablation processing on the cropped any one of the target mask subgraphs and the cropped matching mask subgraph to obtain the mask subgraph after the same ablation processing, and use the mask subgraph after the same ablation processing to determine the second transformed mask subgraph.

8. The method according to any one of claims 1-7, characterized in that The generating of the image change mask graph according to the difference mask subgraph sequence includes: Initialize the image change mask graph, traverse each difference mask subgraph in the difference mask subgraph sequence, and use the currently traversed difference mask subgraph as the current difference mask subgraph; If the effective area of the current difference mask subgraph is greater than or equal to a preset area threshold, superimpose the current difference mask subgraph on the image change mask graph; After traversing each difference mask subgraph in the difference mask subgraph sequence, obtain the image change mask graph.

9. The method according to any one of claims 1 to 7, characterized in that, The obtaining of the target image corresponding to the reference image includes: Obtain an initial image, and the image acquisition time of the initial image is later than the image acquisition time of the reference image; Perform histogram matching on the initial image according to the histogram distribution of the reference image to obtain the target image corresponding to the reference image, so that the histogram distribution of the target image matches the histogram distribution of the reference image, and the image acquisition time of the target image is equal to the image acquisition time of the initial image.

10. An image change detection device, characterized in that, The device includes: An obtaining unit, configured to obtain a reference image and obtain a target image corresponding to the reference image, where the image acquisition time of the target image is later than the image acquisition time of the reference image; A processing unit, configured to call an image instance segmentation model to perform instance segmentation on the target image and the reference image respectively, so as to obtain a target instance mask sub-graph set corresponding to the target image and a reference instance mask sub-graph set corresponding to the reference image. Any instance mask sub-graph corresponding to an image refers to: the mask of the instance corresponding to the any instance mask sub-graph under the any image; wherein, any mask sub-graph includes pixel identifiers of each pixel point in the image corresponding to the any mask sub-graph, and a pixel identifier includes either a pixel identifier with a value or a pixel identifier without a value. The processing unit is further configured to generate a sequence of difference mask sub-graphs based on the target instance mask sub-graph set and the reference instance mask sub-graph set. A difference mask sub-graph is used to indicate: the difference points between two mask sub-graphs. The processing unit is further configured to generate an image change mask graph according to the sequence of difference mask sub-graphs, so as to determine an image change detection result based on the image change mask graph; wherein, the image change mask graph is used to indicate the image change region between the reference image and the target image.

11. An electronic device, characterized in that, Comprising: A processor; And A memory storing a program, wherein, the program includes instructions that, when executed by the processor, cause the processor to execute the method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-9.

Images