Change detection method, electronic device and computer readable storage medium

Abstract

The present application discloses a change detection method, an electronic device, and a computer-readable storage medium, the method comprising: using a twin detection model to obtain the corresponding feature difference between a background image and a detection image, and determining a first change region in the background image and a second change region in the detection image based on the feature difference; wherein the two detection branches in the twin detection model respectively extract features from the background image and the detection image; using an image encoder to obtain a first image feature of the first change region and a second image feature of the second change region, determining a forward description text and a reverse description text matching the first change region and the second change region, and using a text encoder to obtain a first text feature of the forward description text and a second text feature of the reverse description text; based on the first image feature, the second image feature, the first text feature, and the second text feature, determining the change detection result of the detection image compared to the background image. The above scheme can improve the accuracy of change detection.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a change detection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] As an important branch of computer vision, change detection mainly detects the differences between two images captured from the same scene. In the prior art, image features are usually compared to determine the area in the image that has changed, but it is difficult to determine what kind of target caused the change. Therefore, the accuracy of change detection is not high. In view of this, how to improve the accuracy of change detection has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a change detection method, an electronic device and a computer-readable storage medium, which can improve the accuracy of change detection.

[0004] To solve the above technical problems, the first aspect of the present application provides a change detection method, including: using a twin detection model to obtain the corresponding feature difference between a background image and a detection image, and determining a first change area in the background image and a second change area in the detection image based on the feature difference; wherein the two detection branches in the twin detection model respectively extract features from the background image and the detection image; using an image encoder to obtain a first image feature of the first change area and a second image feature of the second change area, determining a forward description text and a reverse description text matching the first change area and the second change area, and using a text encoder to obtain a first text feature of the forward description text and a second text feature of the reverse description text; based on the first image feature, the second image feature, the first text feature and the second text feature, determining a change detection result of the detection image compared to the background image.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] The above scheme uses the twin detection model to obtain the corresponding feature difference between the background image and the detection image, determines the change area in the background image and the detection image based on the feature difference, obtains the first change area from the background image and obtains the second change area from the detection image, wherein the two detection branches in the twin detection model respectively extract features from the background image and the detection image to facilitate the convenient determination of the feature difference. The image encoder is used to extract features from the first change area to obtain the first image feature, and the second change area is used to extract features from the second change area to obtain the second image feature. The forward description text and the reverse description text corresponding to the two description methods matching the first change area and the second change area are determined, and the text encoder is used to extract features from the forward description text to obtain the first text feature, and the reverse description text is used to extract features from the reverse description text to obtain the second text feature. Based on the first image feature, the second image feature, the first text feature and the second text feature, the changes in the detection image compared to the background image are confirmed to obtain a change detection result of the detection image compared to the background image. By comparing the first image feature obtained in the background image with the text features of the two texts of the forward description and the reverse description, and comparing the first image feature obtained in the detection image with the text features of the two texts of the forward description and the reverse description, the change detection result of the detection image compared to the background image is accurately determined, so that the change detection result can be matched with one of the description methods of the forward description and the reverse description, thereby clarifying the change caused by which target and improving the accuracy of change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flow chart of an implementation method of a change detection method of the present application;

[0010] Figure 2 It is a flow chart of another implementation method of the change detection method of the present application;

[0011] Figure 3 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0012] Figure 4 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0015] The change detection method provided in the present application is used to detect changes in a detection image compared to a background image, and its corresponding execution subject is a processing unit capable of performing data processing.

[0016] See also Figure 1 , Figure 1 : is a flow chart of an implementation method of a change detection method of the present application, the method comprising:

[0017] S101: Use the twin detection model to obtain the corresponding feature difference between the background image and the detection image, and determine the first change area in the background image and the second change area in the detection image based on the feature difference; wherein the two detection branches in the twin detection model respectively extract features from the background image and the detection image.

[0018] Specifically, the twin detection model is used to obtain the corresponding feature difference between the background image and the detection image, and the change area in the background image and the detection image is determined based on the feature difference. The first change area is obtained from the background image and the second change area is obtained from the detection image. The two detection branches in the twin detection model respectively extract features from the background image and the detection image to facilitate the convenient determination of the feature difference.

[0019] In some implementation scenarios, the two detection branches of the twin detection model correspond to two convolutional neural networks, one of the convolutional neural networks is used to extract features of the background image to obtain background image features, and the other convolutional neural network is used to extract features of the detection image to obtain detection image features, the feature difference between the background image features and the detection image features is determined, and the position of the changed area is regressed based on the feature difference to obtain the changed area in the background image and the detection image, the first changed area is obtained from the background image, and the second changed area is obtained from the detection image.

[0020] In some implementation scenarios, the two detection branches of the twin detection model correspond to two long short-term memory networks, one of which is used to extract features of the background image to obtain background image features, and the other long short-term memory network is used to extract features of the detection image to obtain detection image features, the feature difference between the background image features and the detection image features is determined, and prompt text matching the feature difference for extracting the change area is obtained, the feature difference and the prompt text are input into the large language model to obtain the change area output by the large language model, the first change area is obtained from the background image, and the second change area is obtained from the detection image.

[0021] Optionally, the features of the background image correspond to multi-channel background image features, and the features of the detection image correspond to multi-channel detection image features. The multi-channel background image features and the multi-channel detection image features are cross-fused according to the channels to obtain a first cross-fused feature corresponding to the background image and a second cross-fused feature corresponding to the detection image. The first cross-fused feature and the second cross-fused feature are subtracted to obtain the feature difference between the background image and the detection image. Therefore, in the process of obtaining the feature difference, the channel cross-fusion is used to effectively reduce the impact of light and shadow changes in the background image and the detection image, and improve the accuracy of the changed area obtained based on the feature difference.

[0022] In addition, the large language model may include but is not limited to deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTM), and generative pre-trained Transformer models, etc. No specific restrictions are imposed on the specific construction and deployment of the large language model.

[0023] S102: Using an image encoder to obtain a first image feature of a first change region and a second image feature of a second change region, determining a forward description text and a reverse description text that match the first change region and the second change region, and using a text encoder to obtain a first text feature of the forward description text and a second text feature of the reverse description text.

[0024] Specifically, an image encoder is used to extract features from the first change region to obtain a first image feature, and a feature is extracted from the second change region to obtain a second image feature. A forward description text and a reverse description text corresponding to two description methods matching the first change region and the second change region are determined. A text encoder is used to extract features from the forward description text to obtain a first text feature, and a feature is extracted from the reverse description text to obtain a second text feature.

[0025] It should be noted that the positive description text and the negative description text correspond to two opposite description methods. The positive description text is usually used to express that a certain category of targets exist in the picture, and the negative description text is usually used to express that a certain category of targets does not exist in the picture.

[0026] In some implementation scenarios, an image encoder is used to perform feature extraction on the first change region and the second change region, respectively, to obtain first image features corresponding to the first change region and second image features corresponding to the second change region, to obtain a preset description text matching the background image, and to construct a forward description text and a reverse description text matching the background image and the detection image based on the preset description text, and to perform feature extraction on the forward description text and the reverse description text, respectively, to obtain a first text feature corresponding to the forward description text and a second text feature corresponding to the reverse description text.

[0027] In some implementation scenarios, an image encoder is used to perform feature extraction on the first change region and the second change region respectively to obtain first image features corresponding to the first change region and second image features corresponding to the second change region, and the target description text and the corresponding description method set for the background image and the detection image are obtained. The target description text is converted based on the target description text and the corresponding description method to obtain forward description text and reverse description text corresponding to the two description methods, and a text encoder is used to perform feature extraction on the forward description text and the reverse description text respectively to obtain first text features corresponding to the forward description text and second text features corresponding to the reverse description text.

[0028] It should be noted that the image features and text features are extracted using a multimodal pre-training model, which includes an image encoder and a text encoder, wherein the first change region and the second change region can correspond to the same image encoder, or each can correspond to an image encoder, and the forward description text and the reverse description text can correspond to the same text encoder, or each can correspond to a text encoder. This application does not impose any specific restrictions on this.

[0029] S103: Determine a change detection result of the detection image compared with the background image based on the first image feature, the second image feature, the first text feature, and the second text feature.

[0030] Specifically, based on the first image feature, the second image feature, the first text feature, and the second text feature, changes in the detection image compared to the background image are confirmed to obtain a change detection result of the detection image compared to the background image.

[0031] It should be noted that by comparing the first image feature obtained in the background image with the text features of the two texts of the forward description and the reverse description, and by comparing the first image feature obtained in the detection image with the text features of the two texts of the forward description and the reverse description, the change detection result of the detection image compared with the background image can be accurately determined, so that the change detection result can be matched with one of the description methods of the forward description and the reverse description, thereby clarifying the change caused by the target and improving the accuracy of change detection.

[0032] In some implementation scenarios, based on the first image feature, the first text feature, and the second text feature, the target recognition result of the first change area and the description of its match are determined; based on the second image feature, the first text feature, and the second text feature, the target recognition result of the second change area and the description of its match are determined; based on the target recognition result of the first change area and the description of its match, and the target recognition result of the second change area and the description of its match, the change detection result of the detection image compared with the background image is determined.

[0033] In some implementation scenarios, based on the similarity between the first image feature and the first text feature and the second text feature, the first descriptive text matching the first image feature is determined; based on the similarity between the second image feature and the first text feature and the second text feature, the second descriptive text matching the second image feature is determined; and based on the first descriptive text and the second descriptive text, a change detection result of the detection image compared to the background image is determined.

[0034] The above scheme uses the twin detection model to obtain the corresponding feature difference between the background image and the detection image, determines the change area in the background image and the detection image based on the feature difference, obtains the first change area from the background image and obtains the second change area from the detection image, wherein the two detection branches in the twin detection model respectively extract features from the background image and the detection image to facilitate the convenient determination of the feature difference. The image encoder is used to extract features from the first change area to obtain the first image feature, and the second change area is used to extract features from the second change area to obtain the second image feature. The forward description text and the reverse description text corresponding to the two description methods matching the first change area and the second change area are determined, and the text encoder is used to extract features from the forward description text to obtain the first text feature, and the reverse description text is used to extract features from the reverse description text to obtain the second text feature. Based on the first image feature, the second image feature, the first text feature and the second text feature, the changes in the detection image compared to the background image are confirmed to obtain a change detection result of the detection image compared to the background image. By comparing the first image feature obtained in the background image with the text features of the two texts of the forward description and the reverse description, and comparing the first image feature obtained in the detection image with the text features of the two texts of the forward description and the reverse description, the change detection result of the detection image compared to the background image is accurately determined, so that the change detection result can be matched with one of the description methods of the forward description and the reverse description, thereby clarifying the change caused by which target and improving the accuracy of change detection.

[0035] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the change detection method of the present application, the method comprising:

[0036] S201: Use the two detection branches of the twin detection model to respectively extract the multi-channel background image features corresponding to the background image and the multi-channel detection image features corresponding to the detection image, and cross-fuse the multi-channel background image features with the multi-channel detection image features according to channels to obtain a first cross-fusion feature corresponding to the background image and a second cross-fusion feature corresponding to the detection image.

[0037] Specifically, one of the two detection branches is used to perform feature extraction on the background image to obtain multi-channel background image features corresponding to the background image, and the other detection branch of the two detection branches is used to perform feature extraction on the detection image to obtain multi-channel detection image features corresponding to the detection image. The multi-channel background image features and the multi-channel detection image features are cross-fused in a manner that the channels are cross-linked to obtain a first cross-fused feature corresponding to the background image and a second cross-fused feature corresponding to the detection image.

[0038] For the sake of illustration, the multi-channel background image features are 256 Hb Wb, multi-channel detection map features are 256 Hd That is, when the number of channels is 256, the first cross-fusion feature corresponding to the background image obtained after channel-level cross-fusion is Hb1 Wb1, Hd2 Wd2、Hb3 Wb3 HD256 Wd256. Similarly, the second cross-fusion feature corresponding to the detection image obtained after channel-level cross-fusion is Hd1. Wd1, Hb2 Wb2、Hd3 Wd3 Hb256 Wb256.

[0039] S202: Subtract the first cross-fusion feature from the second cross-fusion feature to obtain a corresponding feature difference between the background image and the detection image.

[0040] Specifically, the feature difference between the first cross-fusion feature and the second cross-fusion feature is obtained, so as to obtain the corresponding feature difference between the background image and the detection image. The first cross-fusion feature and the second cross-fusion feature realize channel-level cross-fusion, so that when obtaining the feature difference, the changes caused by light and shadow can be effectively filtered out, and the feature difference caused by the target change is mainly retained, thereby improving the accuracy of the feature difference.

[0041] In some implementation scenarios, the two detection branches of the twin detection model are used to extract multi-channel background image features corresponding to the background image and multi-channel detection image features corresponding to the detection image. Subtracting the first cross-fusion feature from the second cross-fusion feature to obtain the feature difference corresponding to the background image and the detection image includes: subtracting the first cross-fusion feature and the second cross-fusion feature obtained from at least part of the feature layers in the two detection branches to obtain at least one feature difference corresponding to the background image and the detection image.

[0042] Specifically, the two detection branches of the twin detection model correspond to the same structure. Both detection branches include multiple feature layers. Each feature layer extracts features of different dimensions in turn, and each feature layer corresponds to channel-level cross-fusion. The first cross-fusion feature and the second cross-fusion feature obtained from at least part of the feature layers in the two detection branches are used to perform the subtraction in the same layer to obtain at least one feature difference between the background image and the detection image. Among them, the feature layer used to obtain the feature difference is usually the feature layer at the back of the detection branch, and multiple feature differences are usually selected to obtain feature differences of different dimensions.

[0043] For the sake of convenience, taking the last three feature layers in the detection branch as an example, the last three feature maps (Fb1, Fb2, Fb3) of the background image detection branch are subtracted from the last three feature maps (Fd1, Fd2, Fd3) of the detection image detection branch to obtain the three-layer feature differences Fdiff1, Fdiff2, and Fdiff3. The sizes of the three-layer feature differences are C1, C2, and C3, respectively. H1 W1, C2 H2 W2, C3 H3 W3.

[0044] Optionally, the detection branches of the twin detection model can all adopt the YOLOv8 (You Only Look Once version 8) backbone network or other open source networks, and this application does not impose any specific restrictions on this.

[0045] S203: Determine the coordinates of the changed region in the detection image compared with the background image based on the feature difference, and obtain a first changed region in the background image and a second changed region in the detection image using the changed region coordinates.

[0046] Specifically, based on the feature difference, a position regression is performed on the position where the detection image has changed compared to the background image to determine the coordinates of the changed area, and the change area coordinates are used to obtain the first change area corresponding to the change area coordinates from the background image, and the change area coordinates are used to obtain the second change area corresponding to the change area coordinates from the detection image.

[0047] In some implementation scenarios, the twin detection network also includes an attention network and a detection head. Determining the coordinates of the changed area of ​​the detection image compared to the background image based on the feature difference includes: using the attention network of the twin detection model to obtain the attention features corresponding to each feature difference; using the detection head of the twin detection model to perform position regression on all attention features to obtain the coordinates of the changed area of ​​the detection image compared to the background image.

[0048] Specifically, the attention network of the twin detection model is used to extract features from the channel attention and spatial attention for each feature difference, so that the attention features corresponding to each feature difference can be obtained, so that the attention features can focus on small or slender targets that are difficult to identify. The detection head of the twin detection model is used to perform position regression based on all attention features to obtain the coordinates of the changed area of ​​the detection image compared to the background image, thereby improving the accuracy of the coordinates of the changed area.

[0049] Optionally, the attention network can adopt a convolutional attention module (Convolutional Block Attention Module, CBAM), or a self-attention module or a Transformer module, and this application does not make any specific restrictions on this.

[0050] S204: Using an image encoder to obtain a first image feature of the first change region and a second image feature of the second change region, determining a forward description text and a reverse description text that match the first change region and the second change region, and using a text encoder to obtain a first text feature of the forward description text and a second text feature of the reverse description text.

[0051] Specifically, an image encoder is used to extract features from the first change region to obtain a first image feature, and a feature is extracted from the second change region to obtain a second image feature. A forward description text and a reverse description text corresponding to two description methods matching the first change region and the second change region are determined. A text encoder is used to extract features from the forward description text to obtain a first text feature, and a feature is extracted from the reverse description text to obtain a second text feature.

[0052] In some implementation scenarios, an image encoder is used to perform feature recognition on the first change region to obtain a first image feature, and an image encoder is used to perform feature recognition on the second change region to obtain a second image feature; a target description text matching the first change region and the second change region is obtained, and a text encoder is used to adjust the target description text according to a forward description method and a reverse description method to obtain a forward description text and a reverse description text; a text encoder is used to perform feature recognition on the forward description text to obtain a first text feature, and a text encoder is used to perform feature recognition on the reverse description text to obtain a second text feature.

[0053] Specifically, image features and text features are extracted using a multimodal pre-training model, which includes an image encoder and a text encoder. The image encoder is used to perform feature recognition on the first change region to obtain a first image feature corresponding to the first change region. The image encoder is used to perform feature recognition on the second change region to obtain a second image feature corresponding to the second change region.

[0054] Furthermore, a target description text matching the first change region and the second change region is obtained, and the target description text is adjusted according to two different description methods using a text encoder to determine the forward description text obtained after adjustment according to the forward description method and the reverse description text obtained after adjustment according to the reverse description method.

[0055] It is understandable that the background image is usually marked with targets included in different areas as a reference standard, and the target description text can be determined based on the targets marked in the background image and the coordinates of the change area. In addition, in addition to the preset partially marked targets in the background image, the targets present in the detection image can be indicated when the detection image is input, and the target description text for the change area can be obtained after obtaining the first change area and the second change area. Therefore, the text encoder can easily obtain the target description text and adjust the target description text according to two opposite description methods to obtain the forward description text and the reverse description text, thereby improving the convenience of text generation.

[0056] Furthermore, a text encoder is used to perform feature recognition on the forward description text to obtain a first text feature corresponding to the forward description text, and a text encoder is used to perform feature recognition on the reverse description text to obtain a second text feature corresponding to the reverse description text, thereby obtaining a multimodal feature corresponding to each change area.

[0057] It should be noted that the target description text includes at least one of a target to be identified and a target to be filtered, and the target to be identified and the target to be filtered have respective categories.

[0058] Specifically, the target description text can indicate the targets to be identified and their corresponding categories, and can also indicate the targets to be filtered and their corresponding categories, so that changes caused by targets of different categories can be flexibly detected, specific target filtering or non-filtering can be achieved, and the changes caused by the targets to be identified can be output.

[0059] S205: Determine a change detection result of the detection image compared with the background image based on the first image feature, the second image feature, the first text feature, and the second text feature.

[0060] Specifically, based on the first image feature, the second image feature, the first text feature, and the second text feature, changes in the detection image compared to the background image are confirmed to obtain a change detection result of the detection image compared to the background image.

[0061] In some implementation scenarios, a first forward similarity between a first image feature and a first text feature, and a first reverse similarity between the first image feature and a second text feature are obtained, and based on the first forward similarity and the first reverse similarity, a first target recognition result in a first change region is determined; and a second forward similarity between a second image feature and the first text feature, and a second reverse similarity between the second image feature and the second text feature are obtained, and based on the second forward similarity and the second reverse similarity, a second target recognition result in a second change region is determined; based on the first target recognition result and the second target recognition result, a change detection result of the detection image compared to the background image is determined.

[0062] Specifically, the first image feature is compared with the first text feature to determine the first forward similarity between the first image feature and the first text feature, the first image feature is compared with the second text feature to determine the first reverse similarity between the first image feature and the second text feature, based on the size of the first forward similarity and the first reverse similarity, it is determined whether there is a corresponding target in the first change area, and the first target recognition result in the first change area is obtained.

[0063] Similarly, the second image feature is compared with the first text feature to determine the second forward similarity between the second image feature and the first text feature, and the second image feature is compared with the second text feature to determine the second reverse similarity between the second image feature and the second text feature. Based on the size of the second forward similarity and the second reverse similarity, it is determined whether there is a corresponding target in the second change area, and the second target recognition result in the second change area is obtained.

[0064] It can be understood that when the forward similarity is greater than the reverse similarity, the target recognition result corresponds to the existence of a target of the corresponding category, and when the forward similarity is less than the reverse similarity, the target recognition result corresponds to the absence of a target of the corresponding category.

[0065] Furthermore, based on the first target recognition result and the second target recognition result, the category corresponding to the target that has changed in the detection image compared to the background image and the change state corresponding to the target are confirmed to obtain a high-precision change detection result.

[0066] It should be noted that, based on the first target recognition result and the second target recognition result, determining the change detection result of the detection image compared with the background image includes: in response to the first target recognition result and the second target recognition result in the same area matching different description methods, based on the change state of the second target recognition result compared with the first target recognition result, determining the change detection result corresponding to the existence state of the target in the detection image compared with the background image; in response to the first target recognition results and the second target recognition results in different areas including targets of the same category and matching the same description method, based on the change state of the second target recognition result compared with the first target recognition result, determining the change detection result corresponding to the position state of the target in the detection image compared with the background image.

[0067] Specifically, when the first target recognition result and the second target recognition result in the same area match different description methods, the change detection result corresponding to the existence state of the target in the detection image compared to the background image is determined based on the change in the description method of the second target recognition result compared to the first target recognition result. When the first target recognition results and the second target recognition results in different areas include targets of the same category, and the first target recognition result and the second target recognition result match the same description method, the change detection result corresponding to the position state of the target in the detection image compared to the background image is determined based on the change state of the second target recognition result compared to the first target recognition result, thereby comprehensively detecting whether the target in the background image has disappeared, whether a new target has been added to the target in the detection image, and whether the position of the target in the background image has changed.

[0068] For ease of explanation, taking the identification of a specific target as an example, if the specific target is not identified in the first target identification result corresponding to the background image, but is identified in the second target identification result corresponding to the detection image, then the target state is determined to be target appearance; if the specific target is identified in the first target identification result corresponding to the background image, but is not identified in the second target identification result corresponding to the detection image, then the target state is determined to be target disappearance; if the target identification results corresponding to the background image and the detection image simultaneously identify the specific target, but the target area changes, then the target state is determined to be target change. Among them, target change at least includes changes in position state caused by position movement or target shape changes.

[0069] In this embodiment, feature extraction is performed on the background image to obtain multi-channel background image features corresponding to the background image, feature extraction is performed on the detection image to obtain multi-channel detection image features corresponding to the detection image, and the multi-channel background image features and multi-channel detection image features are cross-fused in a cross-channel manner, so that when obtaining feature differences, changes caused by light and shadow can be effectively filtered out, and feature differences caused by target changes are mainly retained to improve the accuracy of feature differences. Based on the feature differences, the position of the detection image that has changed compared to the background image is regressed to determine the coordinates of the change area, and the first change area corresponding to the change area coordinates is obtained from the background image using the change area coordinates, and the second change area corresponding to the change area coordinates is obtained from the detection image using the change area coordinates. The multimodal features corresponding to each change area are obtained using a multimodal pre-training model, and the first image features of the first change area and the second image features of the second change area are obtained, as well as the first text features of the forward description text and the second text features of the reverse description text. Obtain the first target recognition result in the first change area and the second target recognition result in the second change area. Based on the first target recognition result and the second target recognition result, confirm the category corresponding to the target that has changed in the detection image compared to the background image and the change state corresponding to the target to obtain a high-precision change detection result.

[0070] See also Figure 3 , Figure 3 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 30 includes a memory 301 and a processor 302 coupled to each other, wherein the memory 301 stores program data (not shown), and the processor 302 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0071] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0072] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0073] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0074] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0075] The above description is only an implementation method of the present application, and does not limit the protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A change detection method, characterized in that: The method comprises: Using the twin detection model to obtain the corresponding feature difference between the background image and the detection image, and determining the first change area in the background image and the second change area in the detection image based on the feature difference; wherein the two detection branches in the twin detection model respectively extract features from the background image and the detection image; Using an image encoder to obtain a first image feature of the first change region and a second image feature of the second change region, determining a forward description text and a reverse description text that match the first change region and the second change region, and using a text encoder to obtain a first text feature of the forward description text and a second text feature of the reverse description text; Determine a change detection result of the detection image compared to the background image based on the first image feature, the second image feature, the first text feature, and the second text feature; The method of using the twin detection model to obtain the corresponding feature difference between the background image and the detection image, and determining the first change area in the background image and the second change area in the detection image based on the feature difference, includes: using the two detection branches of the twin detection model to respectively extract the multi-channel background image features corresponding to the background image and the multi-channel detection image features corresponding to the detection image, cross-fusing the multi-channel background image features with the multi-channel detection image features according to channels to obtain the first cross-fusion feature corresponding to the background image and the second cross-fusion feature corresponding to the detection image; the odd channels in the multi-channel background image features are sequentially crossed with the even channels in the multi-channel detection image features, and the odd channels in the multi-channel detection image features are sequentially crossed with the even channels in the multi-channel background image features to obtain the first cross-fusion feature and the second cross-fusion feature; subtracting the first cross-fusion feature from the second cross-fusion feature to obtain the corresponding feature difference between the background image and the detection image; determining the change area coordinates of the detection image compared with the background image based on the feature difference, and obtaining the first change area in the background image and the second change area in the detection image using the change area coordinates.

2. The change detection method according to claim 1, characterized in that: The step of acquiring a first image feature of the first change region and a second image feature of the second change region, determining a positive description text and a negative description text matching the first change region and the second change region, and acquiring a first text feature of the positive description text and a second text feature of the negative description text includes: Using the image encoder to perform feature recognition on the first changed region to obtain the first image feature, and using the image encoder to perform feature recognition on the second changed region to obtain the second image feature; Acquire a target description text that matches the first change region and the second change region, and use the text encoder to adjust the target description text according to a forward description method and a reverse description method to obtain the forward description text and the reverse description text; The text encoder is used to perform feature recognition on the forward description text to obtain the first text feature, and the text encoder is used to perform feature recognition on the reverse description text to obtain the second text feature.

3. The change detection method according to claim 2, characterized in that: The target description text includes at least one of a target to be identified and a target to be filtered, and the target to be identified and the target to be filtered have respective categories.

4. The change detection method according to claim 1, characterized in that: The determining, based on the first image feature, the second image feature, the first text feature, and the second text feature, a change detection result of the detection image compared to the background image includes: Acquire a first forward similarity between the first image feature and the first text feature, and a first reverse similarity between the first image feature and the second text feature, and determine a first target recognition result in the first change region based on the first forward similarity and the first reverse similarity; and, Acquire a second forward similarity between the second image feature and the first text feature, and a second reverse similarity between the second image feature and the second text feature, and determine a second target recognition result in the second change area based on the second forward similarity and the second reverse similarity; Based on the first target recognition result and the second target recognition result, a change detection result of the detection image compared to the background image is determined.

5. The change detection method according to claim 4, characterized in that: The determining, based on the first target recognition result and the second target recognition result, a change detection result of the detection image compared to the background image includes: In response to the first target recognition result and the second target recognition result of the same area matching different description modes, determining a change detection result corresponding to the existence state of the target in the detection image compared to the background image based on the change state of the second target recognition result compared to the first target recognition result; In response to the first target recognition results and the second target recognition results in different areas including targets of the same category and matching the same description method, based on the change state of the second target recognition result compared with the first target recognition result, the change detection result corresponding to the position state of the target in the detection image compared with the background image is determined.

6. The change detection method according to claim 1, characterized in that: Subtracting the first cross-fusion feature from the second cross-fusion feature to obtain a feature difference corresponding to the background image and the detection image includes: The first cross-fusion feature and the second cross-fusion feature obtained from at least part of the feature layers in the two detection branches are subtracted to obtain at least one feature difference corresponding to the background image and the detection image.

7. The change detection method according to claim 6, characterized in that: The twin detection model also includes an attention network and a detection head; The determining, based on the feature difference, the coordinates of the changed area of ​​the detection image compared to the background image comprises: Using the attention network of the twin detection model, obtaining the attention features corresponding to each of the feature differences; The detection head of the twin detection model is used to perform position regression on all the attention features to obtain the coordinates of the changed area of ​​the detection image compared to the background image.

8. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images