Change detection model training and image change detection method, device and equipment

Abstract

This application discloses a change detection model training and image change detection method, device, and equipment, which can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, assisted driving, and maps. During the training phase of the change detection model, this method combines and adjusts the overall model by performing change detection on two images with content alignment, so as to enhance the single-phase change detection by utilizing the dual-phase change detection between the two images, thereby improving the accuracy of the single-phase change detection branch. At the same time, during the application phase, for the target image, only a single data processing branch, namely single-phase change detection, is required to obtain the area where the target image has changed relative to the standard image, thereby greatly reducing the amount of data processing and improving the detection efficiency of image change detection in actual detection scenarios.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of artificial intelligence technology, and provides a change detection model training and image change detection method, device and equipment. Background Art

[0002] In industrial manufacturing, the introduction of assembly lines has significantly improved production efficiency. However, complex processes inevitably lead to product defects. These defects are probabilistic and require post-production defect detection. Traditional manual inspection methods are costly and difficult. For example, because the defect area may be small, detection is difficult, resulting in missed inspections and affecting the yield of the actual production line.

[0003] To replace manual visual observation and improve the efficiency and accuracy of finished product defect detection, defect localization methods based on computer vision have become a popular research direction. For example, defect localization methods based on image change detection identify and segment defect areas in actual product images. Defect areas are image regions that have changed compared to images of defect-free products.

[0004] However, the accuracy of the image change detection model determines the accuracy of the defect location results. Therefore, how to improve the accuracy of image change detection is an issue worth considering. Summary of the Invention

[0005] The embodiments of the present application provide a change detection model training and image change detection method, device and equipment for improving the accuracy and efficiency of image change detection.

[0006] In one aspect, a change detection model training method is provided, the method comprising:

[0007] Obtaining a plurality of sample image groups, each sample image group comprising a content-registered first image, a second image, and a sample label, wherein the sample label is used to indicate: an actual image region of each of the first image and the second image that has changed relative to a standard image;

[0008] The change detection model to be trained is iteratively trained using the multiple sample image groups to obtain a corresponding target change detection model, wherein each iteration includes the following steps:

[0009] Performing intra-group image change detection on each input sample image group to obtain a first image region where a second image in each sample image group changes relative to the first image;

[0010] Performing single-phase image change detection on each input first image to obtain a second image region where each first image has changed relative to the standard image;

[0011] A model loss is determined based on the sample labels, the first image region, and the second image region corresponding to each sample image group, and parameter adjustment is performed based on the model loss.

[0012] In one aspect, a method for detecting image changes is provided, the method comprising:

[0013] A target change detection model is obtained by training based on any of the above methods;

[0014] The target change detection model is called to perform single-phase image change detection on the target image to be detected, and a target image region where the target image has changed relative to the standard image is obtained.

[0015] In one aspect, a change detection model training apparatus is provided, the apparatus comprising:

[0016] a sample acquisition unit, configured to obtain a plurality of sample image groups, each sample image group comprising a content-registered first image, a second image, and a sample label, wherein the sample label is used to indicate an actual image region in each of the first image and the second image that has changed relative to a standard image;

[0017] A change detection unit is configured to iteratively train a change detection model to be trained using the plurality of sample image groups to obtain a corresponding target change detection model; wherein the change detection unit comprises:

[0018] An intra-group detection subunit, configured to perform intra-group image change detection on each input sample image group, and obtain a first image region where a second image in each sample image group changes relative to the first image;

[0019] a single-phase detection subunit, configured to perform single-phase image change detection on each input first image, and obtain a second image region in each first image that has changed relative to the standard image;

[0020] The joint optimization subunit is used to determine the model loss based on the sample labels, the first image area and the second image area corresponding to each sample image group, and to adjust the parameters based on the model loss.

[0021] Optionally, the change detection unit further includes a feature extraction subunit;

[0022] The feature extraction subunit is configured to extract a first feature set from each first image, each first feature set including first image features corresponding to a plurality of preset image scales of the corresponding first image; and to extract a second feature set from each second image, each second feature set including second image features corresponding to the plurality of image scales of the corresponding second image;

[0023] The intra-group detection subunit is specifically configured to perform intra-group image change detection on each sample image group based on each obtained first feature set and each second feature set, and obtain the first image region corresponding to each sample image group.

[0024] Optionally, the intra-group detection subunit is specifically configured to:

[0025] For each sample image group, perform the following operations respectively:

[0026] For a sample image group, feature fusion is performed on the image features corresponding to the same image scale in the corresponding first feature set and the second feature set to obtain fused image features corresponding to each of the multiple image scales;

[0027] Based on the obtained features of each fused image, intra-group image change detection is performed on the sample image group to obtain the first image region.

[0028] Optionally, the intra-group detection subunit is specifically configured to:

[0029] For the multiple image scales, perform the following operations respectively:

[0030] For an image scale, performing a self-attention mechanism-based update process on a corresponding first image feature in a first feature set to obtain a third image feature, and performing a self-attention mechanism-based update process on a corresponding second image feature in a second feature set to obtain a fourth image feature;

[0031] Based on the third image feature, performing an attention-based fusion process on the fourth image feature to obtain a fifth image feature; and based on the fourth image feature, performing an attention-based fusion process on the third image feature to obtain a sixth image feature;

[0032] Feature fusion is performed based on the fifth image feature and the sixth image feature to obtain the fused image feature of the one image scale.

[0033] Optionally, the intra-group detection subunit is specifically configured to:

[0034] Processing the first image feature through the self-attention network of the change detection model to obtain a first attention weight of each element in the first image feature, each first attention weight being used to represent a degree of association between the corresponding element and other elements;

[0035] Based on the obtained first attention weights, the first image features are weightedly processed through the self-attention network to obtain the third image features.

[0036] Optionally, the intra-group detection subunit is specifically configured to:

[0037] processing the third image feature and the fourth image feature through the attention network of the change detection model to obtain a second attention weight for each element in the fourth image feature, each second attention weight being used to represent a degree of association between a corresponding element of the third image feature and the fourth image feature;

[0038] Based on the obtained second attention weights, the third image feature is weightedly processed by the attention network to obtain the fifth image feature.

[0039] Optionally, the intra-group detection subunit is specifically configured to:

[0040] The fused image feature is obtained based on a difference between corresponding elements in the fifth image feature and the sixth image feature.

[0041] Optionally, the intra-group detection subunit is specifically configured to:

[0042] Starting from the largest image scale, downsampling the fused image features of each image scale is performed in sequence, and the obtained downsampled image features are merged with the fused image features of the next level image scale until the smallest image scale is reached, thereby obtaining the merged image features;

[0043] Performing multiple upsampling processes on the merged image features until a target image feature having the same image scale as the original image is obtained;

[0044] Based on the target image features, the first image region is determined.

[0045] Optionally, the single-phase detection subunit is specifically used to:

[0046] For each first image, perform the following steps respectively:

[0047] For a first image, performing scale conversion processing on first image features of each image scale to obtain a plurality of seventh image features of the same image scale;

[0048] performing feature fusion on the obtained plurality of seventh image features to obtain an eighth image feature;

[0049] performing upsampling processing based on the eighth image feature to obtain a ninth image feature having the same image scale as that of the first image;

[0050] Based on the ninth image feature, the probability that each pixel in the first image belongs to a change point is determined, and the second image region is obtained based on the obtained probabilities.

[0051] Optionally, the single-phase detection subunit is specifically used to:

[0052] performing at least one upsampling process on the first image features of each image scale to obtain a plurality of seventh image features of the same image scale;

[0053] Wherein, starting from the smallest image scale, the number of upsampling processes of each image scale decreases in sequence.

[0054] Optionally, the joint optimization subunit is specifically used to:

[0055] Determining, based on each first image region and the corresponding sample label, a single-phase change detection loss corresponding to the change detection model;

[0056] determining a single-phase change detection loss corresponding to the change detection model based on each second image region and the corresponding sample label;

[0057] The model loss is determined based on the within-group change detection loss, the single-phase change detection loss, and respective weights.

[0058] In one aspect, an image change detection device is provided, the device comprising:

[0059] An image input unit, configured to obtain a target image to be detected, and input the target image into a target change detection model trained based on any of the above methods;

[0060] The image change detection unit is used to perform single-phase image change detection on the target image to be detected by using the target change detection model, and obtain the target image area where the target image has changed relative to the standard image.

[0061] Optionally, the target image is an image of a product to be inspected for defects, and the standard image is a non-defective image of the same product as the target image;

[0062] The image change detection unit is further configured to determine the target image area as a defect area.

[0063] In one aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0064] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0065] In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above methods.

[0066] In an embodiment of the present application, during the training phase of the change detection model, by constructing a sample image group of a first image and a second image with content registration, single-phase change detection is performed on the first image on one data processing branch, and dual-phase change detection is performed within the group on the first image and the second image on another data processing branch, so that the parameters of the change detection model can be adjusted based on the processing results of the two data processing branches in combination with the sample labels. Thus, by performing change detection on the two images with content registration to combine and adjust the overall model, the dual-phase change detection between the two images is utilized to enhance the single-phase change detection, thereby improving the accuracy of the single-phase change detection branch. At the same time, in the application phase, for the target image, the embodiment of the present application only needs to perform single-phase change detection on the single data processing branch to obtain the area where the target image has changed relative to the standard image, thereby greatly reducing the amount of data processing and improving the detection efficiency of image change detection in actual detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0068] Figure 1 A schematic diagram of image change detection provided by an embodiment of the present application;

[0069] Figure 2 Schematic diagram of application scenarios provided by embodiments of the present application;

[0070] Figure 3A schematic diagram of the structure of a change detection model provided in an embodiment of the present application;

[0071] Figure 4 A flow chart of a change detection model training method provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of constructing a sample image group provided in an embodiment of the present application;

[0073] Figure 6 A schematic diagram of the structure of the change fusion module provided in an embodiment of the present application;

[0074] Figure 7 A schematic diagram of the processing flow of the self-interaction submodule provided in an embodiment of the present application;

[0075] Figure 8 A schematic diagram of the processing flow of the cross-interaction sub-module provided in an embodiment of the present application;

[0076] Figure 9 A schematic diagram of the processing flow of a change sensing head provided in an embodiment of the present application;

[0077] Figure 10 A schematic diagram of the architecture of a single-phase image change detection network provided in an embodiment of the present application;

[0078] Figure 11 A flowchart of an image change detection method provided in an embodiment of the present application;

[0079] Figure 12 A schematic diagram of a defect detection process for a camera bracket accessory provided in an embodiment of the present application;

[0080] Figure 13 A comparison diagram of a normal accessory and a defective accessory provided in an embodiment of the present application;

[0081] Figure 14 A schematic diagram of a structure of a change detection model training device using an embodiment of the present application;

[0082] Figure 15 A schematic diagram of a structure of an image change detection method according to an embodiment of the present application is provided;

[0083] Figure 16 A schematic diagram of the structure of an electronic device to which an embodiment of the present application is applied;

[0084] Figure 17 The figure is a schematic diagram of the structure of another electronic device to which the embodiments of the present application are applied. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0086] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0087] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0088] Image change detection: The image change detection task is to detect and segment the image changes between two images, such as the semantic changes between two registered images taken at different times. Figure 1 As shown in the figure, it is a schematic diagram of image change detection. Figure 1 As can be seen from Figures a and b, after comparing them through human visual perception, we can perceive that Figure b has an additional circle and rectangle compared to Figure a, that is, the content in Figure c. Therefore, image change detection simulates human visual perception to achieve the effect of being able to identify changes in the image.

[0089] Defect detection: refers to the detection of product defects in industrial finished products and the location of product defects. It is an application of image change detection in real-world scenarios. Its essence lies in detecting changes between finished product images and images of defect-free products.

[0090] Attention mechanism: A method that uses high-level information to measure intermediate network features, allowing the network to focus on certain information in the image that aids judgment and ignores irrelevant information. The essence of the attention mechanism is derived from the human visual attention mechanism. When people perceive something, they generally do not observe the entire scene from beginning to end. Instead, they often observe and pay attention to specific parts based on their needs. When people find that a certain part of a scene frequently contains something they want to observe, they learn to focus on that part in similar scenes in the future. Therefore, the attention mechanism is essentially a means of filtering high-value information from a large amount of information. Within a large amount of information, different information has different importance to the result, and this importance can be reflected by assigning different weights. In other words, the attention mechanism can be understood as a rule for assigning weights when synthesizing multiple sources.

[0091] Self-attention mechanism: The self-attention mechanism is a variant of the attention mechanism. Its essential principle is the same as the attention mechanism. The difference is that the attention mechanism focuses on the correlation between different images, while the self-attention mechanism focuses more on the intrinsic connection within an image.

[0092] The technical solutions of the embodiments of this application involve artificial intelligence and machine learning technologies. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0093] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0094] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, which uses cameras and computers to replace the human eye in identifying and detecting objects. This involves further processing the images, transforming them into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0095] Machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0096] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0097] The solution provided in the embodiment of the present application relates to computer vision technology and machine learning technology of artificial intelligence. The target change detection model provided in the embodiment of the present application is applied to image change detection (relative to a standard image or between two images). Among them, the training and use methods of the change detection model can be divided into two parts, including a training part and an application part; wherein the training part involves the technical field of machine learning. In the training part, the change detection model is trained by the machine learning technology, and the model parameters are continuously adjusted by the optimization algorithm until the model converges, including the sample image group input to the change detection model to be trained, respectively, by performing a two-phase change detection within the group on the first image and the second image in each image group, obtaining the corresponding first image area, and performing a single-phase change detection on each first image therein, obtaining the corresponding second image area, thereby adjusting the model parameters according to the first image area and the second image area of each sample image group and the sample label; the application part is used to perform a one-way change detection on the input target image by using the one-way change detection branch in the target change detection model trained in the training part to obtain the area where the target image has changed relative to the standard image, etc. In addition, it should be noted that the artificial neural network model in the embodiment of the present application can be trained online or offline, which is not specifically limited here. In this article, offline training is used as an example for illustration.

[0098] The following is a brief introduction to the design concept of the embodiment of this application:

[0099] The defect localization method based on image change detection can replace the manual visual observation method and improve the efficiency of defect localization. For example, the current related technologies usually use target detection or semantic segmentation for positioning. Since most defects are irregular, semantic segmentation methods are mostly used. For example, the change detection method using the Hierarchical Paired Channel Fusion Network (HPCFNet) extracts multi-layer image semantic features and uses a multi-layer paired channel fusion (PCF) module to perform channel-level feature fusion. At the same time, a multi-layer multi-part feature learning (MPFL) module is used to integrate features from the whole to the local to adapt to the scale and location diversity of the scene change area, and finally outputs a change segmentation map.

[0100] However, the design of the PCF module and the MPFL module of this method is too complicated, and a large number of parallel dilated convolutions and cross-feature stacking operations are used, which makes the method consume a lot of computing resources, has low computational efficiency, and cannot adapt to the needs of actual detection scenarios.

[0101] In view of this, the embodiments of the present application provide a change detection model training and image change detection method, device and equipment. Among them, when the model training is performed based on the change detection model training method, by constructing a sample image group of the first image and the second image with content alignment, single-phase change detection is performed on the first image on one data processing branch, and dual-phase change detection is performed within the group on the first image and the second image on another data processing branch, so that the parameters of the change detection model can be adjusted based on the processing results of the two data processing branches combined with the sample labels. Thus, the overall model is adjusted by performing change detection on the two images with content alignment, so as to enhance the single-phase change detection by utilizing the dual-phase change detection between the two images, thereby improving the accuracy of the single-phase change detection branch. At the same time, when applying the image change detection method to perform change detection, for the target image, only the single-phase change detection, which is a single data processing branch, is required to be performed to obtain the area where the target image has changed relative to the standard image, thereby greatly reducing the amount of data processing and improving the detection efficiency of image change detection in actual detection scenarios.

[0102] In addition, in an embodiment of the present application, when performing bidirectional change detection, the first image and the second image are respectively subjected to self-interaction fusion based on the self-attention mechanism, and the first image and the second image are subjected to cross-interaction fusion based on the attention mechanism, thereby increasing the sample diversity and the interaction between samples. Therefore, the subsequent change perception effect is better, and the detection accuracy of the trained target change detection model is higher, and it is also more robust to existing inherent changes.

[0103] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only used to illustrate the embodiments of the present application and are not limiting. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0104] The solution provided in the embodiments of the present application can be applied to most image change detection scenarios, such as product defect location scenarios, street scene change detection (SSCD) scenarios, remote sensing image change detection scenarios, etc., which are not listed one by one here.

[0105] like Figure 2 , which is a schematic diagram of an application scenario provided by an embodiment of the present application, in which an image acquisition device 201, an image change detection device 202 and a terminal device 203 may be included.

[0106] The image acquisition device 201 is an electronic device with an image acquisition function, such as a camera or a camera. The image change detection device 202 is an electronic device that implements image change detection based on the method of the embodiment of the present application. It can be any electronic device with sufficient computing power, such as a mobile phone, a tablet computer (PAD), a laptop computer, a desktop computer, a smart home appliance, a smart car-mounted device, an aircraft, and a smart wearable device. Alternatively, it can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), and big data and artificial intelligence platforms and other basic cloud computing services, but is not limited to this. The terminal device 203 is used to realize the display of image change detection results and reminder functions, etc.

[0107] In actual applications, the image acquisition device 201 can send the captured images to the image change detection device 202, and then the image change detection device 202 can construct a sample image group based on these images, and input the constructed sample image group into the change detection model to be trained for training. During training, the change detection model is used to perform single-phase change detection on the first image of each sample image group, and to perform two-phase change detection within the group for the first image and the second image in each sample image group, so that the parameters of the change detection model can be adjusted based on the single-phase change detection results, the two-phase change detection results and the sample labels to obtain a target change detection model. In addition, the image change detection device 202 can also use the target change detection model to perform image change detection on the images subsequently captured by the image acquisition device 201, and send the detection results to the terminal device 203. The terminal device 203 can display the image change detection results, or send a reminder message to the terminal device 203 when the image change detection results are abnormal.

[0108] In one possible implementation, the method of the embodiment of the present application can be applied to a product defect location scenario. Specifically, the image acquisition device 201 can be set on a product production line, for example, it can be set after each process is completed, and is used to capture the product image obtained by the process. Then, the image change detection device 202 can use the above process training to obtain a target change detection model based on the captured product image, and use the target change detection model to perform change detection on the product image captured by the image acquisition device 201 relative to the defect-free product image, so as to determine whether the corresponding product has defects and locate the defective area, and transmit the corresponding detection results to the terminal device 203, thereby replacing the manual visual inspection method, saving manpower, and improving detection efficiency and accuracy.

[0109] In a possible implementation, the method of the embodiment of the present application can also be applied to the street view change detection scenario. Specifically, the image acquisition device 201 can be set on each street to be detected to capture street view images. Then, the image change detection device 202 can be based on the captured street view images and use the above process to train a target change detection model, and use the target change detection model to perform change detection on the street view image captured by the image acquisition device 201 relative to the set street view image (such as the street view image of the same location N days ago) to determine the changes in the street, and transmit the corresponding detection results to the terminal device 203 to assist relevant units in street management and rectification, avoid manual inspections, save manpower, and improve detection efficiency and accuracy.

[0110] In the embodiment of the present application, the aforementioned devices may be directly or indirectly connected to each other through one or more networks 204. The network 204 may be a wired network or a wireless network, for example, a mobile cellular network or a Wireless Fidelity (WIFI) network, or other possible networks, which are not limited in the embodiment of the present invention.

[0111] It should be noted that Figure 2 The examples shown are for illustration only. In practice, the number of devices is unlimited and is not specifically limited in the embodiments of this application. In addition, in actual scenarios, the model training phase and the model application phase can be performed by different devices. For example, the model training phase can be performed by a server with sufficient computing power. After the model training is completed, the relevant content of the target change detection model obtained through training can be stored on the device performing the change detection, so that the efficiency of the model training phase can be improved, and computing resources can be saved in the application phase.

[0112] The following describes the method provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this respect.

[0113] See also Figure 3 The diagram is a schematic diagram of the structure of the change detection model provided by the embodiment of the present application. The model includes a feature extraction network, a single-phase change detection network, an intra-group change detection network and a joint optimization network. Figure 3 The model structure shown introduces the change detection model training method provided in the embodiment of this application, see Figure 4 FIG. 1 is a flow chart of a change detection model training method provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0114] Step 401: Obtain multiple sample image groups, each sample image group includes a content-registered first image, a second image, and a sample label, where the sample label is used to indicate actual image regions of the first image and the second image that have changed relative to a standard image.

[0115] In the embodiments of the present application, different image samples can be collected for specific task objectives to better suit the task objective scenario. For example, in a product defect location scenario, product images can be collected for a specific product and used to train a model, so that the trained model can achieve higher detection accuracy for that product.

[0116] The essence of the change detection task is to detect semantic changes between two content-aligned images. Content alignment refers to the consistency of the main content in the two images. Specifically, when locating product defects, content alignment can refer to two images with the same or similar product content, such as product images of the same product. Or, when detecting street scene changes, content alignment can refer to two images with the same or similar content, such as street scene images of the same intersection.

[0117] In one possible implementation, image registration technology can be used to construct a sample image group. Image registration is the process of matching different images of the same area acquired under different imaging conditions by matching specific point pairs. Therefore, when two images with similar content but captured under different conditions (e.g., different angles) are acquired, they can be processed using image registration technology and combined as a sample image group.

[0118] In one possible implementation, considering that image registration consumes more resources, and that a sequence of images of the same product can be collected on a product assembly line, in order to reduce the workload and resource consumption of image registration, the embodiment of the present application uses a sequence of images taken at the same point at different times to construct a sample image group.

[0119] For example, the image sequence captured at the same point is represented as (x 0 ,x 1 ,……,x n-1 ), then we can construct a sample image group The first image is denoted as It can be (x 0 ,x 1 ,……,x n-1 ), the second image is recorded as Its A permutation and combination of Any image other than It is obtained by superimposing and combining with any other image.

[0120] See also Figure 5 The figure is a schematic diagram of constructing a sample image group provided by an embodiment of the present application. Here, the shift combination construction method is used as an example to illustrate. For each defective product image in the captured image sequence, for example, the first product image has a notch and a crack defect at the corner, and the second product image has a notch on the left. Then The next image in the image sequence, the first product image corresponds to In fact, it is the second product image. Of course, in the actual operation process, other possible combination construction methods can also be used, such as random selection and the like.

[0121] The training method provided in the embodiment of the present application adopts a supervised training method, and each image needs to be labeled. Specifically, in order to reduce the workload of manual labeling, the embodiment of the present application adopts a method of comparison with a standard image for labeling to mark the differences from the standard image. The standard image refers to an image that can be used as a reference image in a change detection scenario. For example, in a defect location scenario, the standard image can be a defect-free image of the same product, and then each image is subjected to an exclusive OR (XOR) operation with the standard image, that is, the pixel values at corresponding positions in the two images are subjected to an exclusive OR operation. When the two are the same, the exclusive OR result is 1, and when the two are different, the exclusive OR result is 0. In this way, the difference between the image and the standard image can be quickly marked, thereby improving the efficiency of the training stage.

[0122] In the embodiment of the present application, each sample image group can be composed in the following two ways:

[0123] (1) The first method of composition is to adopt and The actual image regions that have changed relative to the standard image are used as sample labels. For example, when a sample image group is Figure 5 shown When the first and second product images are in the same column, their corresponding change graphs can be combined to form the sample label, that is, Figure 5 shown The first and second change maps in a column combine to form the sample label.

[0124] (2) The second composition method is to adopt a labeling method corresponding to the structure of the change detection model provided in the embodiment of the present application. The change detection model includes two branches, and the outputs of the two branches are the first image area where the second image in the same group changes relative to the first image, and the second image area where the first image changes relative to the standard image. Therefore, a corresponding labeling method can be adopted, that is, the sample label of a sample image group can include an actual change map of the labeled first image relative to the standard image, which indicates the actual image area where the first image changes relative to the standard image, and an actual change map between the first image and the second image, which indicates the actual image area where the first image changes relative to the second image. The actual change map between the first image and the second image can also be obtained by performing an XOR operation on the first image and the second image, or can also be obtained by performing an XOR operation on the change map of the first image and the second image relative to the standard image.

[0125] See also Figure 5 As shown, it is specifically shown in the second composition mode, and each row can represent a sample image group, which includes the first image Second image Changes of the first image relative to the standard image The XOR operation result of the change map of the first image and the second image relative to the standard image in, Represents the second image Change graph relative to the standard image.

[0126] It should be noted that, in the above-mentioned images, the first image or the second image may also be a standard image, such as a normal image of a product without defects.

[0127] In the embodiment of the present application, after constructing the multiple sample image groups, the multiple sample image groups can be used to perform multiple iterations of training on the change detection model to be trained to obtain the target change detection model. Since each iteration process is similar, the embodiment of the present application mainly takes one iteration as an example. Figure 4 , one iteration includes the following steps:

[0128] Step 402: Perform intra-group image change detection on each input sample image group to obtain a first image region where the second image in each sample image group changes relative to the first image.

[0129] See also Figure 3As shown, the change detection model provided by the embodiment of the present application includes two branches. One branch is implemented by a single-phase change detection network branch, which is used to predict the change of the first image relative to the standard image. Since its input is only the first image, it belongs to the change map segmentation of a single image, and therefore can also be called a semantic segmentation network branch; the other branch is an intra-group change detection network branch, which is used to predict the change between two images in the same sample image group, and belongs to a two-phase change detection.

[0130] During each iteration, the input for this iteration can be selected from the constructed multiple sample image groups. For example, random selection can be used, or the multiple sample image groups can be pre-divided into batches, with each batch of sample image groups being input. Since the model processing is similar for each sample image group, the following description will focus on the processing of a single sample image group.

[0131] See also Figure 3 As shown, when a sample image group is input After that, the intra-group change detection network performs intra-group image change detection on it and obtains the first image area where the second image in the group changes relative to the first image, which is expressed as

[0132] In a possible implementation, the first image region may be represented in the form of a change graph, for example, see Figure 5 The change diagram shown.

[0133] Step 403: Perform single-phase image change detection on each input first image to obtain a second image region where each first image has changed relative to the standard image.

[0134] See also Figure 3 As shown, when a sample image group is input After that, the single-phase change detection network performs change detection on the first image in the group, which can also be understood as semantic segmentation of the first image, segmenting the area in the first image that may be different from the standard image, and obtaining the second image area where the first image changes relative to the standard image, which is expressed as

[0135] In a possible implementation, the second image region can also be represented in the form of a change graph, for example, see Figure 5 The change diagram shown.

[0136] Step 404: Determine the model loss based on the sample labels, the first image region, and the second image region corresponding to each sample image group.

[0137] In the embodiment of the present application, the model loss can include two parts. One part is the prediction loss of the single-phase change detection network branch included in the change detection model, which is called the single-phase change detection loss. Since the embodiment of the present application mainly uses the single-phase change detection network for image change detection in the application stage, the single-phase change detection network branch can be considered as the main branch of the model. The other part is the prediction loss of the intra-group change detection network branch included in the change detection model, which is called the intra-group change detection loss. Compared with the single-phase change detection network branch, the intra-group change detection network branch serves as an auxiliary branch. Furthermore, the model loss L can be expressed as:

[0138] L=L seg +λL change

[0139] Among them, L seg is the single-phase change detection loss, L change is the intra-group change detection loss, and λ is a hyperparameter for balancing the two losses. Its value can be set according to the importance of the two losses, the order of magnitude of the outputs of the two branches, or an empirical value.

[0140] The intra-group change detection loss may be determined based on each first image region and the corresponding sample label.

[0141] When the sample label includes the first image in the group and the second image Actual changes relative to the standard image and When and After performing the XOR operation, the actual change graph between the first image and the second image is obtained. Then compare it with the first image region predicted by the intra-group change detection network The comparison is done to determine the within-group change detection loss.

[0142] When the sample label includes the first image in the group Actual changes relative to the standard image and a graph of the actual changes between the first and second images When , the first image region predicted by the intra-group change detection network is and actual change chart The comparison is done to determine the within-group change detection loss.

[0143] In one possible implementation, the intra-group change detection loss L change The cross entropy loss function can be used to calculate the pixel-by-pixel cross entropy loss, which is expressed as follows:

[0144]

[0145] in, The output of the intra-group change detection network branch for H and W represent the height (height) and width (width) of the first image or the second image, respectively.

[0146] In the embodiment of the present application, the single-phase change detection loss can be determined based on each second image region and the corresponding sample label.

[0147] In one possible implementation, the single-phase change detection loss can be calculated using a cross entropy loss function, a DICE loss function, or other possible loss functions. Taking the combination of the cross entropy loss function and the DICE loss function as an example, the single-phase change detection loss can be expressed as follows:

[0148] L seg =L dice +L lce

[0149] Among them, L dice is the DICE loss, L lce is the cross entropy loss.

[0150]

[0151]

[0152] in, is the output of the single-phase change detection network branch, y i is the label corresponding to the first image in the sample label.

[0153] Step 405: Determine whether the change detection model meets the convergence condition.

[0154] In an embodiment of the present application, the model loss is obtained after the process described above, and whether the change detection model meets the convergence condition can be determined based on the model loss.

[0155] Specifically, the convergence condition may include any one of the following conditions:

[0156] (1) The model loss is not greater than the preset loss value threshold.

[0157] (2) The number of iterations reaches the preset threshold.

[0158] Step 406: If the result of step 305 is no, then adjust the parameters of the change detection model based on the model loss.

[0159] It should be noted that the change detection model mentioned above refers to the change detection model participating in this iteration. When the change detection model does not meet the convergence conditions, the parameters of the change detection model are adjusted according to the model loss, and the adjusted change detection model is used to enter the next iteration process, that is, jump to step 402.

[0160] In a possible implementation, an optimization algorithm such as a gradient descent method may be used to adjust the parameters.

[0161] If the change detection model does not meet the convergence condition, the training process ends, the model parameters of the current change detection model are stored, and the target change detection model is obtained.

[0162] In the examples of this application, see Figure 3 As shown, the change detection model also includes a feature extraction network for extracting features from the first image and the second image. The extracted image features can be used in subsequent single-phase change detection networks and intra-group change detection networks.

[0163] In one possible implementation, the feature extraction network can adopt a twin network structure, see Figure 3 As shown, the feature extraction network can include two identical Backbones, one of which is used to extract features from the first image of each input sample image group to obtain a first feature set for each first image, each first feature set including: first image features corresponding to the corresponding first image at multiple preset image scales; and the other Backbone is used to extract features from the second image of each input sample image group to obtain a second feature set for each second image, each second feature set including: second image features corresponding to the corresponding second image at the multiple image scales. Therefore, for the single-phase change detection network branch, it can be composed of a combination of Backbone and SegHead, and for the intra-group change detection network branch, it can be composed of a combination of Backbone, ChangeFusion, and CPHead.

[0164] Among them, Backbone can adopt structures such as Convolutional Neural Networks (CNN), Visual Geometry Group (VGG), Inception network or Residual (ResNet) network, and the embodiments of the present application do not limit this.

[0165] Given a set of sample images For example, the dimension of each image is R3×H×W , where 3 represents the number of image channels, H and W represent the size of the image, and the following initial image features are extracted using a twin-structured CNN network:

[0166] First image:

[0167] Second image:

[0168] in, Corresponding to multiple image scales, here we take 5 image scales as an example, and we can extract the initial image features of 5 image scales, for example, the 5 image scales are 1, 1 / 4, 1 / 8, 1 / 16 and 1 / 32 respectively, 1 / 32 means that the obtained initial image features are 1 / 32 of the original image size, It should be noted that the first image feature and the second image feature are used to distinguish the initial image features of the first image and the second image, but do not have any distinction limitations at the feature level.

[0169] In actual application, considering that the information content of some extracted image scale features is weak, the computational complexity is large, or the effect on the change detection task mentioned in the embodiment of this application is weak, then these image scale features can be filtered out and the remaining features can be used to perform subsequent processes. and The semantic information of is weak and the computation is large, so the remaining four scale features are used and Used for subsequent algorithm processing.

[0170] See also Figure 3 As shown, the intra-group image change detection network provided by the embodiment of the present application includes a change fusion module and a change perception head (CPhead). Since the processing process for each sample image group is similar, a sample image group is specifically used here. For example, the first feature set of the sample image group obtained by combining the above extraction and the second feature set Perform intra-group image change detection on the sample image group to obtain a corresponding first image region.

[0171] (1) Change Fusion Module

[0172] In the embodiment of the present application, the change fusion module is an important module of the change detection model, which is used to explore global context information and provide interaction between two spatiotemporal features. The input of this module is the first feature set extracted by the feature extraction network. and the second feature set Feature pairs at a single image scale in It is used to fuse the image features corresponding to the same image scale in the first feature set and the second feature set, so as to obtain the fused image features corresponding to multiple image scales.

[0173] See also Figure 6 FIG. 1 is a schematic diagram of a structure of a change fusion module provided in an embodiment of the present application. The module includes three parts: a self-interaction submodule, a cross-interaction submodule, and a fusion submodule.

[0174] (1) Self-interaction submodule

[0175] The self-interaction submodule acts on a single image feature of the same image to perceive the correlation between the pixels of the image feature itself. That is, for the image feature of a certain image scale output by the feature extraction network, the corresponding spatiotemporal features are obtained after self-interaction.

[0176] For each first image feature in the first feature set, the self-interaction submodule obtains the third image feature corresponding to each first image feature by performing an update process based on the self-attention mechanism. in Then the corresponding third image feature can be obtained through the self-interaction submodule enter in Then the corresponding third image feature can be obtained through the self-interaction submodule And so on.

[0177] For each second image feature in the second feature set, the self-interaction submodule obtains the fourth image feature corresponding to each second image feature by performing an update process based on the self-attention mechanism. in Then the corresponding third image feature can be obtained through the self-interaction submodule enter in Then the corresponding third image feature can be obtained through the self-interaction submodule And so on.

[0178] Since the processing process for each first image feature or second image feature is similar, the first image feature is used here. (i ranges from 2 to 5) as an example to introduce the processing flow of the self-interaction submodule. The self-interaction submodule can be implemented using a self-attention network, which can then be used to analyze the first image feature. Processing is performed to obtain the first attention weight of each element in the first image feature, where each first attention weight is used to characterize the degree of association between the corresponding element and other elements. Then, based on the obtained first attention weights, the first image feature is weightedly processed through the self-attention network to obtain the third image feature.

[0179] See also Figure 7 , which is a schematic diagram of the processing flow of the self-interaction submodule provided in an embodiment of the present application.

[0180] S71: Initialization assignment.

[0181] In the embodiment of the present application, the first image feature Assign the weight vector matrix of the self-attention network respectively. One possible way is to use the first image feature Initialize the query matrix separately key matrix Sum value matrix That is, initialize it as follows:

[0182]

[0183] In a possible implementation, in order to improve the expressiveness of features, the first image feature of the input may be Perform position encoding.

[0184] Specifically, it can be Plus a fixed-dimensional learning parameter ρ(W p ), ρ is the position encoder, W p is the corresponding parameter, so based on the position encoding To initialize the above matrices, namely:

[0185]

[0186] Among them, W p In order to be able to follow the training process, it can represent the current image scale, or the position or size of each element of the current image feature in the original image. The learning parameter ρ(W p ) corresponds to its image scale and is consistent with the size of the image feature. For example, For example, when the corresponding image scale is 1 / 4 of the original image, the corresponding learning parameter ρ(W p ) is also 1 / 4 of the original image.

[0187] In one embodiment, the With W p The corresponding pixels are superimposed (such as adding or multiplying) to encode the position, or Each element in W p The weighted sum of the values in is obtained. Of course, other possible methods can also be used, and the embodiments of the present application do not limit this.

[0188] S72: Convolution processing.

[0189] In the embodiment of the present application, the query matrix after initialization Bond Matrix Sum Matrix After processing with 1*1 convolution parameters, the convolved query matrix is obtained. Bond Matrix Sum Matrix

[0190]

[0191]

[0192]

[0193] in, and is the convolution parameter of the i-th image scale.

[0194] S73: Calculate attention map.

[0195] In the embodiment of the present application, the attention map is based on the query matrix Bond Matrix The similarity between them is obtained, and the query matrix Bond Matrix They are all obtained by assigning values to the first image features, so the attention map here is essentially a representation of the intrinsic correlation of the first image features themselves.

[0196] In one possible implementation, affinity can be used to measure similarity, so the calculation method of the attention map can be expressed as follows:

[0197]

[0198] Where d represents the query matrix Bond Matrix The dimension size after multiplication, the softmax function acts on the channel dimension. By value matrix The same number of elements as the value matrix The first attention weight of each element of is composed.

[0199] S74: Weighting processing.

[0200] Based on the value matrix and attention map Perform weighted processing to obtain the final third image feature

[0201] In a possible implementation, an aggregation method may be used to perform weighted fusion processing, which is expressed as follows:

[0202]

[0203] Similarly, the fourth image feature corresponding to the second image can be obtained by the above method

[0204] (2) Cross-interaction submodules

[0205] The self-interaction submodule is used to improve the two spatiotemporal features and For example, the fourth image feature of the second image is integrated into the third image feature corresponding to the first image, or the third image feature of the second image is integrated into the fourth image feature of the first image. The input is the feature pair of the same image scale in the sample image group. Right now and

[0206] For the first image, the cross-interaction submodule can perform attention-based fusion processing on the fourth image feature based on the third image feature to obtain the fifth image feature. Then the fifth image feature on the image scale can be obtained through the cross-interaction sub-module enter Then the fifth image feature on the image scale can be obtained through the cross-interaction sub-module And so on.

[0207] For the second image, the cross-interaction submodule performs attention-based fusion processing on the third image feature based on the fourth image feature to obtain the sixth image feature. Then the sixth image feature on the image scale can be obtained through the cross-interaction sub-module enter Then the sixth image feature on the image scale can be obtained through the cross-interaction sub-module And so on.

[0208] Since the cross-interaction processing process corresponding to the first image or the second image is similar, here we use the third image feature Incorporating the fourth image feature Taking the example of the cross-interaction submodule, the processing flow is introduced. The cross-interaction submodule can be implemented using an attention network. The attention network can then process the third and fourth image features to obtain second attention weights for each element in the fourth image feature. Each second attention weight is used to represent the degree of association between the corresponding element of the third image feature and the fourth image feature. Based on the obtained second attention weights, the attention network performs weighted processing on the third image feature to obtain the fifth image feature.

[0209] See also Figure 8 As shown, it is a schematic diagram of the processing flow of the cross-interaction sub-module provided in an embodiment of the present application.

[0210] S81: Initialization assignment.

[0211] In the embodiment of the present application, it is similar to the self-interaction submodule, but the difference lies in the initialization input. Incorporating the fourth image feature , it is initialized as follows:

[0212]

[0213] That is Initialize the query matrix by Initialize the bond matrix Sum Matrix

[0214] Similarly, similar to the self-interaction submodule, in order to improve the expressiveness of features, the third image feature can also be and incorporate the fourth image feature After position encoding, the above matrix is initialized, which will not be described in detail here.

[0215] S82: Convolution processing.

[0216] In the embodiment of the present application, the query matrix after initialization Bond Matrix Sum Matrix After processing with 1*1 convolution parameters, the convolved query matrix is obtained. Bond Matrix Sum Matrix Right now:

[0217]

[0218]

[0219]

[0220] in, and is the convolution parameter of the i-th image scale.

[0221] S83: Calculate attention map.

[0222] In the embodiment of the present application, the attention map is based on the query matrix and bond matrix The similarity between them is obtained, and the query matrix Bond Matrix They are obtained by assigning values to the third image feature and the fourth image feature respectively, so the attention map here is essentially a representation of the correlation between the third image feature and the fourth image feature.

[0223] In one possible implementation, affinity can be used to measure similarity, so the calculation method of the attention map can be expressed as follows:

[0224]

[0225] in, By value matrix The same number of elements as the value matrix The second attention weight of each element is composed of

[0226] S84: Weighting processing.

[0227] Based on the value matrix and attention map Perform weighted processing to obtain the final fifth image feature

[0228] In a possible implementation, an aggregation method may be used to perform weighted fusion processing, which is expressed as follows:

[0229]

[0230] Similarly, the sixth image feature corresponding to the second image can be obtained by the above method:

[0231] (2) Fusion submodule

[0232] See also Figure 6 As shown, after the above self-interaction submodule and cross-interaction submodule, a pair of cross-interaction features are obtained and The information of the original image is integrated with the information of another image in the same image group, and the feature expression is more accurate. Therefore, it is crucial to adopt an effective fusion strategy for fusion. The fusion strategy must meet three requirements: first, it can intuitively reflect the semantic change, second, it must be symmetrical, and third, the computational complexity must be as low as possible. After comprehensive consideration, it is possible to consider subtracting the absolute elements of two cross-interaction features to measure their semantic changes, that is, based on the fifth image feature and the sixth image feature The difference between the corresponding elements in is used to obtain the final fused image features, thus meeting all the above requirements.

[0233] In one possible implementation, see Figure 6 As shown, the fused image features can be obtained using the following formula

[0234]

[0235] Furthermore, through the above process, the fusion image features corresponding to each image scale of each sample image group can be obtained. As in the above example, by and Using the above processing process, we can get multi-layer fusion features

[0236] (2) Change-sensing head

[0237] The change-sensing head is used to perform intra-group image change detection on each sample image group based on the acquired fused image features, obtaining intra-group image change detection results. From a machine learning perspective, the change detection task can be considered as a second-class semantic segmentation task. Therefore, semantic segmentation methods can be used to design a change-sensing head. Considering both effectiveness and efficiency, this embodiment of the application proposes a change-sensing head with a simple feature pyramid structure.

[0238] See also Figure 9 FIG2 is a schematic diagram of the processing flow of the change perception head, wherein the change perception head includes a bottom-up processing process and a top-down processing process.

[0239] (1) Bottom-up processing

[0240] In this processing process, starting from the largest image scale, that is, the shallowest features, the fused image features of each image scale are downsampled in turn, and the obtained downsampled image features are merged with the fused image features of the next level image scale until the smallest image scale is reached to obtain the merged image features.

[0241] Taking 5-layer features as an example, from the shallowest feature Initially, we use convolution with a step size of 2 and a kernel size of 3x3 to perform downsampling, and obtain the same Image features Then with Merge, and then continue downsampling and merging operations. Following this operation, we finally get the merged image features

[0242] (2) Top-down processing

[0243] In this processing, in order to obtain a high-resolution feature map, the opposite operation to the bottom-up process is adopted, and the merged image features are upsampled multiple times until the target image features with the same image scale as the original image are obtained.

[0244] Taking the 5-layer feature as an example, from the deepest feature At the beginning, each upsampling process includes 1x1 convolution processing, linear activation (Rectified Linear Units, Relu) function processing, batch normalization (batch Norm, BN) processing and 2x upsampling operation to Perform upsampling update Repeat this operation and finally obtain the target image features at 1 / 4 scale.

[0245] For the target image features, two 1x1 convolutions and 4x upsampling operations are used to obtain the final pixel-level output. The final output is the same size as the original image and indicates which areas belong to the changed areas. For example, each element in the output indicates the probability that the pixel at that position in the first image and the second image has changed. This can be used to determine which areas belong to the first image area between the first and second images, that is, the changed image areas.

[0246] In the embodiment of the present application, the single-phase image change detection network can be implemented using a network structure based on semantic segmentation, such as U-Net, PSPNet, and Deeplabv3. Figure 10 As shown, it is a schematic diagram of the architecture of a single-phase image change detection network provided by an embodiment of the present application. Unlike the U-Net style network, it uses a symmetric decoder to mirror the bottom-up path, but contrary to outputting the final segmentation, the embodiment of the present application uses an asymmetric decoder to merge information from all levels of contrast features. Its top-down path has only one block per stage and uses a shared channel dimension. Based on the characteristics of its asymmetric decoder, it is suitable for encoding sufficiently rich multi-scale semantic information to predict accurate pixel-level segmentation maps.

[0247] In the embodiment of the present application, for a first image, scale conversion processing is performed on the first image features of each image scale to obtain a plurality of seventh image features with the same image scale.

[0248] Among them, the scale conversion processing can be performed in an upsampling manner. According to the different image scales, the number of upsampling is different, so that the first image feature of each image scale is upsampled at least once to obtain multiple seventh image features with the same image scale.

[0249] See also Figure 10 As shown in Figure 1, starting from the smallest image scale, the number of upsampling processes for each image scale decreases in sequence. Taking the image scales of the above five layers as an example, the processing processes for each image scale are as follows:

[0250] (1) When the image scale is 1 / 32, three upsampling processes are required to obtain the seventh image feature with a size of 1 / 4. Each upsampling process can include 3×3 convolution, group normalization (groupNorm), ReLU and 2× bilinear upsampling.

[0251] (2) When the image scale is 1 / 16, two upsampling processes are required to obtain the seventh image feature with a size of 1 / 4.

[0252] (3) When the image scale is 1 / 8, an upsampling process is required to obtain the seventh image feature with a size of 1 / 4.

[0253] (4) When the image scale is 1 / 4, the upsampling process does not need to include 2× bilinear upsampling, but only needs to perform convolution operations, normalization operations, etc.

[0254] In this embodiment of the present application, after obtaining multiple seventh image features of the same image scale for a first image, the multiple seventh image features are fused to obtain an eighth image feature. The obtained eighth image feature is then upsampled to obtain a ninth image feature of the same image scale as the first image. The upsampling process is similar to that described above, except that the upsampling factor is 4x. Furthermore, based on the obtained ninth image feature, the probability of each pixel in the first image being a change point is determined, and a second image region is obtained based on the obtained probabilities.

[0255] Specifically, feature fusion can be performed in a variety of ways, such as superposition, splicing, or pooling.

[0256] See also Figure 10As shown, starting from the deepest image scale, an upsampling process is performed to produce a 1 / 16 scale feature map, where the upsampling process includes 3×3 convolution, groupNorm, ReLU, and 2× bilinear upsampling. This strategy is repeated for 1 / 16, 1 / 8, and 1 / 4 image scales (but the sampling stage is gradually reduced). The result is a 1 / 4 scale feature map, which is then processed by 1×1 convolution, 4× bilinear upsampling, and softmax to generate a defect segmentation map at the original image resolution.

[0257] After the target change detection model is obtained through training through the above process, the embodiment of the present application can use the target change detection model to implement the image change detection method provided by the embodiment of the present application, see Figure 11 As shown, the image change detection method provided in the embodiment of the present application includes the following steps:

[0258] Step 1101: Obtain a target image to be detected, and input the target image into the target change detection model trained by the above training method;

[0259] Step 1102: Perform single-phase image change detection on the target image to be detected using the target change detection model to obtain a target image region where the target image has changed relative to the standard image.

[0260] Compared with other methods, the embodiments of the present application increase sample diversity and interaction between samples, thereby improving defect location accuracy and being more robust to inherent changes in the machine. In addition, this method takes 15ms to infer on a V100 graphics processing unit (GPU), has low computational complexity, high accuracy, and is easy to apply.

[0261] The method of the embodiment of the present application can be used as one of the options for industrial AI quality inspection algorithms.

[0262] Specifically, when applied to industrial AI quality inspection scenarios, the above-mentioned target image is the image of the product to be detected for defects, and the standard image is a defect-free image of the same product as the target image. Then, by calling the target change detection model and obtaining the target image area where the target image has changed relative to the standard image, the target image area can be determined as a defect area.

[0263] Here we take the defect detection of the camera bracket accessories of the mobile phone as an example, which can be done through Figure 12 The process shown in the figure implements the corresponding detection process, see Figure 12 The figure shows a defect detection process diagram of a mobile phone camera bracket accessory.

[0264] Step 1201: Select a sample image from accessory images taken at the same point on a camera bracket accessory production line.

[0265] Step 1202: construct a sample image group from the selected sample images by shift combination. The label of each sample image is obtained by comparing it with the standard image, and the intra-group change label of each sample image group is obtained by performing an XOR operation on the labels of the two sample images.

[0266] Step 1203: Based on the constructed sample image group, the change detection model training method provided in the embodiment of the present application is used to train a target change detection model for defect detection of camera bracket accessories.

[0267] Step 1204: Capture an image of the camera bracket accessory from the same point on the camera bracket accessory production line.

[0268] Step 1205: Input the accessory image to the target change detection model. The target change detection model uses its own single-phase image change detection network to detect the changed area between the accessory image and the standard image.

[0269] Step 1206: The target change detection model outputs the detection result.

[0270] Step 1207: If the test result indicates that there is a changed area (or the degree of change is greater than a certain threshold), the camera bracket accessory is listed as an unqualified product, and the test result is pushed to relevant personnel to issue an alarm when an unqualified product is produced.

[0271] See also Figure 13 The following is a comparison of normal and defective camera bracket accessories. The white box indicates the defective area. The defective images show that the different types of defects exhibit different behaviors, and the size distribution of the defects varies greatly. For example, the defective areas of sticking, missing, and dirty defects are relatively small, making them difficult for the human eye to discern. The change detection model provided in this embodiment enhances the single-phase change detection network branch with a dual-phase change detection network branch, enhancing defect localization. This model exhibits high robustness against a variety of complex situations, and the model processing in the application phase requires low computational effort, resulting in higher detection efficiency.

[0272] See Figure 14 Based on the same inventive concept, the present application also provides a change detection model training device, which includes:

[0273] The sample acquisition unit 1401 is configured to obtain a plurality of sample image groups, each of which includes a content-registered first image, a second image, and a sample label, wherein the sample label indicates an actual image region of each of the first image and the second image that has changed relative to the standard image.

[0274] The change detection unit 1402 is configured to iteratively train the change detection model to be trained using multiple sample image groups to obtain a corresponding target change detection model. The change detection unit 1402 includes:

[0275] Intra-group detection subunit 14021, configured to perform intra-group image change detection on each input sample image group, and obtain a first image region where the second image in each sample image group has changed relative to the first image;

[0276] The single-phase detection subunit 14022 is configured to perform single-phase image change detection on each input first image to obtain a second image region where each first image has changed relative to the standard image;

[0277] The joint optimization subunit 14023 is used to determine the model loss based on the sample labels, the first image area and the second image area corresponding to each sample image group, and to adjust parameters based on the model loss.

[0278] Optionally, the change detection unit 1402 further includes a feature extraction subunit 14024;

[0279] a feature extraction subunit, configured to extract a first feature set from each first image, each first feature set comprising: first image features corresponding to a plurality of preset image scales for the corresponding first image; and to extract a second feature set from each second image, each second feature set comprising: second image features corresponding to a plurality of image scales for the corresponding second image;

[0280] The intra-group detection subunit 14021 is specifically configured to perform intra-group image change detection on each sample image group based on each obtained first feature set and each second feature set, and obtain a first image region corresponding to each sample image group.

[0281] Optionally, the intra-group detection subunit 14021 is specifically configured to:

[0282] For each sample image group, perform the following operations:

[0283] For a sample image group, feature fusion is performed on the image features corresponding to the same image scale in the corresponding first feature set and the second feature set, to obtain fused image features corresponding to multiple image scales;

[0284] Based on the obtained features of each fused image, intra-group image change detection is performed on a sample image group to obtain a first image region.

[0285] Optionally, the intra-group detection subunit 14021 is specifically configured to:

[0286] For multiple image scales, perform the following operations respectively:

[0287] For an image scale, performing a self-attention mechanism-based update process on a corresponding first image feature in a first feature set to obtain a third image feature, and performing a self-attention mechanism-based update process on a corresponding second image feature in a second feature set to obtain a fourth image feature;

[0288] Based on the third image feature, performing an attention-based fusion process on the fourth image feature to obtain a fifth image feature; and, based on the fourth image feature, performing an attention-based fusion process on the third image feature to obtain a sixth image feature;

[0289] Feature fusion is performed based on the fifth image feature and the sixth image feature to obtain a fused image feature at an image scale.

[0290] Optionally, the intra-group detection subunit 14021 is specifically configured to:

[0291] Processing the first image feature through the self-attention network of the change detection model to obtain a first attention weight of each element in the first image feature, where each first attention weight is used to represent the degree of association between the corresponding element and other elements;

[0292] Based on the obtained first attention weights, the first image features are weightedly processed through the self-attention network to obtain the third image features.

[0293] Optionally, the intra-group detection subunit 14021 is specifically configured to:

[0294] Processing the third image feature and the fourth image feature through the attention network of the change detection model to obtain a second attention weight for each element in the fourth image feature, where each second attention weight is used to represent the degree of association between the corresponding element of the third image feature and the fourth image feature;

[0295] Based on the obtained second attention weights, the third image feature is weightedly processed through the attention network to obtain the fifth image feature.

[0296] Optionally, the intra-group detection subunit 14021 is specifically configured to:

[0297] A fused image feature is obtained based on a difference between corresponding elements in the fifth image feature and the sixth image feature.

[0298] Optionally, the intra-group detection subunit 14021 is specifically configured to:

[0299] Starting from the largest image scale, the fused image features of each image scale are downsampled in turn, and the obtained downsampled image features are merged with the fused image features of the next level image scale until the smallest image scale is reached to obtain the merged image features;

[0300] Perform multiple upsampling processes on the merged image features until the target image features with the same image scale as the original image are obtained;

[0301] Based on the target image feature, a first image region is determined.

[0302] Optionally, the single-phase detection subunit 14022 is specifically used to:

[0303] For each first image, perform the following steps respectively:

[0304] For a first image, performing scale conversion processing on first image features of each image scale to obtain a plurality of seventh image features of the same image scale;

[0305] performing feature fusion on the obtained plurality of seventh image features to obtain an eighth image feature;

[0306] Performing upsampling processing based on the eighth image feature to obtain a ninth image feature having the same image scale as the first image;

[0307] Based on the ninth image feature, a probability that each pixel in the first image belongs to a change point is determined, and a second image region is obtained based on each obtained probability.

[0308] Optionally, the single-phase detection subunit 14022 is specifically used to:

[0309] performing at least one upsampling process on the first image features of each image scale to obtain a plurality of seventh image features of the same image scale;

[0310] Among them, starting from the smallest image scale, the number of upsampling processes of each image scale decreases in sequence.

[0311] Optionally, the joint optimization subunit 14023 is specifically configured to:

[0312] Determining, based on each first image region and the corresponding sample label, a single-phase change detection loss corresponding to the change detection model;

[0313] Determining a single-phase change detection loss corresponding to the change detection model based on each second image region and the corresponding sample label;

[0314] The model loss is determined based on the within-group change detection loss, the single-phase change detection loss, and their respective weights.

[0315] See Figure 15 Based on the same inventive concept, the embodiment of the present application further provides an image change detection device 150, which includes:

[0316] An image input unit 1501 is used to obtain a target image to be detected and input the target image into a target change detection model trained based on any of the above methods;

[0317] The image change detection unit 1502 is configured to perform single-phase image change detection on the target image to be detected using a target change detection model, and obtain a target image region where the target image has changed relative to the standard image.

[0318] Optionally, the target image is an image of a product to be inspected for defects, and the standard image is a non-defective image of the same product as the target image;

[0319] The image change detection unit 1502 is further configured to determine the target image area as a defect area.

[0320] Through the above-mentioned device, when model training is performed, by constructing a sample image group of a first image and a second image with content registration, single-phase change detection can be performed on the first image in one data processing branch, and dual-phase change detection can be performed on the first image and the second image in another data processing branch. This allows the parameters of the change detection model to be adjusted based on the processing results of the two data processing branches combined with the sample labels. Thus, by performing change detection on the two content-registered images, the overall model is adjusted to enhance the single-phase change detection using the dual-phase change detection between the two images, thereby improving the accuracy of the single-phase change detection branch. At the same time, when applying the image change detection method to perform change detection, for the target image, only a single data processing branch, namely single-phase change detection, is required to obtain the area of the target image that has changed relative to the standard image, thereby greatly reducing the amount of data processing and improving the detection efficiency of image change detection in actual detection scenarios.

[0321] For the convenience of description, the above parts are divided into various unit modules (or modules) according to their functions and described separately. Of course, when implementing this application, the functions of each unit (or module) can be implemented in the same or multiple software or hardware.

[0322] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0323] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, please refer to the description of the aforementioned embodiments and no further details will be given.

[0324] See Figure 16 Based on the same technical concept, the present application also provides a computer device. In one embodiment, the computer device can be Figure 1 The server mentioned in the corresponding embodiment, the computer device is as follows Figure 16 As shown, it includes a memory 1601 , a communication module 1603 and one or more processors 1602 .

[0325] Memory 1601 is used to store computer programs executed by processor 1602. Memory 1601 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and programs required for running instant messaging functions, while the data storage area may store various instant messaging messages and operating instruction sets.

[0326] Memory 1601 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1601 may be a combination of the aforementioned memories.

[0327] The processor 1602 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 1602 is configured to implement the above-mentioned change detection model training method or image change detection method when calling the computer program stored in the memory 1601 .

[0328] The communication module 1603 is used to communicate with terminal devices and other servers.

[0329] The specific connection medium between the memory 1601, the communication module 1603 and the processor 1602 is not limited in the embodiment of the present application. Figure 16 In the embodiment, the memory 1601 and the processor 1602 are connected via a bus 1604. Figure 16 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus 1604 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Figure 16 The diagram shows a single thick line, but this does not indicate that there is only one bus or one type of bus.

[0330] Memory 1601 stores a computer storage medium, which stores computer-executable instructions for implementing the change detection model training method or image change detection method of the embodiments of the present application. Processor 1602 is configured to execute the change detection model training method or image change detection method of each of the above embodiments.

[0331] In another embodiment, the computer device may also be other computer devices, such as Figure 1 The terminal device mentioned in the corresponding embodiment. In this embodiment, the structure of the computer device can be as follows Figure 17 As shown, it includes: a communication component 1710, a memory 1720, a display unit 1730, a camera 1740, a sensor 1750, an audio circuit 1760, a Bluetooth module 1770, a processor 1780 and other components.

[0332] The communication component 1710 is used to communicate with the server. In some embodiments, it may include a wireless fidelity (WiFi) module. The WiFi module is a short-range wireless transmission technology, and the computer device can help send and receive information through the WiFi module.

[0333] The memory 1720 can be used to store software programs and data. The processor 1780 executes various functions and data processing of the terminal device by running the software programs or data stored in the memory 1720. The memory 1720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1720 stores an operating system that enables the terminal device to run. In the present application, the memory 1720 can store an operating system and various application programs, and may also store code for executing the change detection model training method or image change detection method of the embodiment of the present application.

[0334] The display unit 1730 can also be used to display information input by the user or information provided to the user, as well as a graphical user interface (GUI) of various menus of the terminal device. Specifically, the display unit 1730 may include a display screen 1732 provided on the front of the terminal device. The display screen 1732 can be configured in the form of a liquid crystal display, a light-emitting diode, etc. The display unit 1730 can be used to display various pages in the embodiments of the present application, such as an image capture interface, a feature map display interface, a detection result display interface, etc.

[0335] The display unit 1730 can also be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal device. Specifically, the display unit 1730 may include a touch screen 1731 set on the front of the terminal device, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0336] The touch screen 1731 can be covered on the display screen 1732, or the touch screen 1731 and the display screen 1732 can be integrated to realize the input and output functions of the terminal device. The integrated touch screen can be simply called a touch display screen. In this application, the display unit 1730 can display applications and corresponding operation steps.

[0337] The camera 1740 can be used to capture still images, and users can post comments on the images captured by the camera 1740 through the application. The camera 1740 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the processor 1780 to convert it into a digital image signal.

[0338] The terminal device may further include at least one sensor 1750, such as an acceleration sensor 1751, a distance sensor 1752, a fingerprint sensor 1753, and a temperature sensor 1754. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0339] The audio circuit 1760, speaker 1761, and microphone 1762 provide an audio interface between the user and the terminal device. The audio circuit 1760 can convert the received audio data into an electrical signal and transmit it to the speaker 1761, which converts it into a sound signal for output. The terminal device may also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1760 and converted into audio data. The audio data is then output to the communication component 1710 for transmission to, for example, another terminal device, or the audio data is output to the memory 1720 for further processing.

[0340] The Bluetooth module 1770 is used to exchange information with other Bluetooth devices having a Bluetooth module through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable computer device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1770 to exchange data.

[0341] The processor 1780 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. By running or executing software programs stored in the memory 1720 and calling data stored in the memory 1720, it performs various functions of the terminal device and processes data. In some embodiments, the processor 1780 may include one or more processing units; the processor 1780 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1780. In the present application, the processor 1780 can run the operating system, application programs, user interface display and touch response, as well as the change detection model training method or image change detection method of the embodiment of the present application. In addition, the processor 1780 is coupled to the display unit 1730.

[0342] In some possible implementations, various aspects of the change detection model training method or image change detection method provided in the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the change detection model training method or image change detection method according to the various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the steps of each embodiment.

[0343] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0344] The program product of the embodiment of the present application may be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a computing device. However, the program product of the present application is not limited thereto. In the present application, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with a command execution system, device, or apparatus.

[0345] A readable signal medium may include a data signal transmitted in baseband or as part of a carrier wave, which carries readable program code. Such a transmitted data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with a command execution system, apparatus, or device.

[0346] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0347] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0348] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0349] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0350] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0351] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0352] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A change detection model training method, characterized in that: The method comprises: Obtaining a plurality of sample image groups, each sample image group comprising a content-registered first image, a second image, and a sample label, wherein the sample label is used to indicate: an actual image region of each of the first image and the second image that has changed relative to a standard image; The change detection model to be trained is iteratively trained using the multiple sample image groups to obtain a corresponding target change detection model. In each iteration, the following steps are performed on each input sample image group: Extracting first feature sets from each first image respectively, each first feature set comprising: first image features of the corresponding first image corresponding to a plurality of preset image scales; Extracting a second feature set from each second image respectively, each second feature set comprising: second image features of the corresponding second image corresponding to the plurality of image scales; For each image scale, performing an update process based on the self-attention mechanism on the corresponding first image feature in the first feature set to obtain a third image feature, and performing an update process based on the self-attention mechanism on the corresponding second image feature in the second feature set to obtain a fourth image feature; Based on the third image feature, performing an attention-based fusion process on the fourth image feature to obtain a fifth image feature; and based on the fourth image feature, performing an attention-based fusion process on the third image feature to obtain a sixth image feature; Performing feature fusion based on the fifth image feature and the sixth image feature to obtain a fused image feature of a corresponding image scale; Performing intra-group image change detection based on the obtained features of each fused image to obtain a first image region in which the second image has changed relative to the first image; Performing single-phase image change detection on each input first image to obtain a second image region where each first image has changed relative to the standard image; A model loss is determined based on the sample labels, the first image region, and the second image region corresponding to each sample image group, and parameter adjustment is performed based on the model loss.

2. The method according to claim 1, characterized in that Performing an update process based on the self-attention mechanism on the corresponding first image feature in the first feature set to obtain a third image feature, including: Processing the first image feature through the self-attention network of the change detection model to obtain a first attention weight of each element in the first image feature, each first attention weight being used to represent a degree of association between the corresponding element and other elements; Based on the obtained first attention weights, the first image features are weightedly processed through the self-attention network to obtain the third image features.

3. The method according to claim 1, characterized in that Based on the third image feature, performing a fusion process based on an attention mechanism on the fourth image feature to obtain a fifth image feature, including: processing the third image feature and the fourth image feature through the attention network of the change detection model to obtain a second attention weight for each element in the fourth image feature, each second attention weight being used to represent a degree of association between a corresponding element of the third image feature and the fourth image feature; Based on the obtained second attention weights, the third image feature is weightedly processed by the attention network to obtain the fifth image feature.

4. The method according to claim 1, wherein Performing feature fusion based on the fifth image feature and the sixth image feature to obtain fused image features of corresponding image scales includes: The fused image feature is obtained based on a difference between corresponding elements in the fifth image feature and the sixth image feature.

5. The method according to claim 1, wherein Performing intra-group image change detection based on the obtained features of each fused image to obtain a first image region where the second image has changed relative to the first image includes: Starting from the largest image scale, downsampling the fused image features of each image scale is performed in sequence, and the obtained downsampled image features are merged with the fused image features of the next level image scale until the smallest image scale is reached, thereby obtaining the merged image features; Performing multiple upsampling processes on the merged image features until a target image feature having the same image scale as the original image is obtained; Based on the target image features, the first image region is determined.

6. The method according to any one of claims 1 to 5, characterized in that Performing single-phase image change detection on each input first image to obtain a second image region where each first image has changed relative to the standard image includes: For each first image, perform the following steps respectively: For a first image, performing scale conversion processing on first image features of each image scale to obtain a plurality of seventh image features of the same image scale; performing feature fusion on the obtained plurality of seventh image features to obtain an eighth image feature; performing upsampling processing based on the eighth image feature to obtain a ninth image feature having the same image scale as that of the first image; Based on the ninth image feature, the probability that each pixel in the first image belongs to a change point is determined, and the second image region is obtained based on the obtained probabilities.

7. The method according to claim 6, characterized in that For a first image, performing scale conversion processing on first image features of each image scale to obtain a plurality of seventh image features of the same image scale includes: performing at least one upsampling process on the first image features of each image scale to obtain a plurality of seventh image features of the same image scale; Wherein, starting from the smallest image scale, the number of upsampling processes of each image scale decreases in sequence.

8. The method according to any one of claims 1 to 5, characterized in that: Determining a model loss based on the sample labels, the first image region, and the second image region corresponding to each sample image group includes: Determining, based on each first image region and the corresponding sample label, a single-phase change detection loss corresponding to the change detection model; determining a single-phase change detection loss corresponding to the change detection model based on each second image region and the corresponding sample label; The model loss is determined based on the within-group change detection loss, the single-phase change detection loss, and respective weights.

9. A method for detecting image changes, characterized in that: The method comprises: Obtaining a target image to be detected, and inputting the target image into a target change detection model trained based on the method of any one of claims 1 to 8; By using the target change detection model, single-phase image change detection is performed on the target image to be detected, and a target image region where the target image has changed relative to the standard image is obtained.

10. A change detection model training device, characterized in that: The device comprises: a sample acquisition unit, configured to obtain a plurality of sample image groups, each sample image group comprising a content-registered first image, a second image, and a sample label, wherein the sample label is used to indicate an actual image region in each of the first image and the second image that has changed relative to a standard image; a change detection unit, configured to iteratively train a change detection model to be trained using the plurality of sample image groups to obtain a corresponding target change detection model; wherein the change detection unit comprises: The intra-group detection subunit is configured to extract, for each input sample image group, a first feature set of each first image, each first feature set including: first image features of the corresponding first image corresponding to a plurality of preset image scales; extract a second feature set of each second image, each second feature set including: second image features of the corresponding second image corresponding to the plurality of image scales; for each image scale, perform a self-attention-based updating process on the corresponding first image features in the first feature set to obtain a third image feature, and perform a self-attention-based updating process on the corresponding second image features in the second feature set to obtain a fourth image feature; perform an attention-based fusing process on the fourth image features based on the third image feature to obtain a fifth image feature, and perform an attention-based fusing process on the third image features based on the fourth image feature to obtain a sixth image feature; perform feature fusion based on the fifth image feature and the sixth image feature to obtain a fused image feature of the corresponding image scale; perform intra-group image change detection based on the obtained fused image features to obtain a first image region in which the second image in each sample image group has changed relative to the first image; a single-phase detection subunit, configured to perform single-phase image change detection on each input first image, and obtain a second image region in each first image that has changed relative to the standard image; The joint optimization subunit is used to determine the model loss based on the sample labels, the first image area and the second image area corresponding to each sample image group, and to adjust the parameters based on the model loss.

11. An image change detection device, characterized in that: The device comprises: An image input unit, configured to obtain a target image to be detected, and input the target image into a target change detection model trained based on the method according to any one of claims 1 to 8; The image change detection unit is used to perform single-phase image change detection on the target image to be detected by using the target change detection model, and obtain the target image area where the target image has changed relative to the standard image.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A computer program product, characterized in that The computer program product includes computer instructions stored in a computer storage medium; The processor of the computer device reads the computer instructions from the computer storage medium and executes the computer instructions, so that the computer device implements the steps of the method according to any one of claims 1 to 9.

Images