Image detection model training method, difference detection method and related device

Abstract

The application discloses an image detection model training method, a difference detection method and related devices. The image detection model comprises a feature extraction network and a difference detection network. The method comprises the following steps: inputting a reference image and a detection image into the feature extraction network to obtain a reference feature map and a detection feature map; inputting the reference feature map and the detection feature map into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map; wherein the target region is determined based on feature correction values at the same positions on the reference feature map and the detection feature map, the feature correction values are determined based on a feature value group after a correction operation, the feature value group comprises feature values at the same positions on the reference feature map and the detection feature map, and the correction operation is related to the numerical values of the feature values in the feature value group; and the parameters of the image detection model are adjusted based on the target region until a trained image detection model is obtained. The above scheme can improve the accuracy of difference detection of the image detection model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to an image detection model training method, a difference detection method and related devices. BACKGROUND

[0002] With the rise of the field of computer vision, difference detection as an important branch of the field of computer vision is increasingly valued. Difference detection technology can be used to detect changes in the same scene at different time points. When there are a large number of images that need to be detected, a trained model is usually used to obtain a trained image detection model. The trained image detection model is used for difference detection to improve detection efficiency. The training effect of the image detection model directly determines the accuracy of the difference detection. Therefore, how to improve the accuracy of the image detection model for difference detection has become a problem to be solved. SUMMARY

[0003] The technical problem solved by the present application is to provide an image detection model training method, a difference detection method and related devices, which can improve the accuracy of the image detection model for difference detection.

[0004] To solve the above technical problem, the first aspect of the present application provides an image detection model training method. The image detection model includes a feature extraction network and a difference detection network. The method comprises: inputting a reference image and a detection image into the feature extraction network to obtain a reference feature map and a detection feature map; inputting the reference feature map and the detection feature map into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map; wherein the target region is determined based on feature correction values at the same positions on the reference feature map and the detection feature map, the feature correction values are determined based on a feature value group after a correction operation, the feature value group includes feature values at the same positions on the reference feature map and the detection feature map, and the correction operation is related to the numerical values of the feature values in the feature value group; adjusting the parameters of the image detection model based on the target region until a preset convergence condition is met, and obtaining a trained image detection model.

[0005] To solve the above technical problem, the second aspect of the present application provides a difference detection method. The method comprises: obtaining a group of images to be detected, wherein the group of images to be detected includes a reference image and an image to be detected; inputting the group of images to be detected into an image detection model to obtain a target region with differences; wherein the image detection model is trained by the method of the first aspect.

[0006] To solve the above technical problems, the third aspect of the present application provides an electronic device, comprising a memory and a processor coupled with each other, wherein the memory stores program data, and the processor invokes the program data to execute the method of the first aspect or the second aspect.

[0007] To solve the above technical problems, the fourth aspect of the present application provides a computer readable storage medium having program data stored thereon, wherein the program data is executed by a processor to implement the method of the first aspect or the second aspect.

[0008] The above scheme inputs the reference image and the detection image into the feature extraction network, so that the feature extraction network extracts features from the reference image to obtain a reference feature map, extracts features from the detection image to obtain a detection feature map, and inputs the reference feature map and the detection feature map into the difference detection network to determine the target region with differences on the reference feature map and the detection feature map based on the feature correction value at the same position on the reference feature map and the detection feature map, wherein the feature correction value is determined based on the feature value group after the correction operation, the feature value group includes the feature values at the same position on the reference feature map and the detection feature map, and the correction operation is related to the numerical value of the feature value in the feature value group. Therefore, compared with directly comparing the feature values on the reference feature map and the detection feature map, the feature corresponding to the feature value group is corrected by the correction operation to obtain the feature correction value, and the difference detection is performed based on the feature correction value, which can improve the accuracy of the difference detection to obtain a more accurate target region, and then adjust the parameters of the image detection model based on the target region until the preset convergence condition is met, to obtain the trained image detection model, so as to improve the accuracy of the trained image detection model in difference detection. BRIEF DESCRIPTION OF DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor. Among them:

[0010] Figure 1 is a flowchart of an embodiment of the image detection model training method of the present application;

[0011] Figure 2 is a flowchart of another embodiment of the image detection model training method of the present application;

[0012] Figure 3 is a flowchart of another embodiment of the image detection model training method of the present application;

[0013] Figure 4is a flowchart of another embodiment of the image detection model training method of the present application;

[0014] Figure 5 is a flowchart of another embodiment of the image detection model training method of the present application;

[0015] Figure 6 is a structural diagram of an embodiment of the image detection model of the present application;

[0016] Figure 7 is Figure 6 is a structural diagram of an embodiment of the feature matching module of the present application;

[0017] Figure 8 is Figure 6 is a structural diagram of an embodiment of the feature matching module of the present application;

[0018] Figure 9 is a flowchart of an embodiment of the difference detection method of the present application;

[0019] Figure 10 is a structural diagram of an embodiment of the electronic device of the present application;

[0020] Figure 11 is a structural diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0022] The terms "system" and "network" are often used interchangeably in this document. The term "and / or" in this document is only used to describe the association relationship of the associated objects. It means that there are three kinds of relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, the character " / " in this document generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" in this document means two or more than two.

[0023] The image detection model training method provided by the present application is used for training an image detection model, wherein the image detection model at least includes a feature extraction network and a difference detection network, and the image detection model is used for detecting the difference between images. The image detection model training method provided by the present application corresponds to an execution subject capable of calling the image detection model.

[0024] Referring to Figure 1 , Figure 1 is a flowchart of an embodiment of the image detection model training method of the present application. The method comprises:

[0025] S101: input the reference image and the detection image into the feature extraction network to obtain the reference feature map and the detection feature map.

[0026] Specifically, the reference image and the detection image constitute a training sample pair, the reference image and the detection image in the same training sample pair correspond to the same application scenario, and the reference image and the detection image are input into the feature extraction network to enable the feature extraction network to perform feature extraction on the reference image to obtain the reference feature map and perform feature extraction on the detection image to obtain the detection feature map.

[0027] In an application, the feature extraction network comprises first and second convolution modules with the same structure, and the parameters of the first and second convolution modules remain consistent during adjustment, that is, the first and second convolution modules in the feature extraction network are twin convolution modules. The reference image is input into the first convolution module to obtain the reference feature map, and the detection image is input into the second convolution module to obtain the detection feature map, thereby improving the efficiency of feature extraction based on the twin network.

[0028] In another application, the feature extraction network comprises cascaded convolution modules, and the reference image and the detection image are sequentially input into the cascaded convolution modules to perform feature extraction on the reference image and the detection image to obtain the reference feature map corresponding to the reference image and the detection feature map corresponding to the detection image, thereby reducing the complexity of the feature extraction network.

[0029] S102: input the reference feature map and the detection feature map into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map, wherein the target region is determined based on feature correction values at the same positions on the reference feature map and the detection feature map, the feature correction values are determined based on a feature value group after a correction operation, the feature value group comprises feature values at the same positions on the reference feature map and the detection feature map, and the correction operation is related to the numerical values of the feature values in the feature value group.

[0030] Specifically, the reference feature map and the detection feature map are input into the difference detection network to enable the difference detection network to determine the target region with differences on the reference feature map and the detection feature map based on the feature correction values at the same positions on the reference feature map and the detection feature map, wherein the feature correction values are determined based on a feature value group after a correction operation, the feature value group comprises feature values at the same positions on the reference feature map and the detection feature map, and the correction operation is related to the numerical values of the feature values in the feature value group.

[0031] In an application mode, the reference feature map and the detection feature map are input into the difference detection network, so that the difference detection network performs a correction operation on the feature values on the reference feature map and the detection feature map. The correction operation includes amplifying the difference between the feature values in the feature group, determining a feature correction value based on the amplified difference, and further determining a target region with differences on the reference feature map and the detection feature map based on the feature correction value, so as to improve the accuracy of difference detection.

[0032] In another application mode, the reference feature map and the detection feature map are input into the difference detection network, so that the difference detection network determines a detection region with differences on the reference feature map and the detection feature map based on the difference between the feature values in the feature value group, performs a correction operation on the feature values in the corresponding detection region on the reference feature map and the detection feature map. The correction operation includes amplifying the difference between the feature values in the feature group in the detection region, determining a feature correction value in the detection region based on the amplified difference, and further performing secondary verification on the detection region using the feature correction value to determine a target region with differences on the reference feature map and the detection feature map, so as to improve the accuracy of difference detection.

[0033] In an application scenario, the correction operation includes weighted sum of the absolute difference and the product between the feature values in the feature value group, thereby obtaining the feature correction value.

[0034] S103: Adjust the parameters of the image detection model based on the target region until the preset convergence condition is met, and obtain the trained image detection model.

[0035] Specifically, the reference map and the detection map correspond to an identified region with differences. The target region is compared with the identified region to obtain a loss value, and the parameters of the image detection model are adjusted based on the loss value until the preset convergence condition is met, and the trained image detection model is obtained.

[0036] In an application mode, the target region further includes a confidence, and the preset convergence condition is determined based on the loss value and the confidence of the target region. When the loss value is less than a loss threshold and the confidence exceeds a confidence threshold, the training process is ended, and the trained image detection model is obtained to obtain an image detection model with high confidence.

[0037] In another application mode, the preset convergence condition is determined based on the loss value and the number of training times. When the loss value is less than a loss threshold and the number of training times exceeds a number threshold, the training process is ended, and the trained image detection model is obtained to obtain an image detection model with high stability.

[0038] The above scheme inputs the reference image and the detection image into the feature extraction network, so that the feature extraction network extracts features of the reference image to obtain a reference feature map, extracts features of the detection image to obtain a detection feature map, and inputs the reference feature map and the detection feature map into the difference detection network to determine a target region with differences between the reference feature map and the detection feature map based on feature correction values at the same positions in the reference feature map and the detection feature map, wherein the feature correction values are determined based on a feature value group, the feature value group includes feature values at the same positions in the reference feature map and the detection feature map, and the correction operation is related to the numerical values of the feature values in the feature value group. Therefore, compared with directly comparing the feature values in the reference feature map and the detection feature map, the features corresponding to the feature value group are corrected by the correction operation to obtain the feature correction values, and the difference detection is performed based on the feature correction values, which can improve the accuracy of the difference detection to obtain a more accurate target region, and then the parameters of the image detection model are adjusted based on the target region until the preset convergence condition is met, and the trained image detection model is obtained, so as to improve the accuracy of the trained image detection model in difference detection.

[0039] Please refer to Figure 2 , Figure 2 is a flowchart of another embodiment of the image detection model training method of the present application. The method comprises:

[0040] S201: inputting the reference image and the detection image into the feature extraction network to obtain a reference feature map and a detection feature map.

[0041] Specifically, the reference image and the detection image are input into the feature extraction network, so that the feature extraction network extracts features of the reference image and the detection image respectively to obtain a reference feature map corresponding to the reference image and a detection feature map corresponding to the detection image.

[0042] S202: inputting the reference feature map and the detection feature map into the difference detection network, performing a correction operation on the feature value group based on the difference and the product between the feature values in the feature value group to obtain a feature correction value.

[0043] Specifically, the reference feature map and the detection feature map are input into the difference detection network, so that the difference detection network performs a correction operation on the feature value group based on the difference and the product between the feature values in the feature value group, thereby obtaining a corrected feature correction value.

[0044] In an application mode, the reference feature map and the detection feature map are input into the difference detection network to obtain a difference value feature map and a correlation feature map, wherein the feature values on the difference value feature map are obtained based on the difference between the feature values in the feature value group, and the feature values on the correlation feature map are obtained based on the product between the feature values in the feature value group, the difference value feature map and the correlation feature map are spliced, and the spliced feature map is dimensionally reduced to obtain a matching feature map, wherein the matching feature map has the same dimension as the reference feature map and the detection feature map, and the matching feature map corresponds to a feature correction value.

[0045] In another application mode, the reference feature map and the detection feature map are input into the difference detection network, the difference detection network includes a difference value operation branch and a cross-correlation operation branch, the difference value operation branch obtains the difference value of the features at the same position on the reference feature map and the detection feature map, and determines the absolute value of the difference value to obtain an absolute difference value, the cross-correlation operation branch obtains the product of the feature values at the same position on the reference feature map and the detection feature map to obtain a product value, and the absolute difference value and the product value are weighted and summed to determine the feature correction value on the matching feature map, thereby obtaining the matching feature map, wherein the matching feature map has the same dimension as the reference feature map and the detection feature map.

[0046] S203: determining a target region where the reference feature map and the detection feature map have differences based on the feature correction value.

[0047] Specifically, the feature correction value is compared with the feature value at the same position on the reference feature map, and the feature correction value is compared with the feature value at the same position on the detection feature map, the feature correction value whose difference with the feature value on the reference feature map or the detection feature map exceeds a feature value difference threshold is determined, and the target region where the reference feature map and the detection feature map have differences is determined based on the feature correction value whose difference exceeds the feature value difference threshold.

[0048] Further, when the difference between the feature values in the feature value group is greater, the feature correction value is more obviously different from the reference feature map or the detection feature map, so that the detection region where the reference feature map and the detection feature map have differences is determined based on the feature correction value, and compared with directly comparing the feature values on the reference feature map and the detection feature map, the feature corresponding to the feature value group is corrected by the correction operation to obtain the feature correction value, and the difference detection is performed based on the feature correction value, which can improve the accuracy of the difference detection and obtain a more accurate target region.

[0049] S204: adjusting the parameters of the image detection model based on the target region until a preset convergence condition is met, thereby obtaining a trained image detection model.

[0050] Specifically, the reference image and the detection image correspond to an identification region with a difference, a loss value is obtained by comparing the target region with the identification region, and parameters of the image detection model are adjusted based on the loss value until a preset convergence condition is met, and a trained image detection model is obtained.

[0051] Please refer to Figure 3 , Figure 3 is a flowchart of another embodiment of the image detection model training method of the present application, which comprises:

[0052] S301: input the reference image and the detection image into the feature extraction network to obtain the reference feature map and the detection feature map.

[0053] Specifically, the reference image and the detection image are input into the feature extraction network to enable the feature extraction network to perform feature extraction on the reference image and the detection image respectively, to obtain the reference feature map corresponding to the reference image and the detection feature map corresponding to the detection image.

[0054] S302: input the reference feature map and the detection feature map into the difference detection network, and determine the detection region with a difference on the reference feature map and the detection feature map based on the difference between the feature values in the feature value group.

[0055] Specifically, the reference feature map and the detection feature map are input into the difference detection network to enable the difference detection network to determine the region with a difference on the reference feature map and the detection feature map based on the difference between the feature values in the feature value group, and the region with a difference between the reference feature map and the detection feature map is determined as the detection region whose difference value exceeds the feature value difference threshold.

[0056] S303: based on the detection region, obtain the reference feature sub-map from the reference feature map and the detection feature sub-map from the detection feature map, and perform a correction operation on the feature values in the reference feature sub-map and the detection feature sub-map based on the feature values at the same positions in the reference feature sub-map and the detection feature sub-map to obtain a feature correction value.

[0057] Specifically, the region corresponding to the detection region is extracted from the reference feature map to obtain the reference feature sub-map, and the region corresponding to the detection region is extracted from the detection feature map to obtain the detection feature sub-map, and a correction operation is performed on the feature values in the reference feature sub-map and the detection feature sub-map based on the feature values at the same positions in the reference feature sub-map and the detection feature sub-map to obtain a feature correction value.

[0058] In an application, the correction operation includes amplifying the difference between the feature values at the same positions in the reference feature sub-map and the detection feature sub-map, and obtaining the feature correction value based on the difference between the feature values at the same positions in the reference feature sub-map and the detection feature sub-map and the product.

[0059] In another application mode, the similarity between the reference feature subgraph and the detection feature subgraph is determined based on the feature values at the same positions of the reference feature subgraph and the detection feature subgraph, and the correction operation includes correcting the feature values on the reference feature subgraph and the detection feature subgraph based on the similarity to obtain feature correction values.

[0060] In an application scenario, convolution operations are performed on the feature values at the same positions of the reference feature subgraph and the detection feature subgraph to obtain a reference feature vector corresponding to the reference feature subgraph and a detection feature vector corresponding to the detection feature subgraph, the similarity between the reference feature subgraph and the detection feature subgraph is determined based on the reference feature vector and the detection feature vector, the corresponding feature correction value on the reference feature subgraph is obtained based on the product between the similarity and the feature value on the reference feature subgraph, and the corresponding feature correction value on the detection feature subgraph is obtained based on the product between the square value of the similarity and the feature value on the detection feature subgraph, so as to expand the difference between the feature correction values on the reference feature subgraph and the detection feature subgraph, thereby screening the detection region based on the similarity and improving the accuracy of difference detection.

[0061] S304: Determine the target region with differences between the reference feature map and the detection feature map based on the feature correction values.

[0062] Specifically, the feature correction values are compared with the feature values at the same positions of the reference feature map, and the feature correction values are compared with the feature values at the same positions of the detection feature map, the feature correction values with a difference value exceeding a feature value difference threshold value between the feature values on the reference feature map or the detection feature map are determined, and the target region with differences between the reference feature map and the detection feature map is determined based on the feature correction values with the difference value exceeding the feature value difference threshold value.

[0063] S305: Adjust the parameters of the image detection model based on the target region until a preset convergence condition is met to obtain a trained image detection model.

[0064] Specifically, the reference map and the detection map correspond to an identified region with differences, the loss value is obtained by comparing the target region with the identified region, and the parameters of the image detection model are adjusted based on the loss value until a preset convergence condition is met to obtain a trained image detection model.

[0065] Please refer to Figure 4 , Figure 4 is a flowchart of another embodiment of the image detection model training method of the present application, which comprises:

[0066] S401: Input the reference map and the detection map into the feature extraction network to obtain the reference feature map and the detection feature map.

[0067] Specifically, the reference image and the detection image constitute a training sample pair, the reference image and the detection image in the same training sample pair correspond to the same application scene, and the reference image and the detection image are input into the feature extraction network to enable the feature extraction network to perform feature extraction on the reference image to obtain a reference feature map and perform feature extraction on the detection image to obtain a detection feature map.

[0068] S402: input the reference feature map and the detection feature map into a feature matching module to obtain a matching feature map, wherein a feature correction value on the matching feature map is determined based on a difference and a product between feature values in the feature value group.

[0069] Specifically, the reference feature map and the detection feature map are input into the feature matching module to enable the feature matching module to determine the feature correction value on the matching feature map based on the difference and the product between the feature values at the same positions on the reference feature map and the detection feature map, and obtain the matching feature map.

[0070] In an application mode, the feature matching module includes a difference operation branch, a cross-correlation operation branch, and a splicing module. After the reference feature map and the detection feature map are input into the feature matching module, a difference feature map is obtained based on the difference operation branch, wherein the difference operation branch calculates the difference between the features at the same positions on the reference feature map and the detection feature map, and determines the absolute value of the difference to obtain the feature value on the difference feature map. A correlation feature map is obtained based on the cross-correlation operation branch, wherein the cross-correlation operation branch calculates the product of the feature values at the same positions on the reference feature map and the detection feature map to obtain the correlation feature map on the correlation feature map. The difference feature map and the correlation feature map are spliced by using the splicing module, and the spliced feature map is dimensionally reduced to obtain the matching feature map, wherein the matching feature map has the same dimension as the reference feature map and the detection feature map.

[0071] In another application mode, the feature matching module includes a difference operation branch and a cross-correlation operation branch. The reference feature map and the detection feature map are input into the feature matching module. The difference operation branch calculates the difference between the features at the same positions on the reference feature map and the detection feature map, and determines the absolute value of the difference to obtain an absolute difference value. The cross-correlation operation branch calculates the product of the feature values at the same positions on the reference feature map and the detection feature map to obtain a product value. The absolute difference value and the product value are weighted and summed to determine the feature correction value on the matching feature map, and the matching feature map is obtained, wherein the matching feature map has the same dimension as the reference feature map and the detection feature map.

[0072] It should be noted that the feature correction value on the matching feature map obtained after the feature matching module processes the reference feature map and the detection feature map is equivalent to amplifying the difference between the feature values at the same positions on the reference feature map and the detection feature map. The greater the difference between the feature values at the same positions on the reference feature map and the detection feature map, the more obvious the feature correction value at the corresponding position on the matching feature map is different from the reference feature map or the detection feature map. Therefore, based on the feature correction value on the matching feature map, the detection region with differences between the reference feature map and the detection feature map is determined, which can improve the accuracy of determining the detection region with differences.

[0073] S403: determining a detection region with differences between the reference feature map and the detection feature map based on the feature correction value on the matching feature map.

[0074] Specifically, the feature correction value on the matching feature map is used to find the region with differences between the reference feature map and the detection feature map, and the detection region with differences between the reference feature map and the detection feature map is determined.

[0075] In an application mode, the difference between the feature values corresponds to a feature value difference threshold. The difference between the feature correction value and the feature value at the same position on the matching feature map and the reference feature map, and the difference between the feature correction value and the feature value at the same position on the matching feature map and the detection feature map are compared with the feature value difference threshold. The detection coordinates corresponding to the feature correction value with the difference exceeding the feature value difference threshold on the matching feature map are determined. The region surrounded by the detection coordinates is regarded as the detection region with differences between the reference feature map and the detection feature map, which improves the efficiency of obtaining the detection region.

[0076] In another application mode, the difference between the feature values corresponds to a feature value difference threshold. The difference between the feature correction value and the feature value at the same position on the matching feature map and the reference feature map, and the difference between the feature correction value and the feature value at the same position on the matching feature map and the detection feature map are compared with the feature value difference threshold. The initial feature value with the difference exceeding the feature value difference threshold on the matching feature map is determined. It is determined whether other initial feature values are included within the preset radius centered on each initial feature value. The initial feature value with other initial feature values included within the preset radius is regarded as a target feature value. The region gathered by the target feature value is regarded as the detection region with differences between the reference feature map and the detection feature map, so as to eliminate discrete initial feature values and improve the accuracy of the detection region.

[0077] S404: obtaining a reference feature sub-map from the reference feature map based on the detection region, obtaining a detection feature sub-map from the detection feature map, inputting the reference feature sub-map and the detection feature sub-map into a feature comparison module, and obtaining a target region, wherein the target region is determined based on the feature values on the reference feature sub-map and the detection feature sub-map.

[0078] Specifically, a region corresponding to the detection region is extracted from the reference feature map to obtain a reference feature subgraph, a region corresponding to the detection region is extracted from the detection feature map to obtain a detection feature subgraph, and the reference feature subgraph and the detection feature subgraph are input into the feature comparison module, so that the feature comparison module determines the target region based on the feature values on the reference feature subgraph and the detection feature subgraph, thereby performing secondary confirmation on the detection region and improving the accuracy of the difference detection.

[0079] In an application mode, the feature comparison module includes a convolution module with structural twinning, which includes two convolution modules with the same structure. The reference feature subgraph is input into one of the convolution modules to obtain a reference feature vector, and the detection feature subgraph is input into the other convolution module to obtain a detection feature vector. Based on the reference feature vector and the detection feature vector, the similarity between the reference feature subgraph and the detection feature subgraph is determined, and the detection region is screened using the similarity. The detection region with a similarity exceeding a similarity threshold is taken as the target region.

[0080] In another application mode, the feature comparison module includes a convolution module in cascade. The reference feature subgraph and the detection feature subgraph are sequentially input into the convolution module in cascade to perform feature extraction on the reference feature subgraph and the detection feature subgraph, respectively obtaining a reference feature vector corresponding to the reference feature subgraph and a detection feature vector corresponding to the detection feature subgraph. Based on the reference feature vector and the detection feature vector, the similarity between the reference feature subgraph and the detection feature subgraph is determined, and the detection region is screened using the similarity. The detection region with a similarity exceeding a similarity threshold is taken as the target region.

[0081] S405: Adjust the parameters of the image detection model based on the target region until a preset convergence condition is met, and obtain a trained image detection model.

[0082] Specifically, the reference map and the detection map correspond to an identified region with a difference. The target region is compared with the identified region to obtain a loss value, and the parameters of the image detection model are adjusted based on the loss value until a preset convergence condition is met, and a trained image detection model is obtained.

[0083] Please refer to Figure 5 , Figure 5 is a flowchart of another embodiment of the image detection model training method of the present application. The method includes:

[0084] S501: Input the reference map and the detection map into the feature extraction network to obtain a reference feature map and a detection feature map.

[0085] Specifically, please refer to Figure 6 , Figure 6is a structural schematic diagram of an embodiment of an image detection model of the present application, the feature extraction network includes first and second convolution modules of the same structure, and the parameters of the first and second convolution modules are always consistent when adjusted. That is, the first and second convolution modules are a twin network, and the parameters such as weights are shared between each other.

[0086] In an application scenario, the reference image in the training sample pair is input into the first convolution module to obtain a reference feature map, and the detection image in the training sample pair is input into the second convolution module to obtain a detection feature map, wherein the reference image and the detection image corresponding to the same application scenario form the training sample pair.

[0087] Specifically, the reference image and the detection image in the training sample pair correspond to the same application scenario, and the reference image and the detection image in the training sample pair correspond to different time points, so as to form a control group with differences as much as possible. The reference image in the training sample pair is input into the first convolution module for feature extraction to obtain a reference feature map, and the detection image in the training sample pair is input into the second convolution module for feature extraction to obtain a detection feature map, so as to improve the efficiency of feature extraction by using the twin network and ensure the consistency of the reference image and the detection image in feature extraction.

[0088] S502: inputting the reference feature map and the detection feature map into the feature matching module to obtain a matching feature map, wherein the feature correction value on the matching feature map is determined based on the difference and the product between the feature values in the feature value group.

[0089] Specifically, referring again to Figure 6 , the reference feature map and the detection feature map are input into the feature matching module, so that the feature matching module determines a difference feature map using the difference between the feature values at the same positions on the reference feature map and the detection feature map, and determines a correlation feature map using the product between the feature values at the same positions on the reference feature map and the detection feature map, so as to obtain a matching feature map based on the difference feature map and the correlation feature map.

[0090] In an application scenario, the reference feature map and the detection feature map are input into the feature matching module, a difference feature map is obtained based on the absolute difference between the feature values in the feature value group, and a correlation feature map is obtained based on the product between the feature values in the feature value group; a matching feature map is obtained based on the difference feature map and the correlation feature map.

[0091] Specifically, referring to Figure 7 , Figure 7 is Figure 6An embodiment of the feature matching module corresponds to the structural schematic diagram shown in FIG. 6. The feature matching module includes a difference operation branch and a cross-correlation operation branch. The reference feature map and the detection feature map are input into the feature matching module. The difference operation branch performs difference operation on the feature values at the same positions of the reference feature map and the detection feature map, and obtains absolute values, thereby obtaining absolute difference values as the feature values on the difference feature map. The cross-correlation operation branch performs multiplication on the feature values at the same positions of the reference feature map and the detection feature map, thereby obtaining product values as the feature values on the correlation feature map.

[0092] Further, based on the difference feature map and the correlation feature map, a matching feature map is obtained, thereby synthesizing the feature values on the difference feature map and the correlation feature map, so that a more accurate difference detection result can be obtained based on the matching feature map.

[0093] In a specific application scenario, based on the difference feature map and the correlation feature map, the matching feature map is obtained, including: splicing the difference feature map and the correlation feature map to obtain a spliced feature map; and reducing the dimension of the spliced feature map to obtain the matching feature map; wherein the matching feature map has the same dimension as the reference feature map and the detection feature map.

[0094] Specifically, referring again to FIG. 6, Figure 7 The difference feature map and the correlation feature map are spliced to obtain a spliced feature map, thereby synthesizing the feature values of the difference feature map and the correlation feature map to deeply mine the difference, and obtain the spliced feature map, and then the dimension of the spliced feature map is reduced to obtain the matching feature map, so as to ensure that the matching feature map has the same dimension as the reference feature map and the detection feature map. The dimension reduction operation can specifically use a convolution layer for dimension reduction, for example, a convolution kernel of 1*1 is used for dimension reduction of the matching feature map, so as to keep all the feature maps having the same dimension and improve the accuracy of difference detection.

[0095] S503: inputting the matching feature map into the regression classification module, determining at least one initial region where the difference exists between the reference feature map and the detection feature map based on the feature values on the matching feature map, and outputting the confidence corresponding to each initial region.

[0096] Specifically, referring again to FIG. 6, Figure 6 The difference detection network further includes a regression classification module. The matching feature map is input into the regression classification module, so that the regression classification module mines at least one initial region where the difference exists between the reference feature map and the detection feature map based on the feature values on the matching feature map, and each initial region corresponds to a confidence to represent the reliability of the result.

[0097] Further, the feature values on the reference feature map and the correlation feature map obtained after feature extraction are all between 0 and 1, so that when the difference between the feature values at the same positions on the reference feature map and the correlation feature map is large, the feature correction value on the matching feature map obtained based on the difference feature map and the correlation feature map is at least one value with a large difference compared with the feature values on the reference feature map and the detection feature map, so that at least one initial region with a difference between the reference feature map and the detection feature map is obtained based on the difference, and the confidence corresponding to the initial region is output, so as to improve the accuracy and reliability of the difference detection.

[0098] S504: filtering all the initial regions by using the confidence to obtain a detection region.

[0099] Specifically, the confidence corresponds to a confidence threshold, all the initial regions are filtered based on the confidence, the initial regions with a confidence less than the confidence threshold are deleted, and the initial regions with a confidence greater than or equal to the confidence threshold are taken as the detection region, so as to filter the results with low reliability and improve the precision of the difference detection.

[0100] S505: obtaining a reference feature sub-map from the reference feature map based on the detection region, obtaining a detection feature sub-map from the detection feature map based on the detection region, inputting the reference feature sub-map and the detection feature sub-map into a feature comparison module to obtain a target region, wherein the target region is determined based on the feature values on the reference feature sub-map and the detection feature sub-map.

[0101] Specifically, referring again to Figure 6 , the region of interest is extracted from the reference feature map based on the detection region to obtain the reference feature sub-map, the region of interest is extracted from the detection feature map based on the detection region to obtain the detection feature sub-map; the reference feature sub-map and the detection feature sub-map are input into the feature comparison module to obtain the similarity between the reference feature sub-map and the detection feature sub-map; the probability of the difference between the reference feature sub-map and the detection feature sub-map is determined based on the similarity, and the detection region is filtered by using the probability to obtain the target region.

[0102] In an application scenario, the detection region is used to extract the region of interest from the reference feature map and the detection feature map respectively, so as to obtain the reference feature subgraph corresponding to the reference feature map and the detection feature subgraph corresponding to the detection feature map. The reference feature subgraph and the detection feature subgraph are input to the feature comparison module as input images for re-verification. The similarity between the reference feature subgraph and the detection feature subgraph is determined. Whether there is a difference between the reference feature subgraph and the detection feature subgraph is judged based on the similarity. The probability of the difference between the reference feature subgraph and the detection feature subgraph is obtained. The detection region is filtered by using the probability. The detection region with a probability greater than a probability threshold is removed. The detection region with a probability less than or equal to the probability threshold is taken as a target region, that is, the detection region with a low similarity probability is taken as a target region with a difference. Thus, through the secondary verification of the reference feature subgraph and the detection feature subgraph, the accuracy and reliability of the difference detection result are improved.

[0103] In a specific application scenario, refer to Figure 8 , Figure 8 is Figure 6 the structural schematic diagram of an embodiment corresponding to the feature comparison module in

[0104] S506: Adjust the parameters of the image detection model based on the target region until the preset convergence condition is met, and obtain the trained image detection model.

[0105] Specifically, the reference image and the detection image correspond to an identified region with a difference. The loss value of the image detection model is determined based on the target region and the identified region. The parameters of the image detection model are adjusted based on the loss value until the preset convergence condition is met, and the trained image detection model is obtained.

[0106] In an application, based on the target region and the identified region, the confidence loss and the position prediction loss of the regression classification module, and the similarity loss of the feature comparison module are determined. The confidence loss, the position prediction loss and the similarity loss are weighted and summed to obtain the detection loss of the difference detection network. The total loss of the image detection model is determined based on the detection loss, and the parameters of the image detection model are adjusted based on the total loss.

[0107] Specifically, the confidence loss and the position prediction loss of the regression classification module are determined by comparing the target region and the identification region, and the similarity loss of the feature comparison module is determined, so that the confidence loss, the position prediction loss and the similarity loss are weighted and summed to obtain the detection loss of the difference detection network, wherein the sum of the weights of the confidence loss, the position prediction loss and the similarity loss is 1, and the rationality of the detection loss is improved by weighted summation. The total loss of the image detection model is determined on the basis of the detection loss, and then the parameters of the image detection model are adjusted based on the total loss to improve the training effect of the image detection model, wherein the total loss can also include the loss value of the feature extraction network.

[0108] In an application scenario, based on the target region and the identification region, the confidence loss and the position prediction loss of the regression classification module are determined, and the similarity loss of the feature comparison module is determined, including: using a first loss function, based on the confidence of each initial region and the identification region, the confidence loss of the regression classification module is determined; using a second loss function, based on each target region and the identification region, the position prediction loss of the regression classification module is determined; using a third loss function, based on the difference between the feature values between the benchmark feature subgraph and the detection feature subgraph, the similarity loss of the feature comparison module is determined; wherein the first loss function, the second loss function and the third loss function are mutually distinguished.

[0109] Specifically, the confidence loss, the position prediction loss and the similarity loss each correspond to a loss function, the first loss function is a Focal Loss loss function, and the confidence loss of the regression classification module is determined based on the confidence of each initial region and the identification region using the Focal Loss loss function. The above process is represented by the following formula:

[0110]

[0111] wherein l cls is the confidence loss, and α and β are two hyperparameters, respectively 2 and 4, represents the class prediction probability at coordinates (x, y), and L x,y represents the true class supervision signal at coordinates (x, y).

[0112] Further, the detection region is represented in the form of a detection frame, which can be a rectangular frame or other shapes, and the second loss function is a L1 Loss loss function, which is used to determine the position prediction loss of the regression classification module based on each target region and the identification region. The above process is represented by the following formula:

[0113]

[0114] wherein lbbox for position prediction loss, representing a predicted detection box of the detection map, representing supervision information of a true detection box of the detection map, wherein x center ,y center is a center point of the detection box, h and w are width and height of the detection box.

[0115] Further, the third loss function is a Contrastive Loss loss function, which is used to determine a similarity loss of the feature comparison module based on a difference between feature values between the benchmark feature subgraph and the detection feature subgraph. The above process is represented by the following formula:

[0116]

[0117] wherein l cl is a similarity loss, represents the Euclidean distance (two norm) of two sample features X1 and X2, P represents the feature dimension of the sample, Y is a label indicating whether the two samples match, Y = 1 represents that the two samples are similar or match, m is a set threshold, and N is the number of samples.

[0118] In this embodiment, the image detection model is an end-to-end system framework, and after the image is input, a target region with differences can be obtained. The feature matching module determines a difference feature map based on a difference between feature values at the same position on the benchmark feature map and the detection feature map, and determines a correlation feature map based on a product of feature values at the same position on the benchmark feature map and the detection feature map. Based on the feature correction value on the matching feature map obtained from the difference feature map and the correlation feature map, there is at least one value with a large difference between the feature correction value and the feature values on the benchmark feature map and the detection feature map, so that at least one initial region with differences between the benchmark feature map and the detection feature map is obtained based on the difference, and a confidence corresponding to the initial region is output, so as to improve the accuracy and reliability of the difference detection. The feature comparison module is used to extract a region of interest from the benchmark feature map and the detection feature map based on the detection region, so as to obtain a benchmark feature subgraph corresponding to the benchmark feature map and a detection feature subgraph corresponding to the detection feature map. The benchmark feature subgraph and the detection feature subgraph are input to the feature comparison module as input images for re-verification. The detection region with a probability greater than a probability threshold is removed, and the detection region with a probability less than or equal to the probability threshold is taken as a target region, that is, the detection region with a low similarity probability is taken as a target region with differences. Thus, through the secondary verification of the benchmark feature subgraph and the detection feature subgraph, the accuracy and reliability of the difference detection result are improved.

[0119] Please refer toFigure 9 , Figure 9 is a flowchart of an embodiment of a difference detection method of the present application, the method comprising:

[0120] S901: obtaining a to-be-detected image group, wherein the to-be-detected image group comprises a reference benchmark image and a to-be-detected image.

[0121] Specifically, the reference benchmark image and the to-be-detected image correspond to the same application scenario, the to-be-detected image group is obtained, and the reference benchmark image and the to-be-detected image are extracted from the to-be-detected image group.

[0122] S902: inputting the to-be-detected image group into an image detection model to obtain a target region with a difference.

[0123] Specifically, the to-be-detected image group is inputted into the image detection model, so that the image detection model performs difference detection on the to-be-detected image group to obtain the target region with the difference, wherein the image detection model is obtained based on the image detection model training method in any of the above embodiments, and the related content is described in detail in the above method embodiments, which will not be repeated here.

[0124] Please refer to Figure 10 , Figure 10 is a structural diagram of an embodiment of an electronic device of the present application, the electronic device 100 comprising a memory 1001 and a processor 1002 coupled with each other, wherein the memory 1001 stores program data (not shown in the figure), and the processor 1002 invokes the program data to implement the method in any of the above embodiments, and the related content is described in detail in the above method embodiments, which will not be repeated here.

[0125] Please refer to Figure 11 , Figure 11 is a structural diagram of an embodiment of a computer readable storage medium of the present application, the computer readable storage medium 110 storing program data 1100, the program data 1100 being executed by a processor to implement the method in any of the above embodiments, and the related content is described in detail in the above method embodiments, which will not be repeated here.

[0126] It should be noted that the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0127] In addition, each of the functional units in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (such as a personal computer, a server, or a network device) or a processor to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: various types of U disks, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and various other media that can store program codes.

[0129] The above merely describes the embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation based on the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for training an image detection model, characterized in that, The image detection model comprises a feature extraction network and a difference detection network. The reference feature map and the detection feature map are input into the feature extraction network to obtain a reference feature map and a detection feature map. The reference feature map and the detection feature map are input into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map; wherein the target region is determined based on feature correction values at the same positions on the reference feature map and the detection feature map, the feature correction values are determined based on a feature value group after a correction operation, the feature value group comprises feature values at the same positions on the reference feature map and the detection feature map, and the correction operation is related to the numerical values of the feature values in the feature value group; the correction operation comprises amplifying the difference values between the feature values in the feature group, and the feature correction values are determined based on the amplified difference values. The parameters of the image detection model are adjusted based on the target region until a preset convergence condition is met, and a trained image detection model is obtained.

2. The image detection model training method of claim 1, wherein, The reference feature map and the detection feature map are input into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map, comprising: The reference feature map and the detection feature map are input into the difference detection network, and a correction operation is performed on the feature value group based on the difference and product between the feature values in the feature value group to obtain the feature correction values; The target region with differences on the reference feature map and the detection feature map is determined based on the feature correction values.

3. The image detection model training method of claim 1, wherein, The reference feature map and the detection feature map are input into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map, comprising: The reference feature map and the detection feature map are input into the difference detection network, and a correction operation is performed on the feature value group based on the difference and product between the feature values in the feature value group to obtain the feature correction values; The target region with differences on the reference feature map and the detection feature map is determined based on the feature correction values. The difference detection network comprises a feature matching module and a feature comparison module, and the reference feature map and the detection feature map are input into the difference detection network to obtain a target region with differences on the reference feature map and the detection feature map, comprising:

4. The image detection model training method of claim 1, wherein, The reference feature map and the detection feature map are input into the feature matching module to obtain a matching feature map; wherein the feature correction values on the matching feature map are determined based on the difference and product between the feature values in the feature value group; Based on the feature correction values on the matching feature map, a detection region with differences on the reference feature map and the detection feature map is determined; ​ The reference feature subgraph and the detection feature subgraph are input into the feature comparison module to obtain a target region, wherein the target region is determined based on feature values in the reference feature subgraph and the detection feature subgraph.

5. The image detection model training method of claim 4, wherein, The inputting of the reference feature map and the detection feature map into the feature matching module to obtain a matching feature map comprises: The reference feature map and the detection feature map are input into the feature matching module, a difference value feature map is obtained based on absolute difference values between feature values in the feature value group, and a correlation feature map is obtained based on products between feature values in the feature value group. The matching feature map is obtained based on the difference value feature map and the correlation feature map.

6. The image detection model training method of claim 5, wherein, The inputting of the reference feature map and the detection feature map into the feature matching module to obtain a matching feature map comprises: The difference value feature map and the correlation feature map are spliced to obtain a spliced feature map; The spliced feature map is reduced in dimension to obtain the matching feature map, wherein the matching feature map has the same dimension as the reference feature map and the detection feature map.

7. The image detection model training method of claim 4, wherein, The difference detection network further comprises a regression classification module, and the determination of the detection region with differences between the reference feature map and the detection feature map based on the feature correction value in the matching feature map comprises: The matching feature map is input into the regression classification module, at least one initial region with differences between the reference feature map and the detection feature map is determined based on the feature correction value in the matching feature map, and a confidence degree corresponding to each initial region is output; All the initial regions are filtered by using the confidence degree to obtain the detection region.

8. The image detection model training method of claim 7, wherein, The reference map and the detection map correspond to an identified region with differences, and the adjustment of the parameters of the image detection model based on the target region comprises: The confidence degree loss and the position prediction loss of the regression classification module and the similarity loss of the feature comparison module are determined based on the target region and the identified region; The confidence degree loss, the position prediction loss and the similarity loss are weighted and summed to obtain a detection loss of the difference detection network; The total loss of the image detection model is determined based on the detection loss, and the parameters of the image detection model are adjusted based on the total loss.

9. The image detection model training method of claim 8, wherein, The confidence degree loss and the position prediction loss of the regression classification module and the similarity loss of the feature comparison module are determined based on the target region and the identified region, comprising: The confidence degree loss of the regression classification module is determined based on the confidence degree corresponding to each initial region and the identified region by using a first loss function; The position prediction loss of the regression classification module is determined based on each target region and the identified region by using a second loss function; The similarity loss of the feature comparison module is determined based on the difference between feature values between the reference feature subgraph and the detection feature subgraph by using a third loss function; and The first loss function, the second loss function and the third loss function are mutually distinguished.

10. The image detection model training method of claim 4, wherein, The feature comparison module is configured to: obtain a reference feature subgraph from the reference feature map based on the detection region, obtain a detection feature subgraph from the detection feature map based on the detection region, and input the reference feature subgraph and the detection feature subgraph to obtain a target region. The feature comparison module is configured to: obtain a reference feature subgraph from the reference feature map based on the detection region, obtain a detection feature subgraph from the detection feature map based on the detection region, and input the reference feature subgraph and the detection feature subgraph to obtain a target region. The feature comparison module is configured to: obtain a reference feature subgraph from the reference feature map based on the detection region, obtain a detection feature subgraph from the detection feature map based on the detection region, and input the reference feature subgraph and the detection feature subgraph to obtain a target region. The feature comparison module is configured to: obtain a reference feature subgraph from the reference feature map based on the detection region, obtain a detection feature subgraph from the detection feature map based on the detection region, and input the reference feature subgraph and the detection feature subgraph to obtain a target region.

11. The image detection model training method of claim 1, wherein, The feature extraction network comprises first and second convolution modules having the same structure, and parameters of the first and second convolution modules remain consistent during adjustment. The feature extraction network comprises first and second convolution modules having the same structure, and parameters of the first and second convolution modules remain consistent during adjustment. The feature extraction network comprises first and second convolution modules having the same structure, and parameters of the first and second convolution modules remain consistent during adjustment.

12. A method of differential detection, characterized by, The feature extraction network comprises first and second convolution modules having the same structure, and parameters of the first and second convolution modules remain consistent during adjustment. The method comprises: Obtaining a group of images to be detected, wherein the group of images to be detected comprises a reference reference image and an image to be detected.

13. An electronic device, comprising: Inputting the group of images to be detected into an image detection model to obtain a target region with a difference, wherein the image detection model is obtained after training based on the method of any one of claims 1-11. The method comprises:

14. A computer readable storage medium having stored thereon program data, wherein, A memory and a processor coupled to each other, wherein the memory stores program data, and the processor invokes the program data to execute the method of any one of claims 1-11 or 12. The program data is executed by the processor to implement the method of any one of claims 1-11 or 12. The program data is executed by the processor to implement the method of any one of claims 1-11 or 12.

Images