Image difference comparison method and apparatus

Abstract

This application discloses an image difference comparison method and apparatus, relating to the field of image comparison technology. The method includes: determining at least one pair of feature region images based on a first image, a second image to be compared, and a pre-created feature region recognition model; the feature region recognition model is trained based on multiple pre-acquired target images and a deep learning algorithm; determining multiple corner points and their coordinates from the at least one pair of feature region images, and performing geometric transformations on the first image and the second image based on the corner point coordinates; denoising the first image and the second image after the viewpoint transformation to obtain denoised first and second images; and performing pixel comparison and difference region contour recognition on the denoised first and second images to obtain the difference regions between the first image and the second image. This application has a wider range of applicable scenarios and higher accuracy in difference recognition.

Description

Technical Field

[0001] This application relates to the field of image comparison technology, specifically to an image difference comparison method and apparatus. Background Technology

[0002] In transactions such as international documentation, clients typically submit paper documents, which are then scanned into images and submitted to experts for review. However, these documents may contain inaccuracies requiring revision, leading to multiple submissions of the same image. Currently, a full review of each image is necessary, resulting in repetitive manual labor and low efficiency. Therefore, if image comparison could identify discrepancies between images, allowing for focused review of those discrepancies, review efficiency would be significantly improved.

[0003] Existing image comparison techniques can be mainly divided into two categories: one compares individual pixels, and the other compares the overall similarity of the images. Pixel-level comparison methods typically involve binarizing the images first, then comparing each pixel of the two images one by one. If they are the same, they are considered to be identical; otherwise, the pixel is considered the difference. While this method can find differences between different images, it has high requirements for the original images; both images need to be the same size and resolution. Even a slight deviation will result in unsatisfactory comparison results. This approach is generally suitable for comparing modified images with the original image, such as comparing an annotated image with the original image to find the annotated portion.

[0004] Image similarity comparison methods based on overall image characteristics mainly include MD5, histograms, PSNR, SSIM, and deep learning. The core of these methods is to calculate a certain feature of the image and then compare these features to calculate the similarity between the original images. While these methods have high generalization ability, they can only calculate the overall similarity of the image and cannot find specific differences. They also carry a certain risk of false positives and are not applicable to certain fields with high security requirements.

[0005] It is evident that existing image comparison methods suffer from poor applicability and low accuracy. Summary of the Invention

[0006] To address the problems existing in the prior art, in a first aspect, this application provides an image difference comparison method, comprising:

[0007] Based on the first image to be compared, the second image, and a pre-created feature region recognition model, at least one pair of feature region images are determined. Each pair of feature region images includes a first feature region in the first image and a second feature region with the same content in the second image. The feature region recognition model is trained based on multiple pre-acquired target images and a deep learning algorithm.

[0008] Multiple corner points and their coordinates are determined from the at least one pair of feature region images, and geometric transformations are performed on the first image and / or the second image to be compared based on the coordinates of the corner points.

[0009] Denoising is performed on the first and second images after the viewpoint transformation to obtain the denoised first and second images.

[0010] Pixel comparison and difference region contour recognition are performed on the denoised first and second images to obtain the difference regions between the first and second images.

[0011] In one embodiment, determining at least one pair of feature region images based on the first image to be compared, the second image, and a pre-created feature region recognition model includes:

[0012] Based on the first image and the feature region recognition model, at least one first feature region in the first image and its corresponding content recognition result are determined;

[0013] Based on the second image and the feature region recognition model, at least one second feature region in the second image and its corresponding content recognition result are determined;

[0014] Based on the content recognition results, the first feature region and the second feature region are matched to obtain at least one pair of feature region images.

[0015] In one embodiment, the step of training the feature region recognition model includes:

[0016] Acquire multiple sample images containing preset feature regions, and content identifiers corresponding to the feature regions in the sample images;

[0017] The pre-created deep learning model is trained based on the sample images and their corresponding content identifiers to obtain the feature region recognition model.

[0018] In one embodiment, determining multiple corner points and their coordinates from the at least one pair of feature region images, and performing geometric transformations on the first image and / or the second image to be compared based on the coordinates of the corner points, includes:

[0019] Multiple first corner points and their first coordinates in the first image are determined from the first feature region of each pair of feature region images, and multiple second corner points and their second coordinates in the second image are determined from the second feature region of each pair of feature region images.

[0020] For each pair of feature region images, the first corner point and the second corner point are matched according to the first coordinate and the second coordinate to obtain multiple sets of anchor points;

[0021] Construct a geometric transformation matrix based on the multiple sets of anchor points and their coordinates;

[0022] Perform a geometric transformation on the first image or the second image according to the geometric transformation matrix to obtain a first image and a second image with the same viewpoint.

[0023] In one embodiment, the step of denoising the first image and the second image after the viewpoint transformation to obtain denoised first and second images includes:

[0024] Smooth and blur the first and second images after the viewpoint transformation;

[0025] Perform grayscale conversion and binarization on the first and second images after smoothing and blurring;

[0026] Opening operations are performed on the first and second images after binarization to obtain the first and second images after denoising.

[0027] In one embodiment, the step of performing pixel comparison and difference region contour recognition on the denoised first image and second image to obtain the difference region between the first image and the second image includes:

[0028] Erosion and dilation transformations are applied to the denoised first and second images respectively to obtain a first eroded image, a first dilated image, a second eroded image, and a second dilated image;

[0029] The first difference pixel point between the first image and the second image is determined based on the color of the corresponding pixel points in the first erosion image and the second dilation image;

[0030] The second difference pixel point between the second image and the first image is determined based on the color of the corresponding pixel point in the second erosion image and the first dilation image;

[0031] The difference regions in the first image and the difference regions in the second image are determined based on the first difference pixel, the second difference pixel, and the geometric transformation matrix.

[0032] In one embodiment, determining the first difference pixel point between the first image and the second image based on the color of corresponding pixels in the first erosion image and the second dilation image includes:

[0033] Determine whether the first pixel in the first eroded image is black; if yes, then determine that the first pixel is not a difference pixel; if no, determine whether the second pixel in the second dilated image corresponding to the first pixel is white; if yes, then the first pixel is not a difference pixel; if no, then determine that the first pixel is a difference pixel.

[0034] The step of determining the second difference pixel point between the second image and the first image based on the color of corresponding pixels in the second erosion image and the first dilation image includes:

[0035] Determine whether the second pixel in the second eroded image is black; if yes, then determine that the second pixel is not a difference pixel; if no, determine whether the first pixel in the first dilated image corresponding to the second pixel is white; if yes, then the second pixel is not a difference pixel; if no, then determine that the second pixel is a difference pixel.

[0036] In one embodiment, determining the difference region in the first image and the difference region in the second image based on the first difference pixel, the second difference pixel, and the geometric transformation matrix includes:

[0037] The contours of the first difference pixel in the denoised first image are identified to obtain the first contour coordinates; and the contours of the second difference pixel in the denoised second image are identified to obtain the second contour coordinates.

[0038] Based on the geometric transformation matrix, a geometric transformation is performed on the first contour coordinates or the second contour coordinates to obtain the difference regions in the first image and the difference regions in the second image.

[0039] Secondly, this application provides an image difference comparison device, comprising:

[0040] The feature region determination module is used to determine at least one pair of feature region images based on the first image to be compared, the second image, and a pre-created feature region recognition model. Each pair of feature region images includes a first feature region in the first image and a second feature region with the same content in the second image. The feature region recognition model is trained based on multiple pre-acquired target images and a deep learning algorithm.

[0041] A geometric transformation module is used to determine multiple corner points and the coordinates of each corner point from the at least one pair of feature region images, and to perform geometric transformations on the first image and / or the second image to be compared based on the coordinates of the corner points;

[0042] The denoising module is used to denoise the first image and the second image after the viewpoint transformation, respectively, to obtain the denoised first image and the second image.

[0043] The difference recognition module is used to compare pixels and identify the contours of the difference regions in the first and second images after denoising, so as to obtain the difference regions between the first and second images.

[0044] In one embodiment, the feature region determination module includes:

[0045] A feature region recognition unit is configured to determine at least one first feature region in the first image and its corresponding content recognition result based on the first image and the feature region recognition model; and to determine at least one second feature region in the second image and its corresponding content recognition result based on the second image and the feature region recognition model.

[0046] The feature region matching unit is used to match the first feature region and the second feature region according to the content recognition result to obtain at least one pair of feature region images.

[0047] In one embodiment, the image difference comparison device further includes:

[0048] The sample acquisition module is used to acquire multiple sample images containing preset feature regions, as well as the content identifiers corresponding to the feature regions in the sample images.

[0049] The model training module is used to train a pre-created deep learning model based on the sample image and the corresponding content identifier to obtain the feature region recognition model.

[0050] In one embodiment, the geometric transformation module includes:

[0051] The corner detection unit is used to determine multiple first corner points and the first coordinates of the first corner points in the first image from the first feature region of each pair of feature region images, and to determine multiple second corner points and the second coordinates of the second corner points in the second image from the second feature region of each pair of feature region images.

[0052] The corner matching unit is used to match the first corner and the second corner based on the first coordinate and the second coordinate for each pair of feature region images, so as to obtain multiple sets of anchor points;

[0053] A geometric transformation matrix determination unit is used to construct a geometric transformation matrix based on the multiple sets of anchor points and their coordinates.

[0054] A geometric transformation unit is used to perform geometric transformation on the first image or the second image according to the geometric transformation matrix to obtain a first image and a second image with the same viewpoint.

[0055] In one embodiment, the noise reduction module includes:

[0056] The blur processing unit is used to perform smooth blur processing on the first and second images after the viewpoint transformation;

[0057] The image conversion unit is used to perform grayscale conversion and binarization conversion on the first and second images after smoothing and blurring processing.

[0058] The opening operation processing unit is used to perform opening operation processing on the first and second images after binarization to obtain the first and second images after denoising.

[0059] In one embodiment, the difference recognition module includes:

[0060] The erosion and dilation conversion unit is used to perform erosion and dilation transformations on the denoised first image and the second image respectively to obtain a first eroded image, a first dilated image, a second eroded image, and a second dilated image;

[0061] The difference pixel point determination unit is configured to determine a first difference pixel point between the first image and the second image based on the color of corresponding pixels in the first eroded image and the second dilated image; and to determine a second difference pixel point between the second image and the first image based on the color of corresponding pixels in the second eroded image and the first dilated image.

[0062] The difference region determination unit is used to determine the difference region in the first image and the difference region in the second image based on the first difference pixel, the second difference pixel, and the geometric transformation matrix.

[0063] In one embodiment, the difference pixel determination unit is specifically used for:

[0064] Determine whether the first pixel in the first eroded image is black; if yes, determine that the first pixel is not a difference pixel; if no, determine whether the second pixel in the second dilated image corresponding to the first pixel is white; if yes, determine that the first pixel is not a difference pixel; if no, determine that the first pixel is a difference pixel; and

[0065] Determine whether the second pixel in the second eroded image is black; if yes, then determine that the second pixel is not a difference pixel; if no, determine whether the first pixel in the first dilated image corresponding to the second pixel is white; if yes, then the second pixel is not a difference pixel; if no, then determine that the second pixel is a difference pixel.

[0066] In one embodiment, the difference region determination unit is specifically used for:

[0067] The contours of the first difference pixel in the denoised first image are identified to obtain the first contour coordinates; and the contours of the second difference pixel in the denoised second image are identified to obtain the second contour coordinates.

[0068] Based on the geometric transformation matrix, a geometric transformation is performed on the first contour coordinates or the second contour coordinates to obtain the difference regions in the first image and the difference regions in the second image.

[0069] Thirdly, this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the image difference comparison methods provided in this application.

[0070] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the image difference comparison methods provided in this application.

[0071] Fifthly, this application provides a computer program product, which includes a computer program that, when executed by a processor, implements any of the image difference comparison methods provided in this application.

[0072] The image difference comparison method and apparatus of this application can find and label the differences between different images. Compared with pixel comparison methods, this application can handle images with a certain degree of distortion, such as two images obtained by scanning the same paper document twice, which have certain spatial differences such as perspective, rotation and translation, and has a wider range of applicable scenarios. Compared with image similarity methods, this application can find the specific differences between different images, not just the overall similarity, and the accuracy of difference recognition is improved. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0074] Figure 1 This is a schematic diagram of the image difference comparison method provided in this application.

[0075] Figure 2 A schematic diagram illustrating the steps for determining feature region image pairs provided in this application.

[0076] Figure 3 A schematic diagram illustrating the steps of training the feature region recognition model provided in this application.

[0077] Figure 4 A schematic diagram illustrating the steps of image geometric transformation provided in this application.

[0078] Figure 5 A schematic diagram illustrating the steps of image denoising provided in this application.

[0079] Figure 6 A schematic diagram illustrating the steps for determining image difference regions provided in this application.

[0080] Figure 7 This is a schematic diagram illustrating the steps for identifying differential pixels provided in this application.

[0081] Figure 8 This is a schematic diagram illustrating another step in determining image difference regions provided in this application.

[0082] Figure 9 A schematic diagram of the feature region provided in this application.

[0083] Figure 10 A schematic diagram of corner points in the example image provided for this application.

[0084] Figure 11 A schematic diagram of anchor points in the example image provided for this application.

[0085] Figure 12 This is a schematic diagram of image geometric transformation based on anchor points provided in this application.

[0086] Figure 13 A schematic diagram of Gaussian blur processing provided in this application.

[0087] Figure 14 A schematic diagram of the opening operation provided in this application.

[0088] Figure 15 This is a schematic diagram illustrating the difference between the erosion image and the dilation image provided in this application.

[0089] Figure 16 This application provides a schematic diagram of an image difference comparison device.

[0090] Figure 17 Another schematic diagram of the image difference comparison device provided in this application.

[0091] Figure 18 Another schematic diagram of the image difference comparison device provided in this application.

[0092] Figure 19 Another schematic diagram of the image difference comparison device provided in this application.

[0093] Figure 20 Another schematic diagram of the image difference comparison device provided in this application.

[0094] Figure 21 Another schematic diagram of the image difference comparison device provided in this application.

[0095] Figure 22 A schematic diagram of a computer device provided in this application. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0097] It should be noted that the image difference comparison method and apparatus of this application can be used in the field of image comparison technology, or in any field other than the field of image comparison technology. This application does not limit the application field of the image difference comparison method and apparatus.

[0098] The acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of laws and regulations.

[0099] All user information in this application embodiment was obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been authorized and agreed upon by the customer.

[0100] like Figure 1 As shown, this application provides an image difference comparison method, which includes the following steps S101 to S104:

[0101] Step S101: Determine at least one pair of feature region images based on the first image to be compared, the second image, and the pre-created feature region recognition model.

[0102] The first and second images in this application can be obtained by scanning the same paper document twice, i.e., images of the same thing, or they can be two images obtained at different times in the same location, or they can be two different images. The image difference comparison method in this application has no requirements on the images being compared, their resolution, viewing angle, degree of distortion, etc., and can be applied to various image comparison scenarios.

[0103] The feature region recognition model is trained using multiple pre-collected target images and deep learning algorithms. A well-trained model can automatically identify feature regions in an image, the content within those regions, and their coordinates in the original image. For example, a feature region might be a closed quadrilateral, and its coordinates in the original image might be the coordinates of its four vertices. A feature region refers to a region in an image containing unique content, including but not limited to trademarks, specific font styles, and signatures. A feature region recognition model can be trained to identify specific content based on actual needs. For example… Figure 9 The area marked in the image is the trademark region (box 1), which is a feature region in the image.

[0104] A feature region recognition model is used to identify feature regions in the first and second images to be compared, respectively. This identifies M feature regions in the first image and N feature regions in the second image, where M and N are both positive integers. Each pair of feature region images in this step consists of a first feature region in the first image and a second feature region in the second image, and the content recognition results (determined by the feature region recognition model) of the first and second feature regions in each pair of feature region images are identical.

[0105] Step S102: Determine multiple corner points and the coordinates of each corner point from the at least one pair of feature region images, and perform geometric transformation on the first image and / or the second image based on the coordinates of the corner points.

[0106] Assuming that K pairs of feature region images are matched in step S101, 1≤K≤min{M,N}, where K, M, and N are all positive integers, then step S102 obtains the corner points and coordinates of 2K feature regions in these K pairs of feature region images, and then performs geometric transformations on the first image and / or the second image based on some or all of the corner points and their coordinates. Here, the coordinates of the corner points refer to the coordinates of each corner point in the corresponding original image (first image or second image).

[0107] This step can be performed on the second image only, using the first image as the reference; or on the first image, using the second image as the reference; or a preset transformation reference (including image size, angle, etc.) can be used to perform geometric transformations on the first and second images respectively.

[0108] The geometric transformations in this step include, but are not limited to, scaling, translation, rotation, affine transformation, and perspective transformation. Their purpose is to adjust the viewpoints of the first and second images to be compared to be the same or similar, so that most of the content of the first and second images to be compared can overlap as much as possible, which facilitates subsequent comparison.

[0109] Step S103: Denoise the first image and the second image after the viewpoint transformation to obtain the denoised first image and the second image.

[0110] This step is a preprocessing step before the formal comparison step, designed to remove obvious noise and improve the accuracy of subsequent image difference recognition. The denoising methods used in this step include, but are not limited to, smoothing blurring, binarization, and opening operations.

[0111] Step S104: Perform pixel comparison and difference region contour recognition on the denoised first image and second image to obtain the difference region between the first image and the second image.

[0112] The corresponding pixels in the denoised first and second images are compared sequentially. When a preset difference condition is met, the pixel at that location is marked as a difference pixel. The region formed by these difference pixels is the difference region. It is understood that there can be multiple difference regions in the first and second images to be compared.

[0113] This application employs geometric transformations on the first and second images to be compared, based on feature region recognition and corner detection, to adjust the viewing angles of the two images to be essentially consistent. Then, it performs pixel-by-pixel comparison to determine whether each pixel is a difference pixel, and finally identifies the difference regions based on these difference pixels. This application can identify specific difference regions without requiring special quality from the images to be compared, making it widely applicable to various image comparison scenarios.

[0114] In one embodiment, such as Figure 2 As shown, step S101, determining at least one pair of feature region images based on the first image to be compared, the second image, and a pre-created feature region recognition model, includes:

[0115] Step S1011: Determine at least one first feature region in the first image and its corresponding content recognition result based on the first image and the feature region recognition model.

[0116] For example, the first image to be compared is used as input data and fed into a trained feature region recognition model. The feature region recognition model analyzes and processes the first image, outputting all identifiable feature regions contained in the first image to be compared, such as the M regions mentioned in the previous example. To distinguish it from the second image to be compared, the feature regions identified from the first image to be compared are called the first feature regions, and the number of first feature regions is represented as M.

[0117] Step S1012: Determine at least one second feature region in the second image and its corresponding content recognition result based on the second image and the feature region recognition model.

[0118] Similar to step S1011, this step uses the second image to be compared as input data and inputs it into the trained feature region recognition model. The feature region recognition model analyzes and processes the second image and outputs all the identifiable second feature regions contained in the second image to be compared, such as N in the previous example.

[0119] The content recognition results corresponding to the first and second feature regions, such as signatures, trademarks, specific style 1, specific style 2, etc., are related to the samples used when training the feature region recognition model.

[0120] Step S1013: Match the first feature region and the second feature region according to the content recognition result to obtain at least one pair of feature region images.

[0121] For example, matching the first feature region where the content recognition result is a signature with the second feature region where the content recognition result is a signature yields a pair of feature region images; matching the first feature region where the content recognition result is a trademark with the second feature region where the content recognition result is a trademark yields another pair of feature region images. As mentioned earlier, since the feature region recognition model is trained using the unique content contained in the image in the actual comparison scenario, it can be understood that the content of the M first feature regions is non-repetitive, and the content of the N second feature regions is non-repetitive.

[0122] In one embodiment, such as Figure 3 As shown, the image difference comparison method of this application further includes the step of training the feature region recognition model:

[0123] Step S201: Obtain multiple sample images containing preset feature regions, and content identifiers corresponding to the feature regions in the sample images.

[0124] Sample images refer to images that are frequently compared in actual comparison scenarios. Preset feature regions refer to content that is unique in each sample image but common across all sample images, and the regions located within each sample image. Sample images can be pre-acquired by relevant personnel, feature regions can be pre-determined by relevant personnel, and the preset feature regions and their content in the sample images can be pre-identified by relevant personnel.

[0125] Each sample image and the location and content of its predefined feature regions can be understood as a training sample, and multiple training samples constitute the training dataset.

[0126] Step S202: Train the pre-created deep learning model based on the sample image and the corresponding content identifier to obtain the feature region recognition model.

[0127] For example, this application selects deep learning object detection techniques such as RCNN, SSD, and YOLO to build a deep learning model. Then, it trains the deep learning model using the training dataset in step S201 until the requirements are met, thus obtaining the feature region recognition model. Steps S201 to S202 are executed once. During subsequent applications, training samples can be continuously accumulated to continuously update and train the feature region recognition model, thereby improving its accuracy and applicability.

[0128] In one embodiment, such as Figure 4 As shown, step S102 involves determining multiple corner points and their coordinates from the at least one pair of feature region images, and performing geometric transformations on the first image and / or the second image to be compared based on the coordinates of the corner points, including steps S1021 to S1024:

[0129] Step S1021: Determine multiple first corner points and their first coordinates in the first image from the first feature region of each pair of feature region images, and determine multiple second corner points and their second coordinates in the second image from the second feature region of each pair of feature region images.

[0130] Corner points are inherent feature points in an image, which are suitable for tracking and easy to compare. Corner detection algorithms include, but are not limited to, Harris, Shi-Tomasi, SIFT, and SURF. Figure 10 This is a schematic diagram of the corner points in the example image provided in this application. Figure 10 The small dots marked 1, 2, 3, 4, 5, 6... are the corner points that have been found.

[0131] Taking a pair of feature region images as an example, we determine multiple corner points in the first feature region and their coordinates within the first feature region, as well as multiple corner points in the second feature region and their coordinates within the second feature region. For ease of distinction, we refer to the corner points in the first feature region as the first corner points, the coordinates of the first corner points in the first feature region as the first coordinates, and the corner points in the second feature region as the second corner points, and the coordinates of the second corner points in the second feature region as the second coordinates.

[0132] Since the coordinates of each first feature region in the first image to be compared and the coordinates of each second feature region in the second image to be compared have already been determined when performing feature region recognition based on the feature region recognition model, and this step further determines the coordinates of the first corner point in the first feature region and the coordinates of the second corner point in the second image, the coordinates of the first corner point in the first feature region can be transformed based on the coordinates of the first feature region in the first image to be compared, thus obtaining the coordinates of the first corner point in the first image to be compared; similarly, the coordinates of the second corner point in the second feature region can be transformed based on the coordinates of the second feature region in the second image to be compared, thus obtaining the coordinates of the second corner point in the second image.

[0133] The coordinate restoration and transformation in this step can be performed using a reference coordinate system transformation. In specific implementation, the coordinates of the first corner point in the first feature region are taken as the coordinates of the first corner point in the first coordinate system, and the coordinates of the first corner point in the first image to be compared are taken as the coordinates of the first corner point in the second coordinate system. The coordinates of the first feature region in the first image to be compared are regarded as the transformation relationship between the first coordinate system and the second coordinate system. Those skilled in the art, knowing the coordinates of the first corner point in the first coordinate system and the transformation relationship between the first coordinate system and the second coordinate system, can easily obtain the coordinates of the first corner point in the second coordinate system. The second corner point is similar, and this application does not impose any limitations on this.

[0134] Step S1022: For each pair of feature region images, match the first corner point and the second corner point according to the first coordinate and the second coordinate to obtain multiple sets of anchor points.

[0135] Taking a pair of feature region images as an example, firstly, a first corner point is selected from the first feature region. Then, the first coordinate of the selected first corner point is compared with the second coordinates of each second corner point in the corresponding second feature region. The second corner point closest to the first corner point is then matched to obtain a set of anchor points. Similar operations are performed on other first corner points in the first feature region to obtain multiple sets of anchor points. Figure 11 This is a schematic diagram of the anchor points in the example image provided in this application. Figure 11The two endpoints of the line connecting the two images are a pair of matched corner points, i.e., a set of anchor points.

[0136] Similarly, for the remaining feature region image pairs, performing the same operation as described above can also yield multiple sets of anchor points. This application requires obtaining at least three sets of anchor points from the first and second images to be compared.

[0137] Step S1023: Construct a geometric transformation matrix based on the multiple sets of anchor points and their coordinates.

[0138] As described above, each set of anchor points includes a first corner point and a second corner point in the first image to be compared. The geometric transformation matrix describes the transformation relationship between the first coordinates of the first corner point and the second coordinates of the second corner point. For example, the following relationship can be constructed:

[0139]

[0140] in, The second coordinate, As the first coordinate, Let Ys be the geometric transformation matrix. Here, Ys and Xs (1 ≤ s ≤ m) are the first coordinates of the first corner point and the second coordinates of the second corner point in the same set of anchor points.

[0141] The above describes the construction of a geometric transformation matrix from the first coordinate to the second coordinate. By applying this geometric transformation matrix to the first image, the viewpoint of the first image can be adjusted to be the same as that of the second image. Alternatively, a geometric transformation matrix from the second coordinate to the first coordinate can also be constructed. Similarly, by applying this geometric transformation matrix to the second image, the viewpoint of the second image can be adjusted to be the same as that of the first image. For ease of explanation, this application will subsequently use the example of adjusting the viewpoint of the first image to be the same as that of the second image.

[0142] Step S1024: Perform a geometric transformation on the first image or the second image according to the geometric transformation matrix to obtain a first image and a second image with the same viewpoint.

[0143] Based on the geometric transformation matrix, any image to be compared is selected and subjected to a geometric transformation so that the transformed image has a substantially consistent perspective with the other image to be compared, allowing the main content to largely overlap. The geometric transformations involved include, but are not limited to, scaling, translation, rotation, affine transformation, and perspective. Figure 12 This is a schematic diagram illustrating how the image on the right is obtained by performing an affine transformation on the image on the right using three sets of anchor points to change the image perspective.

[0144] As described above, when the geometric transformation matrix is ​​a geometric transformation matrix from the first coordinate to the second coordinate, performing a geometric transformation on the first image using this matrix can adjust the first image to have the same viewpoint as the second image. Similarly, when the geometric transformation matrix is ​​a geometric transformation matrix from the second coordinate to the first coordinate, performing a geometric transformation on the second image using this matrix can adjust the second image to have the same viewpoint as the first image. The principle is that an affine transformation is performed on the image using multiple sets of anchor points, thereby changing the image's viewpoint.

[0145] The above example only performs geometric transformation on one of the first or second images. In practical applications, a reference viewpoint parameter can be preset, and the first and second images can be geometrically transformed simultaneously to become the reference viewpoint. Compared to performing geometric transformation on only one image, performing geometric transformation on both images simultaneously is more complex, but the principle is similar and it is equally feasible.

[0146] In one embodiment, such as Figure 5 As shown, step S103 involves denoising the first and second images after the viewpoint transformation to obtain denoised first and second images, including:

[0147] Step S1031: Perform smoothing and blurring processing on the first and second images after the viewpoint transformation.

[0148] For example, to blur an image, techniques such as Gaussian blur, median blur, and bilateral filtering can be used. These are primarily used to remove high-frequency components from an image, such as noise and boundaries. Figure 13 The Blured image on the right is the result of Gaussian blurring of the Original image on the left.

[0149] Step S1032: Perform grayscale conversion and binarization conversion on the first and second images after smoothing and blurring.

[0150] For example, an image that has undergone smoothing and blurring can first be converted into a grayscale image, and then into a binarized image. A binarized image is an image with only pure white and pure black pixels. Binarization techniques such as simple thresholding, adaptive thresholding, and Otsu's binarization can be used.

[0151] Step S1033: Perform an opening operation on the first and second images after binarization to obtain the first and second images after denoising.

[0152] An opening operation is performed on the binarized image to further remove noise. Figure 14 The left image in the image is the original binarized image, and the right image is the binarized image after the opening operation.

[0153] In one embodiment, such as Figure 6 As shown, step S104 involves comparing pixels and recognizing the contours of the difference regions in the denoised first and second images to obtain the difference regions between the first and second images, including the following steps S1041 to S1044:

[0154] Step S1041: Perform morphological transformations on the denoised first image and the second image respectively.

[0155] Morphological changes include erosion and dilation. Therefore, this step can obtain the first erosion image and the first dilation image corresponding to the first image to be compared, as well as the second erosion image and the second dilation image corresponding to the second image to be compared.

[0156] Step S1042: Determine the first difference pixel point between the first image and the second image based on the color of the corresponding pixel points in the first erosion image and the second dilation image.

[0157] Step S1043: Determine the second difference pixel point between the second image and the first image based on the color of the corresponding pixel point in the second erosion image and the first dilation image.

[0158] In steps S1042 and S1043 above, one image to be compared (after erosion) and another image to be compared (after dilation) are compared pixel by pixel according to a preset rule to find the difference between the eroded image and the dilated image. Through steps S1042 and S1043 above, the first difference pixel points between the first image and the second image, and the second difference pixel points between the second image and the first image can be obtained.

[0159] The preset rules in steps S1042 and S1043 are as follows: Figure 7 As shown:

[0160] Step (1): Select an erosion image and a dilation image;

[0161] Step (2): Determine whether the pixels of the eroded image are black; if yes, proceed to step (4); if no, proceed to step (3).

[0162] Step (3): Determine whether the pixel corresponding to the pixel in the dilated image is white; if yes, proceed to step (4); if no, proceed to step (5).

[0163] Step (4): Determine that the pixel is not a different pixel;

[0164] Step (5): Determine that the pixel is a difference pixel.

[0165] Steps (1) to (5) above are for judging the difference of a certain pixel in the erosion image and the dilation image. In practical applications, the above operations need to be performed sequentially for each pixel in the erosion image and the corresponding pixel in the dilation image to complete the identification of the difference pixel.

[0166] Based on steps (1) to (5) above, step S1042 specifically includes:

[0167] Determine whether the first pixel in the first eroded image is black; if yes, then determine that the first pixel is not a difference pixel; if no, determine whether the second pixel in the second dilated image corresponding to the first pixel is white; if yes, then the first pixel is not a difference pixel; if no, then determine that the first pixel is a difference pixel.

[0168] Based on steps (1) to (5) above, step S1043 specifically includes:

[0169] Determine whether the second pixel in the second eroded image is black; if yes, then determine that the second pixel is not a difference pixel; if no, determine whether the first pixel in the first dilated image corresponding to the second pixel is white; if yes, then the second pixel is not a difference pixel; if no, then determine that the second pixel is a difference pixel.

[0170] Step S1044: Determine the difference region in the first image and the difference region in the second image based on the first difference pixel, the second difference pixel, and the geometric transformation matrix.

[0171] Specifically, please see Figure 8 Step S1044 specifically includes the following steps:

[0172] Step S10441: Identify the contour of the first difference pixel in the denoised first image to obtain the first contour coordinates; and identify the contour of the second difference pixel in the denoised second image to obtain the second contour coordinates.

[0173] It is understood that the difference region does not consist of only one difference pixel, but is composed of multiple difference pixels. Therefore, this step identifies the graphic contour composed of each difference pixel and determines the contour coordinates. For example, this application can use a boundary rectangle to identify the contour.

[0174] Step S10442: Perform a geometric transformation on the first contour coordinates or the second contour coordinates based on the geometric transformation matrix to obtain the difference region in the first image and the difference region in the second image.

[0175] For example Figure 15 The arrow points to the area where the eroded image differs from the dilated image.

[0176] The contour coordinates in step S10441 are the contour coordinates of the difference pixels in the image after geometric transformation. Therefore, it is necessary to further perform geometric transformation on the first contour coordinates and the second contour coordinates to map them onto the original image, thereby obtaining the difference region in the original image.

[0177] When only the first image to be compared is geometrically transformed, the difference region in the first image can be obtained by transforming the first contour coordinates according to the corresponding geometric transformation matrix; the difference region in the second image to be compared is the same as the difference region in the denoised second image. When only the second image to be compared is geometrically transformed, the difference region in the second image to be compared is obtained by transforming the second contour coordinates according to the corresponding geometric transformation matrix; the difference region in the first image to be compared is the same as the difference region in the denoised first image.

[0178] The image difference comparison method of this application can find and label the differences between different images. Compared with pixel comparison methods, this application can handle images with a certain degree of distortion, such as two images obtained by scanning the same paper document twice, which have certain spatial differences such as perspective, rotation and translation, and has a wider range of applicable scenarios. Compared with image similarity methods, this application can find the specific differences between different images, not just the overall similarity, and the accuracy of difference recognition is improved.

[0179] Based on the same inventive concept, this application also provides an image difference comparison device, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Since the principle of the image difference comparison device in solving the problem is similar to that of the image difference comparison method, the implementation of the image difference comparison device can refer to the implementation of the image difference comparison method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0180] Secondly, this application provides an image difference comparison device, such as... Figure 16 As shown, the device includes:

[0181] The feature region determination module 301 is used to determine at least one pair of feature region images based on the first image to be compared, the second image, and a pre-created feature region recognition model. Each pair of feature region images includes a first feature region in the first image and a second feature region with the same content in the second image. The feature region recognition model is trained based on multiple pre-acquired target images and a deep learning algorithm.

[0182] The geometric transformation module 302 is used to determine multiple corner points and the coordinates of each corner point from the at least one pair of feature region images, and to perform geometric transformation on the first image and / or the second image to be compared based on the coordinates of the corner points.

[0183] The denoising module 303 is used to perform denoising processing on the first image and the second image after the viewpoint transformation, respectively, to obtain the denoised first image and the second image.

[0184] The difference recognition module 304 is used to perform pixel comparison and difference region contour recognition on the denoised first image and the second image to obtain the difference region between the first image and the second image.

[0185] In one embodiment, such as Figure 17 As shown, the feature region determination module 301 includes:

[0186] The feature region recognition unit 3011 is configured to determine at least one first feature region in the first image and its corresponding content recognition result based on the first image and the feature region recognition model; and to determine at least one second feature region in the second image and its corresponding content recognition result based on the second image and the feature region recognition model.

[0187] The feature region matching unit 3012 is used to match the first feature region and the second feature region according to the content recognition result to obtain at least one pair of feature region images.

[0188] In one embodiment, such as Figure 18 As shown, the image difference comparison device further includes:

[0189] The sample acquisition module 305 is used to acquire multiple sample images containing preset feature regions, and the content identifiers corresponding to the feature regions in the sample images.

[0190] The model training module 306 is used to train a pre-created deep learning model based on the sample image and the corresponding content identifier to obtain the feature region recognition model.

[0191] In one embodiment, such as Figure 19As shown, the geometric transformation module 302 includes:

[0192] The corner detection unit 3021 is used to determine a plurality of first corner points and the first coordinates of the first corner points in the first image from the first feature region of each pair of feature region images, and to determine a plurality of second corner points and the second coordinates of the second corner points in the second image from the second feature region of each pair of feature region images.

[0193] The corner matching unit 3022 is used to match the first corner and the second corner according to the first coordinate and the second coordinate for each pair of feature region images to obtain multiple sets of anchor points;

[0194] The geometric transformation matrix determination unit 3023 is used to construct a geometric transformation matrix based on the multiple sets of anchor points and their coordinates.

[0195] The geometric transformation unit 3024 is used to perform geometric transformation on the first image or the second image according to the geometric transformation matrix to obtain a first image and a second image with the same viewpoint.

[0196] In one embodiment, such as Figure 20 As shown, the noise reduction module 303 includes:

[0197] The blur processing unit 3031 is used to perform smooth blur processing on the first image and the second image after the viewpoint transformation;

[0198] Image conversion unit 3032 is used to perform grayscale conversion and binarization conversion on the first image and the second image after smoothing and blurring processing;

[0199] The opening operation processing unit 3033 is used to perform opening operation processing on the first image and the second image after binarization to obtain the first image and the second image after denoising.

[0200] In one embodiment, such as Figure 21 As shown, the difference recognition module 304 includes:

[0201] The erosion and dilation conversion unit 3041 is used to perform erosion and dilation transformations on the denoised first image and the second image respectively to obtain a first eroded image, a first dilated image, a second eroded image and a second dilated image;

[0202] The difference pixel point determination unit 3042 is used to determine a first difference pixel point between the first image and the second image based on the color of corresponding pixels in the first eroded image and the second dilated image; and to determine a second difference pixel point between the second image and the first image based on the color of corresponding pixels in the second eroded image and the first dilated image.

[0203] The difference region determination unit 3043 is used to determine the difference region in the first image and the difference region in the second image based on the first difference pixel, the second difference pixel, and the geometric transformation matrix.

[0204] In one embodiment, the difference pixel determination unit 3042 is specifically used for:

[0205] Determine whether the first pixel in the first eroded image is black; if yes, determine that the first pixel is not a difference pixel; if no, determine whether the second pixel in the second dilated image corresponding to the first pixel is white; if yes, determine that the first pixel is not a difference pixel; if no, determine that the first pixel is a difference pixel; and

[0206] Determine whether the second pixel in the second eroded image is black; if yes, then determine that the second pixel is not a difference pixel; if no, determine whether the first pixel in the first dilated image corresponding to the second pixel is white; if yes, then the second pixel is not a difference pixel; if no, then determine that the second pixel is a difference pixel.

[0207] In one embodiment, the difference region determination unit 3043 is specifically used for:

[0208] The contours of the first difference pixel in the denoised first image are identified to obtain the first contour coordinates; and the contours of the second difference pixel in the denoised second image are identified to obtain the second contour coordinates.

[0209] Based on the geometric transformation matrix, a geometric transformation is performed on the first contour coordinates or the second contour coordinates to obtain the difference regions in the first image and the difference regions in the second image.

[0210] The image difference comparison device of this application can find and label the differences between different images. Compared with pixel comparison methods, this application can handle images with a certain degree of distortion, such as two images obtained by scanning the same paper document twice, which have certain spatial differences such as perspective, rotation and translation, and has a wider range of applicable scenarios. Compared with image similarity methods, this application can find the specific differences between different images, not just the overall similarity, and the accuracy of difference recognition is improved.

[0211] In one embodiment, this application also provides a computer device, see [link to relevant documentation]. Figure 22 The electronic device 100 specifically includes:

[0212] The system includes a central processing unit (CPU) 110, a memory 120, a communication module 130, an input unit 140, an output unit 150, and a power supply 160.

[0213] The memory 120, communication module 130, input unit 140, output unit 150, and power supply 160 are all connected to the central processing unit 110. The memory 120 stores a computer program, which the central processing unit 110 can call. When the central processing unit 110 executes the computer program, it implements all the steps of the image difference comparison method in the above embodiments.

[0214] In one embodiment, the present application also provides a computer-readable storage medium for storing a computer program that can be executed by a processor. When executed by the processor, the computer program implements any of the image difference comparison methods provided by the present invention.

[0215] In one embodiment, the present invention also provides a computer program product, the computer program product including a computer program, which, when executed by a processor, implements any of the image difference comparison methods provided in the above embodiments.

[0216] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0217] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0218] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0219] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0220] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image difference comparison method, characterized in that, include: Based on the first image to be compared, the second image, and the pre-created feature region recognition model, at least one pair of feature region images are determined. Each pair of feature region images includes a first feature region in the first image and a second feature region in the second image that has the same content. The feature region recognition model is trained based on multiple pre-acquired target images and deep learning algorithms. Multiple first corner points and their first coordinates in the first image are determined from the first feature region of each pair of feature region images, and multiple second corner points and their second coordinates in the second image are determined from the second feature region of each pair of feature region images. For each pair of feature region images, the first corner point and the second corner point are matched according to the first coordinate and the second coordinate to obtain multiple sets of anchor points; A geometric transformation matrix is ​​constructed based on the multiple sets of anchor points and their coordinates; wherein, the geometric transformation matrix is ​​a matrix describing the transformation relationship between the first coordinate of the first corner point and the second coordinate of the second corner point; Perform a geometric transformation on the first image or the second image according to the geometric transformation matrix to obtain a first image and a second image with the same viewpoint; Denoising is performed on the first and second images after the viewpoint transformation to obtain the denoised first and second images. Erosion and dilation transformations are applied to the denoised first and second images respectively to obtain a first eroded image, a first dilated image, a second eroded image, and a second dilated image; Determine whether the first pixel of the first eroded image is black; if so, determine that the first pixel is not the first difference pixel. If not, determine whether the second pixel in the second dilated image corresponding to the first pixel is white; if yes, then the first pixel is not the first difference pixel; if no, then determine that the first pixel is the first difference pixel. The second difference pixel point between the second image and the first image is determined based on the color of the corresponding pixel point in the second erosion image and the first dilation image; The difference regions in the first image and the difference regions in the second image are determined based on the first difference pixel, the second difference pixel, and the geometric transformation matrix; the difference region is a region composed of the first difference pixel or the second difference pixel.

2. The image difference comparison method according to claim 1, characterized in that, The method for determining at least one pair of feature region images based on the first image to be compared, the second image, and a pre-created feature region recognition model includes: Based on the first image and the feature region recognition model, at least one first feature region in the first image and its corresponding content recognition result are determined; Based on the second image and the feature region recognition model, at least one second feature region in the second image and its corresponding content recognition result are determined; Based on the content recognition results, the first feature region and the second feature region are matched to obtain at least one pair of feature region images.

3. The image difference comparison method according to claim 2, characterized in that, The steps for training the feature region recognition model include: Acquire multiple sample images containing preset feature regions, and content identifiers corresponding to the feature regions in the sample images; The pre-created deep learning model is trained based on the sample images and their corresponding content identifiers to obtain the feature region recognition model.

4. The image difference comparison method according to claim 1, characterized in that, The step of denoising the first and second images after the viewpoint transformation to obtain denoised first and second images includes: Smooth and blur the first and second images after the viewpoint transformation; Perform grayscale conversion and binarization on the first and second images after smoothing and blurring; Opening operations are performed on the first and second images after binarization to obtain the first and second images after denoising.

5. The image difference comparison method according to claim 1, characterized in that, The step of determining the difference regions in the first image and the second image based on the first difference pixel, the second difference pixel, and the geometric transformation matrix includes: The contours of the first difference pixel in the denoised first image are identified to obtain the first contour coordinates; and the contours of the second difference pixel in the denoised second image are identified to obtain the second contour coordinates. Based on the geometric transformation matrix, a geometric transformation is performed on the first contour coordinates or the second contour coordinates to obtain the difference regions in the first image and the difference regions in the second image.

6. An image difference comparison device, characterized in that, include: The feature region determination module is used to determine at least one pair of feature region images based on the first image to be compared, the second image and a pre-created feature region recognition model. Each pair of feature region images includes a first feature region in the first image and a second feature region with the same content in the second image. The feature region recognition model is trained based on multiple pre-acquired target images and deep learning algorithms. The geometric transformation module is used to determine multiple first corner points and their first coordinates in the first image from the first feature region of each pair of feature region images, and to determine multiple second corner points and their second coordinates in the second image from the second feature region of each pair of feature region images; for each pair of feature region images, the first corner points and the second corner points are matched according to the first coordinates and the second coordinates to obtain multiple sets of anchor points; A geometric transformation matrix is ​​constructed based on the multiple sets of anchor points and their coordinates; wherein, the geometric transformation matrix is ​​a matrix describing the transformation relationship between the first coordinate of the first corner point and the second coordinate of the second corner point; the first image or the second image is geometrically transformed based on the geometric transformation matrix to obtain a first image and a second image with the same viewpoint; The denoising module is used to denoise the first image and the second image after the viewpoint transformation, respectively, to obtain the denoised first image and the second image. The difference recognition module includes: The erosion and dilation conversion unit is used to perform erosion and dilation transformations on the denoised first image and the second image respectively to obtain a first eroded image, a first dilated image, a second eroded image, and a second dilated image; The difference pixel determination unit is configured to: determine whether a first pixel in the first eroded image is black; if yes, determine that the first pixel is not a first difference pixel; if no, determine whether a second pixel in the second dilated image corresponding to the first pixel is white; if yes, determine that the first pixel is not a first difference pixel; if no, determine that the first pixel is a first difference pixel; and determine the second difference pixel in the second image compared to the first image based on the colors of corresponding pixels in the second eroded image and the first dilated image. The difference region determination unit is used to determine the difference region in the first image and the difference region in the second image based on the first difference pixel, the second difference pixel, and the geometric transformation matrix; the difference region is a region composed of the first difference pixel or the second difference pixel.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image difference comparison method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the image difference comparison method according to any one of claims 1 to 5.

Images