Image comparison method, system, electronic device and computer readable storage medium

Abstract

The embodiment of the application relates to the technical field of image processing, and discloses an image comparison method, system, electronic equipment and computer readable storage medium, the method comprising: respectively pre-processing and adaptively binarizing a test image and a template image corresponding to the test image to obtain a first image and a second image; performing target detection on the first image, respectively removing a detected target region in the first image and a region in the second image with the same position as the target region to obtain a third image and a fourth image; extracting feature points in the third image and the fourth image and performing feature point matching to determine matched feature points; performing global binarization on the pre-processed test image and the template image and performing difference processing to obtain a difference image; and obtaining an image comparison result based on the matched feature points and the difference image, so that different elements between images are efficiently identified, elements that do not need to be focused on are excluded, and the workload of a tester during rechecking is reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image comparison method, system, electronic device, and computer-readable storage medium. Background Technology

[0002] Product development is always accompanied by continuous version iterations. There will be certain differences between the new and old versions. Some differences are necessary for product upgrades, while others are bugs caused by version iterations. Testers need to compare a large number of product interface screenshots to determine whether the differences between the new and old versions are abnormal.

[0003] Manual comparison and annotation are time-consuming and labor-intensive, and testers may experience visual fatigue from prolonged work, leading to misjudgments. Therefore, a number of automated image comparison tools based on technologies such as image histograms, template matching, and perceptual hashing have emerged. These tools can annotate all differences between two images. However, for interfaces after product iterations, these differences include many unnecessary details (such as elements that naturally change over time), resulting in many worthless and erroneous detections, increasing the workload for testers during review. Summary of the Invention

[0004] The purpose of this application is to provide an image comparison method, system, electronic device, and computer-readable storage medium that can efficiently identify different elements between images and exclude elements that do not need attention, greatly reducing the workload of testers during review.

[0005] To address the aforementioned technical problems, embodiments of this application provide an image comparison method, comprising the following steps: preprocessing and adaptive binarizing a test image and a template image corresponding to the test image to obtain a first image and a second image; wherein the test image is obtained by cropping the interface of the product to be tested; performing target detection on the first image, removing the detected target region in the first image and the region in the second image that is at the same position as the target region, to obtain a third image and a fourth image; extracting feature points in the third image and the fourth image and performing feature point matching to determine the matched feature points; performing global binarization and differential processing on the preprocessed test image and the preprocessed template image to obtain a differential image; and obtaining an image comparison result based on the matched feature points and the differential image.

[0006] Embodiments of this application also provide an image comparison system, including: a screenshot module, a template matching module, an algorithm server, and an output module. The algorithm server includes a preprocessing unit, an adaptive binarization unit, a target detection unit, a feature point matching unit, a difference processing unit, and a comparison unit. The screenshot module is used to capture the interface of the product to be tested to obtain a test image. The template matching module is used to determine a template image corresponding to the test image. The preprocessing unit is used to preprocess the test image and the template image respectively. The adaptive binarization unit is used to adaptively binarize the preprocessed test image and the preprocessed template image respectively to obtain a first image and a second image. The target detection unit is used to perform target detection on the first image, removing the detected target region in the first image and the region in the second image that is in the same position as the target region, respectively, to obtain a third image and a fourth image; the feature point matching unit is used to extract feature points in the third image and the fourth image and perform feature point matching to determine the matched feature points; the difference processing unit is used to perform global binarization and difference processing on the preprocessed test image and the preprocessed template image to obtain a difference image; the comparison unit is used to obtain an image comparison result based on the matched feature points and the difference image; the output module is used to output the image comparison result.

[0007] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image comparison method described above.

[0008] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image comparison method described above.

[0009] The image comparison method, system, electronic device, and computer-readable storage medium provided in the embodiments of this application obtain a test image by capturing the interface of the product under test during image comparison, find a template image corresponding to the test image, and then preprocess and adaptively binarize the test image and the template image corresponding to the test image to obtain a first image and a second image. Subsequently, target detection is performed on the first image, and the target regions detected in the first image and the regions in the second image that are in the same position as the target regions detected in the first image are removed to obtain a third image and a fourth image. Feature points in the third image and the fourth image are extracted and feature point matching is performed to determine the matched feature points. After preprocessing the test image and the template image, global binarization and differential processing are performed on the preprocessed test image and the preprocessed template image to obtain a differential image. Finally, based on the matched feature points and the differential image, the image comparison result is obtained. Considering that existing automated image comparison tools based on image histograms, template matching, and perceptual hashing algorithms can only simply select the differences between two images, which cannot meet the actual needs of product iteration, this application proposes an image comparison method based on adaptive binarization and feature point matching technology. It introduces the concept of target detection to exclude target regions (areas that do not need to be focused on during product iteration), effectively improving the speed and accuracy of image comparison and greatly reducing the workload of testers during review and analysis.

[0010] Furthermore, the preprocessing includes image normalization and grayscale conversion. The preprocessing and adaptive binarization of the test image and the corresponding template image to obtain the first image and the second image includes: converting the preprocessed test image and the preprocessed template image into an integral test image and an integral template image, respectively; determining the window size based on the width of the integral test image and establishing an adaptive binarization window; sequentially traversing the integral test image and the integral template image with the width of the adaptive binarization window as the step size, and determining the sub-threshold of the current adaptive binarization window based on the Gaussian mixture distribution of the histogram of the pixel values ​​of each pixel within the current adaptive binarization window; and reassigning the pixel values ​​of each pixel within the current adaptive binarization window based on the pixel values ​​of each pixel within the current adaptive binarization window and the sub-threshold to obtain the first image and the second image. By setting a sliding window for local adaptive binarization to achieve adaptive binarization of the entire image, processing speed and accuracy can be balanced, efficiently distinguishing between detection elements and background elements in the image, thereby effectively improving the accuracy of subsequent target detection.

[0011] Furthermore, the target detection includes target icon detection, and the detected target region includes target icon region. The target detection of the first image includes: inputting the first image into a pre-trained icon detection model for target icon detection, and obtaining the target icon region in the first image output by the icon detection model. For interfaces after product iterations, battery icons, signal icons, etc., may change due to different screenshot times. These only reflect the amount of battery power and the strength of the signal, which are not differences that testers need to focus on. Therefore, based on the pre-trained icon detection model, these icon regions can be effectively detected and removed, reducing the workload of testers during review and analysis.

[0012] In addition, the target detection also includes target field detection, and the detected target region also includes target field region. The target detection of the first image further includes: inputting the first image into a pre-trained text detection model for OCR recognition, obtaining each character in the first image output by the text detection model; grouping the characters into several fields according to a preset regional proximity relationship, and comparing each field with a preset vocabulary to determine the target field; and taking the region where the target field is located in the first image as the target field region. For interfaces after product iterations, fields representing time, date, and battery percentage may change due to different screenshot times. These only reflect the amount of battery power and the time of day, not differences that testers need to focus on. Therefore, based on the pre-trained text detection model, these field regions can be effectively detected and eliminated, reducing the workload of testers during review and analysis.

[0013] Furthermore, the step of globally binarizing and differentially processing the preprocessed test image and the preprocessed template image to obtain a difference image includes: globally binarizing the preprocessed test image and the preprocessed template image respectively to obtain a fifth image and a sixth image; differentially processing the fifth image and the sixth image to obtain the difference value corresponding to each pixel point; assigning the pixel value of the pixel point whose difference value is greater than a preset difference threshold to 1, and assigning the pixel value of the pixel point whose difference value is less than or equal to the preset difference threshold to 0, to obtain the difference image; the step of obtaining an image comparison result based on the matched feature points and the difference image includes: assigning the pixel value of the matched feature points and each pixel point in their neighborhood in the difference image to 0; obtaining the bounding rectangle of each region with a pixel value of 1 in the difference image, and integrating the neighboring bounding rectangles according to a preset clustering algorithm to obtain several bounding regions; and marking the several bounding regions on the test image to obtain the image comparison result. Preprocessing introduces interpolation noise into the image. Setting the pixel values ​​of the matched feature points and their neighbors in the difference image to 0 can eliminate the interpolation noise. Integrating the adjacent bounding rectangles can reduce the number of selected regions and avoid excessive detail that leads to complex data.

[0014] In addition, after determining the matching feature points, the method further includes: calculating a first ratio between the number of matching feature points and the total number of feature points; determining whether the first ratio is less than a first preset threshold; if the first ratio is less than the first preset threshold, directly outputting the image comparison result that the test image and the template image are completely different; if the first ratio is greater than or equal to the first preset threshold, obtaining the image comparison result based on the matching feature points and the difference image. If the proportion of matching feature points in the total number of feature points is too small, it indicates that the two images are completely different. This could be due to a large-scale failure in the iterated product or an incorrect template image selection. In this case, directly outputting the image comparison result that the two images are completely different avoids wasting computational resources.

[0015] Furthermore, the icon detection model includes: an input layer, several convolutional pooling layers, several batch normalization layers, a flattening layer, a fully connected layer, and an output layer. The input layer is connected to the first convolutional pooling layer, each convolutional pooling layer is followed by a batch normalization layer, the last batch normalization layer is followed by the flattening layer, and the flattening layer is followed sequentially by the fully connected layer and the output layer. The addition of several batch normalization layers in the icon detection model accelerates model convergence while ensuring that prediction accuracy does not decrease.

[0016] Furthermore, the text detection model includes: an input layer, several separable convolutional layers, several pooling layers, several normalization layers, a mapping sequence layer, a recurrent layer, a transcription layer, and an output layer. The input layer is connected to the first separable convolutional layer. Each pooling layer is connected to a separable convolutional layer before and after it. Each normalization layer (except the last one) is connected to a separable convolutional layer before and after it. The last normalization layer is preceded by a separable convolutional layer, followed by the mapping sequence layer. The mapping sequence layer is then sequentially connected to the recurrent layer, the transcription layer, and the output layer. By replacing the convolutional pooling layers with separable convolutional layers and converting the feature map into a feature vector sequence through the mapping sequence layer, the model parameters and size are significantly reduced, the computational load is greatly decreased, and the training and running speed of the text detection model is effectively improved. Attached Figure Description

[0017] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0018] Figure 1 This is a flowchart of an image comparison method provided in one embodiment of this application;

[0019] Figure 2 This is a flowchart of obtaining image comparison results based on matched feature points and difference images in one embodiment of this application;

[0020] Figure 3 This is a flowchart illustrating how, in one embodiment of this application, a test image and a template image corresponding to the test image are preprocessed and adaptively binarized to obtain a first image and a second image.

[0021] Figure 4 This is a schematic diagram of the model architecture of an icon detection model provided in one embodiment of this application;

[0022] Figure 5 This is a flowchart of target field detection for a first image in one embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the model architecture of a text detection model provided in one embodiment of this application;

[0024] Figure 7 This is a schematic diagram of an image comparison system provided in another embodiment of this application;

[0025] Figure 8 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0027] One embodiment of this application relates to an image comparison method applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the image comparison method of this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution.

[0028] The specific process of the image comparison method in this embodiment can be described as follows: Figure 1 As shown, it includes:

[0029] Step 101: Preprocess and adaptively binarize the test image and the template image corresponding to the test image to obtain the first image and the second image. The test image is obtained by cropping the interface of the product to be tested.

[0030] Specifically, when performing image comparison, the server first obtains the test image by capturing the interface of the product under test, then finds the template image corresponding to the test image, and then performs preprocessing and adaptive binarization on the test image and the template image corresponding to the test image respectively to obtain the first image and the second image. The first image is the test image after preprocessing and adaptive binarization, and the second image is the template image after preprocessing and adaptive binarization.

[0031] In some examples, after the server captures the interface of the product under test to obtain a test image, it can generate a test file based on the test image. The test file contains at least the interface name of the captured interface of the product under test. Based on the interface name in the test file, the server determines the template file that matches the test file from a preset template library. The template image corresponding to the test image can then be obtained from the template file. The preset template library can be set by those skilled in the art according to the actual needs of the product under test.

[0032] In some cases, the preprocessing performed by the server on the test image and the template image may include filtering and denoising, image normalization, and grayscale conversion.

[0033] In some examples, the server can adaptively binarize the preprocessed test image and the preprocessed template image according to a preset adaptive binarization algorithm to obtain the first image and the second image.

[0034] Step 102: Perform target detection on the first image, remove the target region detected in the first image and the region in the second image that is in the same position as the target region, to obtain the third image and the fourth image.

[0035] In a practical implementation, after obtaining the first and second images, the server can perform target detection on the first image to identify the target region. Then, it removes the detected target region from the first image. Simultaneously, the server also needs to identify regions in the second image that share the same location as the target region detected in the first image, and remove these regions from the second image, thus obtaining the third and fourth images. The third image is the first image after removing the target region, and the fourth image is the second image after removing the regions that share the same location as the target region. The target detected by the server can be set by those skilled in the art according to the actual needs of the product under test, such as target icons, target fields, etc.

[0036] Step 103: Extract feature points from the third and fourth images and perform feature point matching to determine the matched feature points.

[0037] In the specific implementation, after the server obtains the third and fourth images, it can perform feature point detection on the third and fourth images respectively to determine the feature points in the third image and the feature points in the fourth image. Then, it performs feature point matching on the feature points in the third image and the feature points in the fourth image to determine the matched feature points.

[0038] In some cases, the server can use the SIFT (Scale Invariant Feature Transform) matching algorithm or the AKAZE matching algorithm to perform nearest neighbor matching of feature points. Feature points with a feature vector distance ratio greater than 0.93 and an absolute coordinate error within 5 pixels are considered as matched feature points, forming a set of matched feature points.

[0039] Step 104: Globally binarize and perform difference processing on the preprocessed test image and the preprocessed template image to obtain the difference image.

[0040] In the specific implementation, after the server preprocesses the test image and the template image respectively, it can perform global binarization and difference processing on the preprocessed test image and the preprocessed template image to obtain the difference image. That is, first, the server performs global binarization on the preprocessed test image and the preprocessed template image to obtain the fifth image and the sixth image. The fifth image is the preprocessed test image after global binarization, and the sixth image is the preprocessed template image after global binarization. The server then performs difference processing on the fifth image and the sixth image, such as subtracting the pixel values ​​of the corresponding pixels in the sixth image from the pixel values ​​of each pixel in the fifth image, thereby obtaining the difference image.

[0041] In some examples, step 104 can be performed concurrently with step 101.

[0042] Step 105: Based on the matched feature points and the difference image, obtain the image comparison result.

[0043] In practice, the server can determine and label the locations of matching feature points in the difference image to obtain the image comparison results.

[0044] In some examples, the server performs global binarization on the preprocessed test image and the preprocessed template image respectively to obtain the fifth image and the sixth image. Then, it performs difference processing on the fifth and sixth images to obtain the difference value corresponding to each pixel. Pixels with difference values ​​greater than a preset difference threshold are assigned a value of 1, while pixels with difference values ​​less than or equal to the preset difference threshold are assigned a value of 0, resulting in a difference image. Based on the matched feature points and the difference image, the server obtains the image comparison result, which can be achieved through methods such as... Figure 2 The implementation of each sub-step shown includes:

[0045] Sub-step 1051: Assign the pixel value of the matched feature points and each pixel in their neighborhood to 0 in the difference image.

[0046] Specifically, after the server obtains the difference image and determines the matching feature points, it can set the pixel values ​​of the matching feature points and their neighbors in the difference image to 0. The matching feature points are the same positions in the image. The pixel values ​​of the neighbors of the matching feature points are also set to 0, which can effectively eliminate the interpolation noise introduced into the image during preprocessing.

[0047] In one example, the server can assign the pixel value of each pixel in the two neighborhoods of the matched feature point in the difference image to 0, that is, assign the pixel value of all 25 pixels in a 5×5 range centered on the matched feature point to 0.

[0048] Sub-step 1052: Obtain the bounding rectangle of each region with a pixel value of 1 in the difference image, and integrate the neighboring bounding rectangles according to the preset clustering algorithm to obtain several selected regions.

[0049] Sub-step 1053: Mark several selected regions on the test image to obtain the image comparison results.

[0050] In the specific implementation, pixels with a value of 1 in the difference image are considered distinct pixels. The server obtains the bounding rectangles of the regions with a pixel value of 1 in the difference image and integrates neighboring bounding rectangles according to a preset clustering algorithm to obtain several bounding regions. This reduces the number of bounding regions and avoids excessive data complexity due to overly detailed clustering. The bounding regions are the different areas in the two images that require the tester's attention. The server marks several bounding regions on the test image to obtain the image comparison results. The preset clustering algorithm can be selected and set by those skilled in the art according to actual needs.

[0051] In some cases, the default clustering algorithm can be the DBSACN density clustering algorithm.

[0052] In some examples, after obtaining the image comparison results, the server can acquire selection information of the bounding boxes in the image comparison results and generate an image comparison result report based on the selection information and the image comparison results. That is, after automatically performing image comparison and generating image comparison results, the server allows testers to review the results, supporting testers to manipulate the bounding boxes in the image comparison results, thereby further improving the accuracy of image comparison.

[0053] In some examples, after determining the matching feature points, the server can calculate a first ratio between the number of matching feature points and the total number of feature points. It then determines whether this first ratio is less than a first preset threshold. If the first ratio is less than the first preset threshold, the image comparison result is directly output as "the test image and the template image are completely different." If the first ratio is greater than or equal to the first preset threshold, an image comparison result is obtained based on the matching feature points and the difference image. Considering that if the proportion of matching feature points in the total number of feature points is too small, indicating that the two images are completely different, it could be due to a large-scale failure in the iterated product or an incorrect template image selection. In this case, directly outputting the image comparison result showing that the two images are completely different avoids wasting computational resources.

[0054] In this embodiment, a test image is obtained by capturing the interface of the product under test. A template image corresponding to the test image is found. Then, the test image and the template image corresponding to the test image are preprocessed and adaptively binarized to obtain a first image and a second image. Subsequently, target detection is performed on the first image. The target regions detected in the first image and the regions in the second image that are in the same position as the target regions detected in the first image are removed to obtain a third image and a fourth image. Feature points in the third image and the fourth image are extracted and feature point matching is performed to determine the matching feature points. After preprocessing the test image and the template image, global binarization and differential processing are performed on the preprocessed test image and the preprocessed template image to obtain a differential image. Finally, based on the matched feature points and the differential image, the image comparison result is obtained. Considering that existing automated image comparison tools based on image histograms, template matching, and perceptual hashing algorithms can only simply select the differences between two images, which cannot meet the actual needs of product iteration, this application proposes an image comparison method based on adaptive binarization and feature point matching technology. It introduces the concept of target detection to exclude target regions (areas that do not need to be focused on during product iteration), effectively improving the speed and accuracy of image comparison and greatly reducing the workload of testers during review and analysis.

[0055] In one embodiment, preprocessing includes image normalization and grayscale conversion. The server performs preprocessing and adaptive binarization on the test image and the template image corresponding to the test image, respectively, to obtain a first image and a second image. This can be achieved through methods such as... Figure 3 The steps shown are implemented as follows:

[0056] Step 201: Convert the preprocessed test image and the preprocessed template image into an integral test image and an integral template image.

[0057] Step 202: Determine the window size based on the width of the integral test image and establish an adaptive binarization window.

[0058] In the specific implementation, after the server performs preprocessing on the test image and template image, including image normalization and grayscale conversion, it can obtain the preprocessed test image and the preprocessed template image, namely the grayscale test image and the grayscale template image. The server converts the grayscale test image and the grayscale template image into the integral test image and the integral template image, respectively, and determines the window size according to the width of the integral test image, and establishes an adaptive binarization window.

[0059] In some examples, the size of the adaptive binarization window can be set to 1 / 10 of the width of the integral test image.

[0060] Step 203: Using the width of the adaptive binarization window as the step size, traverse the integral test image and the integral template image sequentially, and determine the sub-threshold of the current adaptive binarization window based on the Gaussian mixture distribution of the histogram of the pixel values ​​of each pixel in the current adaptive binarization window.

[0061] In the specific implementation, the integration test image and the integration template image are traversed sequentially with the width of the adaptive binarization window as the step size. That is, the traversal starts from the first pixel in the upper left corner of the integration test image and starts with the width of the adaptive binarization window as the step size. Then, the traversal starts from the first pixel in the upper left corner of the integration template image and starts with the width of the adaptive binarization window as the step size. The server determines the sub-threshold of the current adaptive binarization window based on the Gaussian mixture distribution of the histogram of the pixel values ​​of each pixel in the current adaptive binarization window. This sub-threshold is used to perform local adaptive binarization on each pixel in the current adaptive binarization window.

[0062] Step 204: Based on the pixel values ​​of each pixel in the current adaptive binarization window and the sub-threshold, reassign the pixel values ​​of each pixel in the current adaptive binarization window to obtain the first image and the second image.

[0063] In the specific implementation, the server can reassign the pixel values ​​of each pixel in the current adaptive binarization window based on the pixel values ​​of each pixel in the current adaptive binarization window and the sub-threshold. If the pixel value of a pixel is greater than the sub-threshold, the pixel value of the pixel is assigned to 1; if the pixel value of a pixel is less than or equal to the sub-threshold, the pixel value of the pixel is assigned to 0, thereby obtaining the first image and the second image.

[0064] In this embodiment, local adaptive binarization is performed by setting a sliding window to achieve adaptive binarization of the entire image. This approach balances processing speed and accuracy, efficiently distinguishing between detection elements and background elements in the image, thereby effectively improving the accuracy of subsequent target detection.

[0065] In some examples, object detection includes target icon detection. The detected target regions include target icon regions. The server can input the first image into a pre-trained icon detection model for target icon detection, and obtain the target icon regions in the first image output by the icon detection model. Target icons can include battery icons, signal icons, etc. Considering that for the interface after product iteration, battery icons, signal icons, etc., may change due to different screenshot times, they only reflect the amount of battery power and the strength of the signal, which are not differences that testers need to pay attention to. Therefore, based on the pre-trained icon detection model, these icon regions can be effectively detected and removed, reducing the workload of testers during review and analysis.

[0066] In some examples, the icon detection model includes: an input layer, several convolutional pooling layers, several batch normalization layers, a flattening layer, a fully connected layer, and an output layer. The input layer is connected to the first convolutional pooling layer, each convolutional pooling layer is followed by a batch normalization layer, the last batch normalization layer is followed by a flattening layer, and the flattening layer is followed by a fully connected layer and the output layer in sequence. The model architecture of the icon detection model can be as follows: Figure 4 As shown, the icon detection model adds several batch normalization layers, which can accelerate model convergence while ensuring that prediction accuracy does not decrease.

[0067] In some examples, object detection also includes object field detection; the detected object region also includes the object field region. The server performs object field detection on the first image, which can be achieved through methods such as... Figure 5 The steps shown are implemented as follows:

[0068] Step 301: Input the first image into the pre-trained text detection model for OCR recognition and obtain each character in the first image output by the text detection model.

[0069] Specifically, after obtaining the first image, the server can input the first image into a pre-trained text detection model for OCR recognition, and obtain each character in the first image output by the text detection model, that is, recognize all the text content in the first image.

[0070] In some examples, a text detection model includes: an input layer, several separable convolutional layers, several pooling layers, several normalization layers, a mapping sequence layer, a recurrent layer, a transcription layer, and an output layer. The input layer is connected to the first separable convolutional layer. Each pooling layer is connected to a separable convolutional layer before and after it. Each normalization layer, except for the last one, is connected to a separable convolutional layer before and after it. The last normalization layer is connected to a separable convolutional layer before it. The last normalization layer is connected to a mapping sequence layer after it. The recurrent layer, transcription layer, and output layer are then connected sequentially after the mapping sequence layer. The model architecture of a text detection model can be as follows: Figure 6 As shown in the figure. In this embodiment, the text detection model replaces the convolutional pooling layer with a separable convolutional layer, and converts the feature map into a feature vector sequence through a mapping sequence layer. This significantly reduces the model parameters and model size, greatly reduces the amount of computation, and effectively improves the training speed and running speed of the text detection model.

[0071] Step 302: According to the preset regional proximity relationship, each character is divided into several fields, and each field is compared with the preset dictionary to determine the target field.

[0072] Step 303: The region where the target field is located in the first image is taken as the target field region.

[0073] In the specific implementation, after the server recognizes all the text content in the first image, it can group the text into several fields according to a preset regional proximity relationship, and compare each field with a preset dictionary to determine the target field. The area where the target field is located in the first image is taken as the target field area. The preset regional proximity relationship can be selected and set by those skilled in the art according to actual needs. The target field may include fields representing time, date, battery percentage, etc.

[0074] In this embodiment, considering that for the interface after product iteration, the fields representing time, date, and battery percentage may change due to different screenshot times, they only reflect the amount of battery and the time, which are not differences that testers need to pay attention to. Therefore, based on the pre-trained text detection model, these field areas can be detected and removed very well, reducing the workload of testers when reviewing and analyzing.

[0075] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0076] Another embodiment of this application relates to an image comparison system. The implementation details of this image comparison system are described below. The following details are for ease of understanding and are not essential for implementing this solution. The schematic diagram of the image comparison system in this embodiment can be seen as follows: Figure 7 As shown, it includes a screenshot module 401, a template matching module 402, an algorithm server 403, and an output module 404. The algorithm server 403 includes a preprocessing unit 4031, an adaptive binarization unit 4032, a target detection unit 4033, a feature point matching unit 4034, a difference processing unit 4035, and a comparison unit 4036.

[0077] The screenshot module 401 is used to capture the interface of the product under test to obtain a test image.

[0078] The template matching module 402 is used to determine the template image corresponding to the test image.

[0079] The preprocessing unit 4031 is used to preprocess the test image and the template image respectively.

[0080] The adaptive binarization unit 4032 is used to perform adaptive binarization on the preprocessed test image and the preprocessed template image respectively to obtain the first image and the second image.

[0081] The target detection unit 4033 is used to perform target detection on the first image, remove the target region detected in the first image and the region in the second image that is in the same position as the target region, respectively, to obtain the third image and the fourth image.

[0082] The feature point matching unit 4034 is used to extract feature points from the third and fourth images and perform feature point matching to determine the matched feature points.

[0083] The differential processing unit 4035 is used to perform global binarization and differential processing on the preprocessed test image and the preprocessed template image to obtain a differential image.

[0084] The comparison unit 4036 is used to obtain image comparison results based on the matched feature points and the difference image.

[0085] The output module 4037 is used to output the image comparison results.

[0086] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0087] Another embodiment of this application relates to an electronic device, such as... Figure 8 As shown, it includes: at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; wherein the memory 502 stores instructions executable by the at least one processor 501, the instructions being executed by the at least one processor 501 to enable the at least one processor 501 to perform the image comparison methods in the above embodiments.

[0088] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0089] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0090] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0091] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0092] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A method of image comparison, characterized by, The method comprises the following steps: respectively pre-process and adaptive binarize a test image and a template image corresponding to the test image to obtain a first image and a second image; wherein the test image is obtained by intercepting an interface of a product to be tested; perform target detection on the first image, and respectively remove a target region detected in the first image and a region in the second image having the same position as the target region to obtain a third image and a fourth image; extract feature points in the third image and the fourth image and perform feature point matching to determine matched feature points; perform global binarization on the pre-processed test image and the pre-processed template image and perform difference processing to obtain a difference image; obtain an image comparison result based on the matched feature points and the difference image; the pre-processing comprises image normalization and grayscale, and the pre-processing and adaptive binarization of the test image and the template image corresponding to the test image to obtain the first image and the second image comprises the following steps: convert the pre-processed test image and the pre-processed template image into an integral test image and an integral template image; determine a window size according to the width of the integral test image, and establish an adaptive binarization window; traverse the integral test image and the integral template image in turn with the width of the adaptive binarization window as a step, and determine a sub-threshold value of a current adaptive binarization window according to the mixed Gaussian distribution of the histogram of the pixel values of each pixel point in the current adaptive binarization window; reassign the pixel values of each pixel point in the current adaptive binarization window according to the sub-threshold value and the pixel values of each pixel point in the current adaptive binarization window to obtain the first image and the second image; after the matched feature points are determined, the method further comprises the following steps: calculate a first ratio between the number of the matched feature points and the total number of feature points; determine whether the first ratio is less than a first preset threshold value; if the first ratio is less than the first preset threshold value, directly output an image comparison result that the test image is different from the template image as a whole; if the first ratio is greater than or equal to the first preset threshold value, obtain an image comparison result based on the matched feature points and the difference image; the global binarization on the pre-processed test image and the pre-processed template image and the difference processing to obtain the difference image comprises the following steps: respectively perform global binarization on the pre-processed test image and the pre-processed template image to obtain a fifth image and a sixth image; perform difference processing on the fifth image and the sixth image to obtain a difference value corresponding to each pixel point, assign a pixel value of 1 to a pixel point whose difference value is greater than a preset difference threshold value, and assign a pixel value of 0 to a pixel point whose difference value is less than or equal to the preset difference threshold value to obtain the difference image; the obtaining of the image comparison result based on the matched feature points and the difference image comprises the following steps: assign a pixel value of 0 to each pixel point in the neighborhood of the matched feature points in the difference image. An outer rectangle of a region in which each pixel value in the difference image is 1 is obtained, and adjacent outer rectangles are integrated according to a preset clustering algorithm to obtain a plurality of framed regions; The plurality of framed regions are marked on the test image to obtain an image comparison result.

2. The image comparison method of claim 1, wherein, The target detection includes target icon detection, and the detected target region includes a target icon region, and the target detection on the first image includes: The first image is input into a pre-trained icon detection model for target icon detection, and the target icon region in the first image output by the icon detection model is obtained.

3. The image comparison method of claim 2, wherein, The target detection also includes target field detection, and the detected target region also includes a target field region, and the target detection on the first image also includes: The first image is input into a pre-trained text detection model for OCR recognition, and each text in the first image output by the text detection model is obtained. According to a preset region proximity relationship, the texts are grouped into a plurality of fields, and each field is compared with a preset word library one by one to determine a target field. The region of the target field in the first image is taken as the target field region.

4. The image comparison method of claim 2, wherein, The icon detection model includes: an input layer, a plurality of convolution pooling layers, a plurality of batch normalization layers, a flattening layer, a full connection layer, and an output layer; The input layer is connected with the first convolution pooling layer, each convolution pooling layer is connected with a batch normalization layer, the last batch normalization layer is connected with the flattening layer, and the flattening layer is sequentially connected with the full connection layer and the output layer.

5. The image comparison method of claim 3, wherein, The text detection model includes: an input layer, a plurality of separable convolution layers, a plurality of pooling layers, a plurality of normalization layers, a mapping sequence layer, a cycle layer, a transcription layer, and an output layer; The input layer is connected with the first separable convolution layer, each pooling layer is connected with a separable convolution layer before and after it, each normalization layer is connected with a separable convolution layer before and after it except the last normalization layer, the last normalization layer is connected with a separable convolution layer before it, the last normalization layer is connected with the mapping sequence layer after it, and the mapping sequence layer is sequentially connected with the cycle layer, the transcription layer, and the output layer.

6. An image comparison system characterized by, It includes: A screenshot module, a template matching module, an algorithm server, and an output module, the algorithm server includes a preprocessing unit, an adaptive binarization unit, a target detection unit, a feature point matching unit, a difference processing unit, and a comparison unit; The screenshot module is used to capture the interface of the product to be tested to obtain a test image; The template matching module is used to determine a template image corresponding to the test image; The preprocessing unit is used to preprocess the test image and the template image respectively; The adaptive binarization unit is used to perform adaptive binarization on the preprocessed test image and the preprocessed template image respectively to obtain a first image and a second image; The target detection unit is configured to perform target detection on the first image, remove a target region detected in the first image and a region in the second image having a same position as the target region respectively, and obtain a third image and a fourth image; The feature point matching unit is configured to extract feature points in the third image and the fourth image and perform feature point matching, and determine matched feature points; The difference processing unit is configured to perform global binarization on the preprocessed test image and the preprocessed template image, and perform difference processing to obtain a difference image; The comparison unit is configured to obtain an image comparison result based on the matched feature points and the difference image; The output module is configured to output the image comparison result; The preprocessing includes image normalization and grayscale, and the preprocessing and adaptive binarization of the test image and the template image corresponding to the test image respectively to obtain the first image and the second image, including: Converting the preprocessed test image and the preprocessed template image into an integral test image and an integral template image; Determining a window size according to a width of the integral test image, and establishing an adaptive binarization window; Iterating through the integral test image and the integral template image with the width of the adaptive binarization window as a step, determining a sub-threshold value of a current adaptive binarization window according to a mixture Gaussian distribution of a histogram of pixel values of each pixel point in the current adaptive binarization window; Reassigning pixel values of each pixel point in the current adaptive binarization window according to the sub-threshold value and the pixel values of each pixel point in the current adaptive binarization window, to obtain the first image and the second image; After the matched feature points are determined, the method further includes: Calculating a first ratio between the number of the matched feature points and a total number of feature points; Determining whether the first ratio is less than a first preset threshold value; If the first ratio is less than the first preset threshold value, directly outputting an image comparison result that the test image is different from the template image as a whole; If the first ratio is greater than or equal to the first preset threshold value, obtaining an image comparison result based on the matched feature points and the difference image; The global binarization on the preprocessed test image and the preprocessed template image and the difference processing to obtain the difference image include: Performing global binarization on the preprocessed test image and the preprocessed template image respectively to obtain a fifth image and a sixth image; Performing difference processing on the fifth image and the sixth image to obtain a difference value corresponding to each pixel point, assigning a pixel value of a pixel point whose difference value is greater than a preset difference threshold value as 1, and assigning a pixel value of a pixel point whose difference value is less than or equal to the preset difference threshold value as 0, to obtain the difference image; The obtaining of the image comparison result based on the matched feature points and the difference image includes: Assigning pixel values of each pixel point in a neighborhood of the matched feature points in the difference image as 0; An outer rectangle of a region in which each pixel value in the difference image is 1 is obtained, and adjacent outer rectangles are integrated according to a preset clustering algorithm to obtain a plurality of framed regions; The plurality of framed regions are labeled on the test image to obtain an image comparison result.

7. An electronic device, comprising: The image comparison method comprises the following steps: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image comparison method according to any one of claims 1 to 5.

8. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the image comparison method according to any one of claims 1 to 5. The computer program is executed by the processor to implement the image comparison method according to any one of claims 1 to 5.

Images