Image processing method and related products

Abstract

This application discloses an image processing method and related products. The image processing method includes: acquiring a screenshot image, wherein the screenshot image is an image generated by taking a screenshot of a target page; performing black-and-white anomaly image recognition on the screenshot image; if the screenshot image is identified as a black-and-white anomaly image, then performing misjudgment detection on the screenshot image to obtain a misjudgment detection result; wherein the misjudgment detection result is a detection result determined based on one or more of a first misjudgment detection result and a second misjudgment detection result; determining the black-and-white screen detection result of the target page based on the misjudgment detection result, and outputting the black-and-white screen detection result of the target page. Using this application improves the detection efficiency and accuracy of black-and-white screen pages.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method and related products. Background Technology

[0002] In today's era of rapid information technology development, the Internet has become a vital channel for people to quickly obtain, publish, and disseminate information, playing a significant role in various aspects of the economy and daily life. Information dissemination on the Internet is primarily achieved through web pages, allowing users to acquire and share information by browsing them.

[0003] A black-and-white screen refers to an abnormal screen that appears during page loading (the page content is not displayed or only part of the page content is displayed). If a page displays a black-and-white screen, an emergency patch needs to be released to fix the anomaly. Therefore, detecting black-and-white screen anomalies in advance during the testing phase can ensure the quality of the application. Currently, manual inspection is used to detect whether the page under test is a black-and-white screen page. Manual inspection of black-and-white screen pages results in low detection efficiency and accuracy. Summary of the Invention

[0004] This application provides an image processing method and related products that can improve the detection efficiency and accuracy of black-and-white screen pages.

[0005] One embodiment of this application provides an image processing method, including:

[0006] Acquire a screenshot image, wherein the screenshot image is an image generated by taking a screenshot of the target page;

[0007] Perform black-and-white anomaly image recognition on the screenshot image;

[0008] If the screenshot image is identified as a black and white abnormal image, then a false positive detection is performed on the screenshot image to obtain a false positive detection result; wherein, the false positive detection result is a detection result determined based on one or more of a first false positive detection result and a second false positive detection result, the first false positive detection result is a detection result based on the existence judgment of the screenshot image loading flag, and the second false positive detection result is a detection result based on the matching judgment between the screenshot image and the set of false positive images; the screenshot image loading flag includes a loading progress bar and a loading icon;

[0009] Based on the false positive detection results, determine the black-and-white screen detection results of the target page, and output the black-and-white screen detection results of the target page.

[0010] One embodiment of this application provides an image processing apparatus, including:

[0011] The acquisition module is used to acquire a screenshot image, which is an image generated by taking a screenshot of the target page;

[0012] The recognition module is used to perform black-and-white abnormal image recognition on the screenshot image;

[0013] The detection module is used to perform misjudgment detection on the screenshot image if it is identified as a black and white abnormal image, and obtain a misjudgment detection result; wherein, the misjudgment detection result is a detection result determined based on one or more of a first misjudgment detection result and a second misjudgment detection result, the first misjudgment detection result is a detection result based on the existence judgment of the screenshot image loading flag, and the second misjudgment detection result is a detection result based on the matching judgment between the screenshot image and the set of misjudged images; the screenshot image loading flag includes a loading progress bar and a loading icon;

[0014] The output module is used to determine the black-and-white screen detection result of the target page based on the false positive detection result, and output the black-and-white screen detection result of the target page.

[0015] One aspect of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the methods described in the above embodiments.

[0016] One aspect of this application provides a computer storage medium storing a computer program, which includes program instructions. When the program instructions are executed by a processor, they perform the methods described in the above embodiments.

[0017] One embodiment of this application provides a computer program product, which includes a computer program / instruction stored in a computer-readable storage medium. When the computer program / instruction is executed by the processor of a computer device, it performs the methods described in the above embodiments.

[0018] This application enables terminal devices to automatically detect whether a target page is a black-and-white screen page, eliminating the need for manual intervention and improving the detection efficiency and accuracy of black-and-white screen pages. It converts black-and-white screen page recognition into image recognition, a simple and highly operable method that expands the application scope of black-and-white screen page recognition. Furthermore, when a screenshot image is identified as an abnormal black-and-white image, further misjudgment detection is required. The misjudgment detection results determine whether the page to be detected is a black-and-white screen page, refining the detection granularity and improving recognition accuracy. Moreover, the misjudgment detection results are determined based on the presence of loading markers in the screenshot image or the matching between the screenshot image and the set of misjudged images, avoiding misjudgments of loaded pages or pages with limited content, further ensuring the accuracy of black-and-white screen page recognition. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a system architecture diagram of image processing provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of an image processing flow provided in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of an image header area provided in an embodiment of this application;

[0023] Figures 4a-4b This is a schematic diagram of a segmented screenshot image provided in an embodiment of this application;

[0024] Figures 5a-5c This is a schematic diagram of image binarization provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of a detection result display platform provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of a false positive detection process provided in an embodiment of this application;

[0027] Figure 8 This is a schematic diagram illustrating the identification of a loading progress bar provided in an embodiment of this application;

[0028] Figure 9 This is a schematic diagram illustrating the identification of a loading icon provided in an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of a screenshot image provided in an embodiment of this application;

[0030] Figure 11 This is a schematic diagram of an image processing framework provided in an embodiment of this application;

[0031] Figure 12 This is a schematic diagram of an image preprocessing module provided in an embodiment of this application;

[0032] Figure 13 This is a schematic diagram of an image detection module provided in an embodiment of this application;

[0033] Figure 14 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0034] Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0036] Please see Figure 1This is a system architecture diagram for image processing provided in an embodiment of this application. Server 10f establishes a connection with a cluster of terminal devices via switch 10e and communication bus 10d. The cluster of terminal devices may include: terminal device 10a, terminal device 10b, ..., terminal device 10c. Taking terminal device 10a as an example, when terminal device 10a needs to test a page related to a certain step, it can take a screenshot of the page currently displayed on the screen (called the target page) when the test case executes to that step. The captured image is called a screenshot image. Terminal device 10a first performs black and white abnormal image recognition on the screenshot image to identify whether the screenshot image is a black screen image, a white screen image, or a normal image. If the screenshot image is detected as either a black screen or a white screen, the terminal device 10a performs a false positive detection on the screenshot image, obtaining a false positive detection result. This result is determined based on a first false positive detection result and / or a second false positive detection result. The first result is based on the existence of loading markers (including the loading progress bar and loading icon) in the screenshot image, while the second result is based on the matching between the screenshot image and the set of false positive images. The terminal device 10a can determine the black / white screen detection result of the target page based on the false positive detection result. This result can mark the target page as either a black screen, a white screen, or a normal page. The terminal device 10a then outputs the black / white screen detection result of the target page.

[0037] The step of determining the black-and-white screen detection result of the target page can also be performed by server 10f. That is, after the terminal device captures a screenshot, it sends the screenshot to server 10f. Server 10f performs black-and-white anomaly image recognition, false positive detection, and determines the black-and-white screen detection result of the target page from the screenshot. Server 10f then sends the black-and-white screen detection result of the target page to the terminal device.

[0038] Terminal equipment can also be referred to as terminal, user equipment (UE), access terminal, user unit, mobile device, user terminal, wireless communication equipment, user agent, or user device. Terminal equipment can be smart home appliances, handheld devices with wireless communication capabilities (such as smartphones and tablets), computing devices (such as personal computers (PCs), in-vehicle terminals, smart voice interaction devices, wearable devices, or other smart devices, but is not limited to these.

[0039] Server 10f can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0040] This application can be applied to scenarios involving the discovery of abnormal pages (e.g., discovering abnormal pages during testing or during online operation), primarily focusing on identifying whether a page is a black screen or a white screen. When applied in the field of software testing, this application can detect black screen or white screen pages early in the testing process, improving software quality; when applied in the field of software maintenance, it can detect black screen or white screen pages during operation, preventing situations where page functionality is unavailable.

[0041] Please see Figure 2 This is a schematic diagram of an image processing process provided in an embodiment of this application. Image processing may include the following steps:

[0042] Step S101: Obtain a screenshot image, which is an image generated by taking a screenshot of the target page.

[0043] Specifically, the target page can be the page displayed when the test reaches a preset step during automated testing of the target program using test cases. The preset step can be a step corresponding to a key business function set by the tester. For example, if the preset step is clicking button A, and button A's function is to display a loading animation, then the target page could be the animation display page. Another example is that the preset step is to long-press a blank area of ​​the page to test whether it will redirect from page A to page B; in this case, the target page could be both page A and page B. It's worth noting that when the test cases reach a preset step, since page loading may occur, the target page could also be a loading page.

[0044] The target page can also be the page displayed by the target program at any step during automated testing using random testing (also known as monkey testing). Since random testing explores the software's response to random operations, it is inherently uncertain. Therefore, taking a screenshot of the currently displayed page at any step expands the testing scope of black-and-white screen testing.

[0045] Both of the above scenarios are in the testing domain. When the target program is already online and running, the target page can be a page with a blocking duration exceeding a preset threshold, in order to discover whether the page blocking is due to a black and white screen or other reasons, and to ensure the normal operation of the target program.

[0046] Step S102: Perform black and white abnormal image recognition on the screenshot image.

[0047] Specifically, the terminal device performs black-and-white anomaly image recognition on the screenshot to identify whether the screenshot is a normal image or a black-and-white anomaly image. Black-and-white anomaly images can be further subdivided into black screen images and white screen images. There are two ways to determine whether a screenshot is a normal image or a black-and-white anomaly image; the first method will be explained below:

[0048] The terminal device first determines the header and footer areas of the screenshot image. The header area can be understood as the top area of ​​the screenshot image, and the footer area can be understood as the bottom area of ​​the screenshot image. The header area of ​​the screenshot image corresponds to the top bar of the target page's system status bar.

[0049] The terminal device crops the header and footer areas of the screenshot image, and the remaining image after cropping is called the screenshot body image. The terminal device then performs overall black-and-white image recognition on the screenshot body image to identify whether the screenshot body image is a normal image or an abnormal black-and-white image. If the screenshot body image is identified as an abnormal black-and-white image, it is determined that the screenshot image is an abnormal black-and-white image; if the screenshot body image is identified as a normal image, it is determined that the screenshot image is normal.

[0050] The reason for determining the recognition result of a screenshot image based on the overall recognition result of the screenshot's main text image is that the header and footer areas of the screenshot image are rich in content, increasing the overall image richness. However, these two parts are not the focus of black-and-white screen detection; they are irrelevant to the overall result. Therefore, it is necessary to remove the influence of the header and footer to avoid missed detections. For example, suppose the target page is a blank page, but because the header and footer are rich in content, the screenshot image is recognized as a normal image, leading to a missed detection.

[0051] Please see Figure 3 , Figure 3 This is a schematic diagram of an image header area provided in an embodiment of this application. Figure 3The image shown is a screenshot. As you can see, the header area of ​​the screenshot contains rich elements, directly increasing the image's richness. If the header area is not removed during black-and-white anomaly image recognition, the screenshot, which is actually a black-and-white anomaly, will be judged as a normal image, resulting in missed detections. Therefore, to avoid missed detections, the header and footer areas are removed first during black-and-white anomaly image recognition, and only the remaining area is used for overall black-and-white image recognition.

[0052] The second identification method is:

[0053] The terminal device divides the screenshot image into N unit screenshots based on the layout of the screenshot image, where N is a positive integer. Each content area within a screenshot can be a unit screenshot. For example, the area corresponding to a piece of text in a screenshot can be a unit screenshot, and the area corresponding to an icon in a screenshot can also be a unit screenshot.

[0054] The process of dividing a screenshot image into N unit screenshots is as follows: The terminal device first converts the screenshot image into a grayscale image, then uses image erosion to eliminate glitches and filter out some unimportant details; finally, image dilation is applied to enlarge the key content of the image. This operation of erosion followed by dilation is also known as opening, which can remove glitches from the screenshot image while retaining the original information. For the pre-processed screenshot image, the findContours module in the OpenCV library is called to find multiple rectangular regions of the screenshot image, and regions with an area smaller than a threshold are removed. The images corresponding to the remaining rectangular regions are extracted to obtain the N unit screenshots.

[0055] The terminal device then performs overall black and white image recognition on each unit screenshot image. If all unit screenshot images are recognized as normal images, the screenshot image is determined to be a normal image; otherwise, if at least one unit screenshot image is recognized as an abnormal black and white image, the screenshot image is determined to be an abnormal black and white image.

[0056] The reason for dividing the screenshot image into multiple sub-images for separate recognition is that black and white screens may appear in individual areas instead of filling the entire page. To address this issue, the screenshot image needs to be divided into rectangular sub-images of various sizes based on the image content to avoid situations where areas with white or black screens cannot be detected.

[0057] Please see Figures 4a-4b , Figures 4a-4b This is a schematic diagram of a segmented screenshot image provided in an embodiment of this application. Figure 4aThe image shown is a screenshot. Region detection is performed on the screenshot to obtain multiple image regions. Images corresponding to each image region are extracted from the screenshot, and these extracted images are used as unit screenshot images. A piece of text or an icon corresponds to one image region. Figure 4b The dashed rectangle represents the region detection result. Figure 4b As can be seen, the title "Workbench" corresponds to an image area, and the sub-image of this image area is a screenshot of the unit; the computer icon also corresponds to an image area, and the sub-image of this image area is a screenshot of the unit; the button "Resident Contact" also corresponds to an image area, and the sub-image of this image area is also a screenshot of the unit.

[0058] As can be seen from the above, the object of overall black and white image recognition may be a screenshot of text or a screenshot of an organization. The following uses a screenshot of text as an example to illustrate the specific process of overall black and white image recognition:

[0059] The terminal device performs black screen image recognition on the screenshot text image, obtaining a black screen recognition result, which includes a black screen image or a normal black screen image. The terminal device also performs white screen image recognition on the screenshot text image, obtaining a white screen recognition result, which includes a white screen image or a normal white screen image. If both the black screen recognition result and the white screen recognition result of the screenshot text image are normal, then the screenshot text image can be determined to be a normal image. Conversely, if the black screen recognition result is a black screen image, or the white screen recognition result is a white screen image, then the screenshot text image can be determined to be an abnormal black and white image.

[0060] Of course, if the screenshot of the text image is identified as a black screen image, it can be determined that the screenshot image is specifically a black screen image within a black-and-white abnormal image; similarly, if at least one screenshot image is identified as a black screen image, it can be determined that the screenshot image is specifically a black screen image within a black-and-white abnormal image.

[0061] The specific process for performing black screen image recognition on a screenshot of the text image to obtain the black screen recognition result is as follows:

[0062] The terminal device first converts the screenshot image into a grayscale image and obtains a preset black pixel threshold. In the converted grayscale image, it counts the number of pixels whose pixel values ​​are not greater than the black pixel threshold (the black pixel threshold can be equal to 0, 0.01, or 0.1, etc.). If the ratio between this number of pixels and the total number of pixels in the grayscale image is greater than a preset percentage threshold, the black screen recognition result of the screenshot image is determined to be a black screen image; conversely, if the ratio between this number of pixels and the total number of pixels in the grayscale image is not greater than the preset percentage threshold, the black screen recognition result of the screenshot image is determined to be a normal black screen image.

[0063] The specific process for performing white screen image recognition on a screenshot of the text image to obtain the white screen recognition result is as follows:

[0064] The terminal device first converts the screenshot image into a grayscale image. It then uses the Otsu method to determine a binary pixel threshold based on the pixel values ​​of the grayscale image. The Otsu method finds a threshold number through exhaustive search. Based on this threshold number, the pixels of the grayscale image are divided into two categories (foreground and background) such that the pixel values ​​of these two categories satisfy the minimum intra-class variance. The threshold number found through exhaustive search is the binary pixel threshold. The terminal device then converts the grayscale image back to a binary image based on the binary pixel threshold. Pixel values ​​less than the binary pixel threshold are set to 0, and pixel values ​​not less than the binary pixel threshold are set to 1, resulting in a binary image with only 0 or 1 pixel values. The terminal device then acquires a preset white image, where all pixel values ​​are greater than or equal to a white threshold (which can be 1, 0.99, or 0.9). Finally, the similarity (referred to as first similarity) between the binary image and the white image is calculated. If the first similarity is less than the first similarity threshold, then the white screen recognition result of the screenshot text image can be determined as a normal white screen image; conversely, if the first similarity is not less than the first similarity threshold, then the white screen recognition result of the screenshot text image can be determined as a white screen image. It should be noted that the essence of the similarity between a binary image and a white image refers to the difference between the pixel value distribution of the binary image and the pixel value distribution of the white image.

[0065] Please see Figures 5a-5c , Figures 5a-5c This is a schematic diagram of image binarization provided in an embodiment of this application. Figure 5a It's a screenshot image. Screenshot images can be color images. First, convert the screenshot image to a grayscale image (e.g., grayscale image without color information). Figure 5b (As shown). Then, the big-law method is used to determine the binary pixel threshold. In the grayscale image, the pixel values ​​less than the binary pixel threshold are adjusted to 0, and the pixel values ​​not less than the binary pixel threshold are adjusted to 1. This yields the binary image (as shown). Figure 5c (As shown). Comparison Figure 5b and Figure 5c It can be seen that Figure 5c The pixels in the image are either black or white. The user's avatar's pixels changed from gray to white, therefore... Figure 5c You won't be able to see the user's profile picture.

[0066] Optionally, the two methods described above for identifying whether a screenshot image is a normal image or a black-and-white abnormal image can be combined. For example, if both methods determine that the screenshot image is a black-and-white abnormal image, then it can be definitively determined that the screenshot image is a black-and-white abnormal image; or, if either method determines that the screenshot image is a black-and-white abnormal image, then it can be definitively determined that the screenshot image is a black-and-white abnormal image. As another example, if both methods determine that the screenshot image is a normal image, then it can be definitively determined that the screenshot image is a normal image; or, if either method determines that the screenshot image is a normal image, then it can be definitively determined that the screenshot image is a normal image.

[0067] Alternatively, we can first use method one. If method one identifies the screenshot text image as an abnormal black and white image, then we can determine that the screenshot image is an abnormal black and white image. If method one identifies the screenshot text image as a normal image, then we can use method two. If method two identifies at least one of the N screenshot images as an abnormal black and white image, then we can determine that the screenshot image is an abnormal black and white image. If method two identifies all N screenshot images as normal images, then we can determine that the screenshot image is a normal image.

[0068] Alternatively, we can first use Method 1. If Method 1 identifies the screenshot image as an abnormal black and white image, then the screenshot image is confirmed to be abnormal. If Method 1 identifies the screenshot image as a normal image, then we can use Method 2 (dividing the screenshot image into multiple unit screenshot images and then performing overall black and white image recognition on each unit screenshot image). If Method 2 identifies at least one unit screenshot image as an abnormal black and white image among the multiple unit screenshot images, then the screenshot image is confirmed to be abnormal. If Method 2 identifies all the unit screenshot images as normal images, then the screenshot image is confirmed to be normal.

[0069] Optionally, the terminal can directly perform overall black-and-white abnormal image recognition on the screenshot image. The specific process is as follows: the terminal device performs black screen image recognition and white screen image recognition on the screenshot image. If the black screen recognition result of the screenshot image is a black screen image, or the white screen recognition result of the screenshot image is a white screen image, then the screenshot image is determined to be a black-and-white abnormal image; if the black screen recognition result of the screenshot image is a black screen normal image, or the white screen recognition result of the screenshot image is a white screen normal image, then the screenshot image is determined to be a normal image.

[0070] Step S103: If the screenshot image is identified as a black and white abnormal image, then a misjudgment detection is performed on the screenshot image to obtain a misjudgment detection result; wherein, the misjudgment detection result is a detection result determined based on one or more of the first misjudgment detection result and the second misjudgment detection result, the first misjudgment detection result is a detection result based on the existence judgment of the screenshot image loading flag, and the second misjudgment detection result is a detection result based on the matching judgment between the screenshot image and the set of misjudged images, and the screenshot image loading flag includes a loading progress bar and a loading icon.

[0071] Specifically, the preceding steps can identify whether the screenshot image is an abnormal black-and-white image (specifically, a black screen image or a white screen image) or a normal image. If the screenshot image is identified as an abnormal black-and-white image, the terminal device can also perform false positive detection on the screenshot image to obtain a false positive detection result, which can be either a false positive or a true negative. Of course, if the false positive detection result is a false positive, it means that the result obtained from identifying the abnormal black-and-white image of the screenshot image is incorrect (i.e., the screenshot image is a normal image, not an abnormal black-and-white image). If the false positive detection result is a true negative, it means that the result obtained from identifying the abnormal black-and-white image of the screenshot image is correct (i.e., the screenshot image is indeed an abnormal black-and-white image).

[0072] Step S104: Determine the black-and-white screen detection result of the target page based on the misjudgment detection result, and output the black-and-white screen detection result of the target page.

[0073] Specifically, if the false positive detection result is a false positive, then the black-and-white screen detection result of the target page is determined to be a normal page, i.e., the target page is a normal page; if the false positive detection result is not a false positive, then the black-and-white screen detection result of the target page is determined to be a black-and-white screen page, i.e., the target page is a black-and-white screen page. Furthermore, if the false positive detection result is not a false positive, the terminal device can determine the black-and-white screen detection result of the target page based on the black-and-white anomaly image recognition result of the screenshot image. Specifically: if the false positive detection result is not a false positive, and the screenshot image is identified as a black screen image, then the black-and-white screen detection result of the target page can be determined to be a black screen page within a black-and-white screen page; if the false positive detection result is not a false positive, and the screenshot image is identified as a white screen image, then the black-and-white screen detection result of the target page can be determined to be a white screen page within a black-and-white screen page.

[0074] Optionally, if the target page is detected as a black-and-white screen, a screenshot can be displayed, along with a sequence of adjacent images. This sequence is a combination of screenshots from multiple pages belonging to the same path as the target page. For example, if the target page corresponds to a specific step in the application, the pages corresponding to the preceding and subsequent steps can be considered as belonging to the same path as the target page. Restoring the path facilitates subsequent troubleshooting.

[0075] Please see Figure 6 , Figure 6 This is a schematic diagram of a detection result display platform provided in an embodiment of this application. Detection results are categorized by platform, application, and task. Detected black-and-white anomaly images can be displayed in an image sequence: screenshots that are detected as black-and-white anomalies are marked with dashed rectangles. In addition to marking these screenshots, the path of the black-and-white anomaly image is also reconstructed, displaying other screenshots belonging to the same path. This sequential display of black-and-white anomaly images facilitates subsequent troubleshooting. It should be noted that for the black-and-white anomaly image numbered 01, since there are no other screenshots along its path, only one image is displayed.

[0076] As mentioned above, this application avoids missed detections by cropping the header and footer to remove their influence. Furthermore, segmenting the screenshot image into rectangular sub-images of various sizes based on the image content avoids situations where areas of white or black screen cannot be detected. In addition, different recognition strategies are used for white screen image recognition and black screen image recognition, which can improve the recognition accuracy. When the target page is an abnormal black and white page, displaying the target page on the display platform provides a more intuitive display effect. Moreover, by restoring the path, the problem can be quickly located, making it easier to find the cause of the problem.

[0077] Please see Figure 7 , Figure 7 This is a schematic flowchart of a false positive detection method provided in an embodiment of this application. False positive detection is as described above. Figure 2 In a specific embodiment corresponding to step S203 in the embodiments, the false detection includes the following steps:

[0078] Step S701: If the screenshot image is identified as a black and white abnormal image, then a first misjudgment detection is performed on the screenshot image to obtain the first misjudgment detection result.

[0079] Specifically, if the terminal device detects that the screenshot image is an abnormal black and white image, a first false positive detection is performed on the screenshot image. The first false positive detection is to detect whether there are loading indicators (including loading progress bars and loading icons) in the screenshot image. If a loading progress bar or a loading icon is detected in the screenshot image, it means that the target page is a loading page. Therefore, the identification of the black and white abnormality of the screenshot image is a false positive, that is, the first false positive result is a false positive. Conversely, if no loading progress bar or loading icon is detected in the screenshot image, it means that the target page is not a loading page. Therefore, the identification of the black and white abnormality of the screenshot image is not a false positive, that is, the first false positive result is a true false positive.

[0080] The method for determining whether a loading progress bar exists in a screenshot image is as follows:

[0081] To determine if a loading progress bar exists in a screenshot, the loading progress bar area must first be accurately located. This area is typically below the top status bar; therefore, simply measure the width of the top status bar and then take 5-10 pixels down. This area represents the loading progress bar. The terminal device extracts the image corresponding to the loading progress bar area from the screenshot and uses this extracted image as the progress bar image. The progress bar image undergoes overall black-and-white image recognition to determine whether it is a normal image or an abnormal black-and-white image. The process of performing overall black-and-white image recognition on the progress bar image is consistent with the aforementioned process for performing overall black-and-white image recognition on the screenshot body image (i.e., performing black-and-white image recognition and white-and-white image recognition separately, and using the results of both to determine the overall black-and-white image recognition result of the progress bar). If the progress bar image is identified as an abnormal black-and-white image, it is determined that a loading progress bar does not exist in the screenshot; conversely, if the progress bar image is identified as a normal image, it is determined that a loading progress bar exists in the screenshot.

[0082] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating a method for recognizing a loading progress bar, as provided in an embodiment of this application. Figure 8 The image shown is a screenshot. The area 5-10 pixels wide below the system status bar is called the loading progress bar area. Extracting the sub-image corresponding to this area yields the progress bar image. The terminal device performs white-screen and black-screen image recognition on this progress bar image. If the recognized progress bar image is either black or white, it can be assumed that the screenshot does not contain a loading progress bar; conversely, if the recognized progress bar image is neither black nor white, it can be assumed that the screenshot contains a loading progress bar.

[0083] The method to determine whether a loading icon exists in a screenshot image is as follows:

[0084] The terminal device obtains a set of loading icons, which includes M loading icons, where M is a positive integer. Loading icons can be spinning top icons or include loading text prompts. The terminal device calculates the similarity (referred to as second similarity) between the screenshot image and each loading icon. If all M second similarities are less than a preset second similarity threshold, the screenshot image is determined to contain no loading icon; conversely, if at least one of the M second similarities is not less than the second similarity threshold, the screenshot image contains a loading icon.

[0085] The second similarity between the screenshot and the loading icon is calculated using SIFT features. Specifically, the SIFT features of the screenshot and the loading icon are extracted, and the similarity between these two features is calculated. This similarity is the second similarity between the screenshot and the loading icon.

[0086] In layman's terms, if a screenshot can match one or more loading icons, then it is determined that a loading icon exists in the screenshot, and thus the first false positive detection result can be considered a false positive. Conversely, if a screenshot cannot match any loading icon, then it is determined that a loading icon does not exist in the screenshot.

[0087] Please see Figure 9 , Figure 9 This is a schematic diagram illustrating a method for recognizing a loading icon, provided in an embodiment of this application. Figure 9 The image shown is a screenshot. Since a loading icon ("Loading") is present in the center of the screenshot, matching the screenshot with multiple loading icons will inevitably result in one matching icon. At this point, it can be concluded that there was a misjudgment in the identification of the black-and-white abnormal image in the screenshot. Therefore, the first misjudgment detection result is a misjudgment, meaning the screenshot is a normal image.

[0088] Step S702: Perform a second misjudgment detection on the screenshot image to obtain the second misjudgment detection result.

[0089] Specifically, the terminal device acquires a set of misjudged images, which includes K misjudged images, where K is a positive integer. Misjudged images can be those whose detection threshold falls within the threshold for black-and-white abnormal images, but which are considered normal images by manual review. For example, misjudged images can be images with very little content.

[0090] The terminal device calculates the similarity (referred to as third similarity) between the screenshot image and each misjudged image. If all K third similarities are less than a preset third similarity threshold, the second misjudgment detection result is determined to be a non-misjudgment. Conversely, if at least one of the K third similarities is not less than the preset third similarity threshold, the second misjudgment detection result is determined to be a misjudgment.

[0091] This application employs ORB features to compare the similarity between screenshot images and misclassified images. Specifically, ORB features are extracted from both the screenshot and misclassified images, and the similarity between these two ORB features is used as the similarity between the screenshot and misclassified images. SIFT features are not used here due to speed considerations. The advantage of ORB features is that they are two orders of magnitude faster than SIFT, enabling feature comparison of hundreds of images within seconds. In the ORB feature extraction process, FAST is first used as the feature point detection operator, followed by descriptor calculation using an improved BRIEF algorithm to extract and compute feature points. Finally, the KNN (K Nearest Neighbors) method is used to filter the number of points whose Hamming distance meets a threshold, and the maximum number of matching points is checked. The similarity between the two images is then calculated by determining the matching degree.

[0092] Please see Figure 10 , Figure 10 This is a schematic diagram of a screenshot image provided in an embodiment of this application. Figure 10 The image shown is a screenshot. While a screenshot is a capture of a normal webpage, it contains relatively little content and lacks a loading progress bar and icon. If only the first false positive detection is used, it will be judged as a normal image (i.e., the screenshot will be identified as a black and white abnormal image). In this case, a second false positive detection is required, using the similarity between the set of false positive images and the screenshot to determine if the screenshot is a normal image.

[0093] Optionally, if the second false positive detection result is a false positive, the terminal device can add the screenshot image as a new false positive image to the set of false positive images, then the set of false positive images will include K+1 false positive images.

[0094] Please see Figure 6 In addition to displaying black-and-white abnormal pages, the platform also supports manual review. This manual review process continuously increases the number of false positive images in the set, reducing the likelihood of future misjudgments. If a tester clicks the "False Positive" button on the platform, the screenshot marked with the corresponding dotted rectangle will be added to the set of false positive images. This screenshot will also be adjusted from a black-and-white abnormal image to a normal image.

[0095] Step S703: Determine the misjudgment detection result based on the first misjudgment detection result and the second misjudgment detection result.

[0096] Specifically, if both the first and second false positive detection results are not false positives, then the false positive detection result is determined to be not false positive; if at least one of the first and second false positive detection results is false positive, then the false positive detection result is determined to be false positive.

[0097] Optionally, steps S701-S703 above involve determining the first false positive detection result and the second false positive detection result respectively, and then using these two results to jointly determine the false positive detection result. Alternatively, the false positive detection result can be directly determined solely from the first false positive detection result (or the second false positive detection result). Specifically: if the first false positive detection result is a false positive, then the false positive detection result is directly determined to be a false positive; if the first false positive detection result is not a false positive, then the false positive detection result is directly determined to be not a false positive. If the second false positive detection result is a false positive, then the false positive detection result is directly determined to be a false positive; if the second false positive detection result is not a false positive, then the false positive detection result is directly determined to be not a false positive. In general, the first false positive detection result (or the second false positive detection result) equals the false positive detection result.

[0098] Please see Figure 11 , Figure 11 This is a schematic diagram of an image processing framework provided in an embodiment of this application. The image processing framework is divided into three main modules. The first module is the image preprocessing module, which includes image acquisition, image cleaning, and image data organization and storage. The second module is the image detection module, which includes white screen detection (corresponding to the white screen image recognition in this application), black screen detection (corresponding to the black screen image recognition in this application), detail region detection, missed detection removal, false positive removal, and storage of detection results in the database. The third module is the image result display module, which provides result display and front-end display for manual review (e.g., Figure 6 The front-end interface shown mainly includes image hierarchical display, image addition operation sequence (which can correspond to the black and white abnormal images in this application being displayed according to the image sequence), and manual screening and filtering.

[0099] Please see Figure 12 , Figure 12This is a schematic diagram of an image preprocessing module provided in an embodiment of this application. Images are acquired from two main sources: screenshots taken at key known steps in test cases and screenshots taken during random testing to explore unknown steps. The images acquired by the image acquisition unit can be called raw images. After acquiring the raw images, they enter the image cleaning unit. The main function of the image cleaning unit is to remove useless screenshots according to filtering rules. For example, in automated testing, images are retrieved by configuring specific test IDs, and data with useless IDs is filtered out. The same applies to random testing, where images are retrieved based on random task IDs. The raw images after image cleaning can correspond to the screenshot images in this application. The raw images after image cleaning enter the image data processing and storage unit. This unit assigns unique IDs to the images and facilitates the subsequent display of image sequences. The image data processing and storage unit also records screenshot sequences under the same path and performs black-and-white screen detection on images based on different sequences. This allows the detection results to be displayed sequentially, rather than just showing a single black-and-white abnormal image. For example, in automated test cases, the division is based on test cases, while in random testing, it is based on minutes.

[0100] Please see Figure 13 , Figure 13 This is a schematic diagram of an image detection module provided in an embodiment of this application. The image detection module includes three main modules: white screen detection, black screen detection, and detail region detection. For white screen detection, this application uses the big rule method to binarize the image. The similarity between the processed binary image and the white image is calculated. Images with a similarity lower than a set threshold are considered white screen images. This process corresponds to the white screen image recognition in this application.

[0101] However, the above strategy is not suitable for black screen detection because the binarized image with rich elements will have too high a similarity to a black image, easily leading to misjudgment. Therefore, for black screen detection, this application adopts a method of converting the image to a grayscale image, counting the proportion of darker elements in the image, and if the proportion exceeds a set threshold, the image is considered a black screen image. This process corresponds to the black screen image recognition in this application.

[0102] The above describes an overall assessment of the entire image. However, detecting detailed areas requires rectangular segmentation. By dividing the image into multiple rectangular regions and performing white-screen and black-screen detection on each region separately, the black-and-white details of the image can be better captured. This process corresponds to the process described in this application of dividing a screenshot image into N unit screenshot images and performing overall black-and-white anomaly image recognition on each unit screenshot image.

[0103] During manual review, some images were found to have been missed in detection. This occurred because the header and footer sections were rich in content, increasing the overall image richness. However, these two sections are not the focus of this application's black-and-white anomaly image detection and are irrelevant to the overall result. Therefore, to remove the influence of the header and footer, these two sections are cropped directly after the screenshot image is read to avoid missed detections. This process corresponds to the process of cropping the screenshot image to obtain the screenshot text image in this application.

[0104] During the manual review, some images were found to be misjudged. The main issue was the misjudgment of the page loading status, because the page was still loading when the image was captured; that is, the image captured a loading page. These images can be categorized into two types based on their characteristics: one is a loading progress bar; the other is a loading icon (such as a spinning wheel, loading text prompts, etc.). This process corresponds to the first misjudgment detection process in this application, which detects the presence of a loading progress bar or loading icon in the screenshot image to determine if a misjudgment has occurred.

[0105] Besides misjudging page loading status, it may also misjudge some normal pages with limited content. Therefore, to further optimize the performance of black-and-white screen detection, this application introduces the concept of a misjudged image library (which can correspond to the misjudged image set in this application). For misjudged images whose detection threshold is within the threshold for black-and-white screen anomalies, but which are manually reviewed and deemed normal, they will be stored. If a similar misjudgment occurs again, the data in the misjudged image library can be compared with the current screenshot image to reduce the number of misjudgments. This process corresponds to the second misjudgment detection process in this application, which checks whether at least one of the K misjudged images matches the screenshot image to determine if a misjudgment has occurred. By constructing the misjudged image library, the detection process is continuously iterated and optimized to reduce the misjudgment rate.

[0106] To verify the accuracy of this application, the detection results are compared below. Fifty normal images and 50 black-and-white abnormal images were randomly selected, totaling 100 images, as experimental data. The following algorithm was used to compare the detection results. Overall black and white image recognition refers to directly performing overall black and white image recognition on a screenshot image, and using the recognition result to directly determine the black and white screen detection result of the target page; Overall black and white image recognition + missed detection elimination refers to cropping the header and footer of the screenshot image, performing overall black and white image recognition on the remaining image (i.e., the screenshot text image of this application), and using the recognition result to determine the black and white screen detection result of the target page; Overall black and white image recognition + false positive elimination (progress bar, loading icon, similarity comparison) refers to performing overall black and white image recognition on a screenshot image, and if it is identified as a black and white abnormal image, false positive detection is performed, and the false positive detection result determines the black and white screen detection result of the target page; Overall black and white image recognition + missed detection elimination (removing header and footer) + false positive elimination (progress bar, loading icon, similarity comparison) refers to cropping the header and footer of the screenshot image, performing overall black and white image recognition on the remaining image (i.e., the screenshot text image of this application), and if it is identified as a black and white abnormal image, false positive detection is performed, and the false positive detection result determines the black and white screen detection result of the target page. The comparison results are shown in Table 1:

[0107] Table 1

[0108]

[0109] Analysis of Table 1 shows that for the already tested sample data, accurate identification is already achievable by adding optimization strategies to remove missed detections and false positives. However, removing only missed detections (i.e., removing the influence of the footer and header) can decrease the accuracy of some normal images. This is because some normal images with few page elements but rich footers and headers are misidentified, which needs to be avoided by similarity comparison in the false positive removal process. Algorithms that only remove false positives can accurately identify normal images, but they cannot completely and accurately identify previously detected black-and-white abnormal images because they are affected by the footer and header, failing to identify abnormal images. Therefore, this application combines optimization of both approaches to accurately identify both normal and black-and-white abnormal images.

[0110] To verify the algorithm's generalization ability to detect undetected data, 50 normal images and 50 black-and-white abnormal images (a total of 100 images) were randomly selected as experimental data. The detection results of the following algorithm were compared. The comparison results are shown in Table 2:

[0111] Table 2

[0112]

[0113] Analysis of Table 2 shows that for previously undetected sample data, by adding methods to remove missed and false positives, the accuracy for normal and black-and-white abnormal images can reach over 90%. False positives for normal images mainly occur because false positive removal fails to eliminate these pages, indicating that they contain previously undiscovered loading indicators or are normal pages with very little content. Therefore, a manual review process needs to be added to the display platform. For incorrectly detected images, manually adding false positive images and loading indicator samples can prevent the same situation from recurring, thus gradually achieving convergence.

[0114] As mentioned above, when a screenshot image is identified as an abnormal black and white image, further false positive detection is required. Based on the false positive detection results, it is determined whether the page to be detected is a black and white screen page. Refining the detection granularity can improve the recognition accuracy. Moreover, the false positive detection results are determined based on the existence of the loading flag in the screenshot image or the matching between the screenshot image and the set of false positive images. This can avoid false positives for loaded pages or pages with less content, and ensure the recognition accuracy of black and white screen pages.

[0115] For further details, please see Figure 14 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Figure 14 As shown, the image processing device 1 can be applied to the above-mentioned... Figures 1-13 The terminal device in the corresponding embodiment. Specifically, the image processing device 1 may be a computer program (including program code) running on a computer device, for example, the image processing device 1 may be an application software; the image processing device 1 may be used to execute the corresponding steps in the method provided in the embodiments of this application.

[0116] The image processing device 1 may include: an acquisition module 11, a recognition module 12, a detection module 13, and an output module 14.

[0117] The acquisition module 11 is used to acquire a screenshot image, which is an image generated by taking a screenshot of the target page;

[0118] The recognition module 12 is used to perform black and white abnormal image recognition on the screenshot image;

[0119] The detection module 13 is used to perform misjudgment detection on the screenshot image if it is identified as a black and white abnormal image, and obtain a misjudgment detection result; wherein, the misjudgment detection result is a detection result determined based on one or more of a first misjudgment detection result and a second misjudgment detection result, the first misjudgment detection result is a detection result based on the existence judgment of the screenshot image loading flag, and the second misjudgment detection result is a detection result based on the matching judgment between the screenshot image and the set of misjudged images; the screenshot image loading flag includes a loading progress bar and a loading icon;

[0120] Output module 14 is used to determine the black-and-white screen detection result of the target page based on the misjudgment detection result, and output the black-and-white screen detection result of the target page.

[0121] In one possible implementation, when the recognition module 12 is used to perform black-and-white anomaly image recognition on the screenshot image, it is specifically used for:

[0122] The header and footer areas of the screenshot image are cropped to obtain the screenshot text image;

[0123] Perform overall black-and-white image recognition on the screenshot text image;

[0124] If the screenshot image is identified as an abnormal black and white image, then the screenshot image is determined to be an abnormal black and white image.

[0125] In one possible implementation, when the recognition module 12 performs overall black-and-white image recognition on the screenshot text image, it is specifically used for:

[0126] Perform black screen image recognition on the screenshot text image to obtain a black screen recognition result; the black screen recognition result includes a black screen image.

[0127] Perform white screen image recognition on the screenshot text image to obtain a white screen recognition result; the white screen recognition result includes a white screen image.

[0128] If the black screen recognition result is a black screen image, or the white screen recognition result is a white screen image, then the screenshot text image is determined to be an abnormal black and white image.

[0129] In one possible implementation, when the recognition module 12 performs black screen image recognition on the screenshot text image to obtain a black screen recognition result, it is specifically used for:

[0130] Convert the screenshot image to a grayscale image;

[0131] In the grayscale image, the number of pixels whose pixel values ​​are not greater than the black pixel threshold is counted.

[0132] If the ratio between the number of pixels and the total number of pixels in the grayscale image is greater than the percentage threshold, then the black screen recognition result is determined to be a black screen image.

[0133] In one possible implementation, when the recognition module 12 performs white screen image recognition on the screenshot text image to obtain a white screen recognition result, it is specifically used for:

[0134] Convert the screenshot image to a grayscale image;

[0135] A binary pixel threshold is determined based on the pixel values ​​of the grayscale image, and the foreground and background types of the grayscale image pixels are divided by the binary pixel threshold to satisfy the minimum intra-class variance.

[0136] The grayscale image is converted into a binary image based on the binary pixel threshold, and a first similarity is determined between the white image and the binary image;

[0137] If the first similarity is not less than the first similarity threshold, then the white screen recognition result is determined to be a white screen image.

[0138] In one possible implementation, when the recognition module is used to perform black-and-white anomaly image recognition on the screenshot image, it is specifically used for:

[0139] Based on the layout of the screenshot image, the screenshot image is divided into N unit screenshot images, where N is a positive integer;

[0140] Perform overall black-and-white image recognition on each unit's screenshot image;

[0141] If at least one screenshot is identified as an abnormal black and white image, then the screenshot is determined to be an abnormal black and white image.

[0142] In one possible implementation, the false detection result includes either a non-false detection or a false detection, and the black-and-white screen detection result includes either a normal page or a black-and-white screen page.

[0143] When the output module 14 is used to determine the black-and-white screen detection result of the target page based on the false positive detection result, it is specifically used for:

[0144] If the false detection result is a false detection, then the black and white screen detection result of the target page is determined to be a normal page;

[0145] If the false positive detection result is not a false positive, then the black and white screen detection result of the target page is determined to be a black and white screen page.

[0146] In one possible implementation, when the false positive detection result is a detection result determined based on the first false positive detection result and the second false positive detection result, the detection module 13, when performing false positive detection on the screenshot image to obtain the false positive detection result, is specifically used for:

[0147] Perform a first false positive detection on the screenshot image to obtain the first false positive detection result;

[0148] Perform a second false positive detection on the screenshot image to obtain the second false positive detection result;

[0149] The misjudgment detection result is determined based on the first misjudgment detection result and the second misjudgment detection result.

[0150] In one possible implementation, the first false positive detection result includes either a non-false positive or a false positive. When the detection module 13 performs the first false positive detection on the screenshot image to obtain the first false positive detection result, it is specifically used for:

[0151] If the screenshot image contains a loading progress bar or a loading icon, then the first false detection result is determined to be a false judgment;

[0152] If the screenshot image does not contain a loading progress bar or a loading icon, then the first false positive detection result is determined to be a true false positive.

[0153] In one possible implementation, the detection module 13 is further configured to:

[0154] Extract the image corresponding to the loading progress bar area from the screenshot image, and use the extracted image as the progress bar image;

[0155] Perform overall black and white image recognition on the progress bar image;

[0156] If the progress bar image is identified as an abnormal black and white image, it is determined that there is no loading progress bar in the screenshot image;

[0157] If the progress bar image is identified as a normal image, then it is determined that a loading progress bar exists in the screenshot image.

[0158] In one possible implementation, the detection module is further configured to:

[0159] Get a set of loading icons, where the set of loading icons includes M loading icons, and M is a positive integer;

[0160] Determine the second similarity between the screenshot images and each loading icon;

[0161] If all M second similarities are less than the second similarity threshold, then it is determined that there is no loading icon in the screenshot image;

[0162] If at least one of the M second similarities is not less than the second similarity threshold, then it is determined that a loading icon exists in the screenshot image.

[0163] In one possible implementation, the second false positive detection result includes either no false positive or false positive. When the detection module 13 performs second false positive detection on the screenshot image to obtain the second false positive detection result, it is specifically used for:

[0164] Obtain a set of misjudged images, wherein the set of misjudged images includes K misjudged images;

[0165] Determine the third similarity between the screenshot image and each misclassified image;

[0166] If all K third similarities are less than the third similarity threshold, then the second misjudgment detection result is determined to be a non-misjudgment.

[0167] If at least one of the K third similarities is not less than the third similarity threshold, then the second misjudgment detection result is determined to be a misjudgment.

[0168] The detection module 13 is also used for:

[0169] If the second false positive detection result is a false positive, then the screenshot image is added as a false positive image to the set of false positive images.

[0170] In one possible implementation, the target page is the page of the target program at a preset step during the testing process using test cases; or...

[0171] The target page is the page of the target program at any step during the testing process using a random testing method; or,

[0172] The target page is the page whose blocking time exceeds a time threshold during the online operation of the target program.

[0173] In one possible implementation, the black-and-white screen detection result includes a black-and-white screen page, and the output module 14, when outputting the black-and-white screen detection result of the target page, is specifically used for:

[0174] If the target page is detected as a black and white screen, then the screenshot image is displayed, as well as the adjacent image sequence of the screenshot image. The adjacent image sequence is an image sequence composed of screenshots of multiple pages belonging to the same path as the target page.

[0175] In one possible implementation, the black-and-white screen detection result also includes a normal page, and the output module 14 is further used for:

[0176] If the screenshot image is identified as a normal image, then the black-and-white screen detection result of the target page is determined to be a normal page.

[0177] According to an embodiment of the present invention, Figures 1-13 Each step involved in the method shown can be performed by... Figure 14 The image processing apparatus shown is executed by various modules within it. For example, Figure 2 Steps S201-S203 shown in the figure Figure 7 Steps S701-S703 shown can be respectively performed by Figure 14 The acquisition module 11, identification module 12, detection module 13 and output module 14 shown are used to perform this.

[0178] Further, please see Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figures 1-13 In the corresponding embodiment, the terminal device can be a computer device 1000. For example... Figure 15 As shown, computer device 1000 may include a user interface 1002, a processor 1004, an encoder 1006, and a memory 1008. A signal receiver 1016 is used to receive or transmit data via a cellular interface 1010, a Wi-Fi interface 1012, ..., or an NFC interface 1014. The encoder 1006 encodes the received data into a data format that can be processed by a computer. The memory 1008 stores a computer program, and the processor 1004 is configured to execute the steps in any of the above method embodiments via the computer program. The memory 1008 may include volatile memory (e.g., dynamic random access memory DRAM) and may also include non-volatile memory (e.g., one-time programmable read-only memory OTPROM). In some instances, the memory 1008 may further include memory remotely located relative to the processor 1004, which can be connected to the computer device 1000 via a network. The user interface 1002 may include a keyboard 1018 and a display 1020.

[0179] exist Figure 15 In the computer device 1000 shown, the processor 1004 can be used to call computer programs stored in the memory 1008 to achieve:

[0180] Acquire a screenshot image, wherein the screenshot image is an image generated by taking a screenshot of the target page;

[0181] Perform black-and-white anomaly image recognition on the screenshot image;

[0182] If the screenshot image is identified as a black and white abnormal image, then a false positive detection is performed on the screenshot image to obtain a false positive detection result; wherein, the false positive detection result is a detection result determined based on one or more of a first false positive detection result and a second false positive detection result, the first false positive detection result is a detection result based on the existence judgment of the screenshot image loading flag, and the second false positive detection result is a detection result based on the matching judgment between the screenshot image and the set of false positive images; the screenshot image loading flag includes a loading progress bar and a loading icon;

[0183] Based on the false positive detection results, determine the black-and-white screen detection results of the target page, and output the black-and-white screen detection results of the target page.

[0184] In one embodiment, when the processor 1004 performs black-and-white anomaly image recognition on the screenshot image, it specifically executes the following steps:

[0185] The header and footer areas of the screenshot image are cropped to obtain the screenshot text image;

[0186] Perform overall black-and-white image recognition on the screenshot text image;

[0187] If the screenshot image is identified as an abnormal black and white image, then the screenshot image is determined to be an abnormal black and white image.

[0188] In one embodiment, when the processor 1004 performs overall black-and-white image recognition on the screenshot text image, it specifically executes the following steps:

[0189] Perform black screen image recognition on the screenshot text image to obtain a black screen recognition result; the black screen recognition result includes a black screen image.

[0190] Perform white screen image recognition on the screenshot text image to obtain a white screen recognition result; the white screen recognition result includes a white screen image.

[0191] If the black screen recognition result is a black screen image, or the white screen recognition result is a white screen image, then the screenshot text image is determined to be an abnormal black and white image.

[0192] In one embodiment, when the processor 1004 performs black screen image recognition on the screenshot text image and obtains the black screen recognition result, it specifically performs the following steps:

[0193] Convert the screenshot image to a grayscale image;

[0194] In the grayscale image, the number of pixels whose pixel values ​​are not greater than the black pixel threshold is counted.

[0195] If the ratio between the number of pixels and the total number of pixels in the grayscale image is greater than the percentage threshold, then the black screen recognition result is determined to be a black screen image.

[0196] In one embodiment, when the processor 1004 performs white screen image recognition on the screenshot text image and obtains the white screen recognition result, it specifically performs the following steps:

[0197] Convert the screenshot image to a grayscale image;

[0198] A binary pixel threshold is determined based on the pixel values ​​of the grayscale image, and the foreground and background types of the grayscale image pixels are divided by the binary pixel threshold to satisfy the minimum intra-class variance.

[0199] The grayscale image is converted into a binary image based on the binary pixel threshold, and a first similarity is determined between the white image and the binary image;

[0200] If the first similarity is not less than the first similarity threshold, then the white screen recognition result is determined to be a white screen image.

[0201] In one embodiment, when the processor 1004 performs black-and-white anomaly image recognition on the screenshot image, it specifically executes the following steps:

[0202] Based on the layout of the screenshot image, the screenshot image is divided into N unit screenshot images, where N is a positive integer;

[0203] Perform overall black-and-white image recognition on each unit's screenshot image;

[0204] If at least one screenshot is identified as an abnormal black and white image, then the screenshot is determined to be an abnormal black and white image.

[0205] In one embodiment, the false detection result includes either a non-false detection or a false detection, and the black-and-white screen detection result includes either a normal page or a black-and-white screen page.

[0206] When processor 1004 determines the black-and-white screen detection result of the target page based on the misjudgment detection result, it specifically performs the following steps:

[0207] If the false detection result is a false detection, then the black and white screen detection result of the target page is determined to be a normal page;

[0208] If the false positive detection result is not a false positive, then the black and white screen detection result of the target page is determined to be a black and white screen page.

[0209] In one embodiment, when the false positive detection result is a detection result determined based on the first false positive detection result and the second false positive detection result, the processor 1004 performs the following steps when performing false positive detection on the screenshot image and obtaining the false positive detection result:

[0210] Perform a first false positive detection on the screenshot image to obtain the first false positive detection result;

[0211] Perform a second false positive detection on the screenshot image to obtain the second false positive detection result;

[0212] The misjudgment detection result is determined based on the first misjudgment detection result and the second misjudgment detection result.

[0213] In one embodiment, the first false positive detection result includes either a non-false positive or a false positive;

[0214] When the processor 1004 performs a first false positive detection on the screenshot image and obtains the first false positive detection result, it specifically performs the following steps:

[0215] If the screenshot image contains a loading progress bar or a loading icon, then the first false detection result is determined to be a false judgment;

[0216] If the screenshot image does not contain a loading progress bar or a loading icon, then the first false positive detection result is determined to be a true false positive.

[0217] In one embodiment, the processor 1004 further performs the following steps:

[0218] Extract the image corresponding to the loading progress bar area from the screenshot image, and use the extracted image as the progress bar image;

[0219] Perform overall black and white image recognition on the progress bar image;

[0220] If the progress bar image is identified as an abnormal black and white image, it is determined that there is no loading progress bar in the screenshot image;

[0221] If the progress bar image is identified as a normal image, then it is determined that a loading progress bar exists in the screenshot image.

[0222] In one embodiment, the processor 1004 further performs the following steps:

[0223] Get a set of loading icons, where the set of loading icons includes M loading icons, and M is a positive integer;

[0224] Determine the second similarity between the screenshot images and each loading icon;

[0225] If all M second similarities are less than the second similarity threshold, then it is determined that there is no loading icon in the screenshot image;

[0226] If at least one of the M second similarities is not less than the second similarity threshold, then it is determined that a loading icon exists in the screenshot image.

[0227] In one embodiment, the second false positive detection result includes either no false positive or false positive. When the processor 1004 performs second false positive detection on the screenshot image and obtains the second false positive detection result, it specifically executes the following steps:

[0228] Obtain a set of misjudged images, wherein the set of misjudged images includes K misjudged images;

[0229] Determine the third similarity between the screenshot image and each misclassified image;

[0230] If all K third similarities are less than the third similarity threshold, then the second misjudgment detection result is determined to be a non-misjudgment.

[0231] If at least one of the K third similarities is not less than the third similarity threshold, then the second misjudgment detection result is determined to be a misjudgment.

[0232] The processor 1004 further performs the following steps:

[0233] If the second false positive detection result is a false positive, then the screenshot image is added as a false positive image to the set of false positive images.

[0234] In one embodiment, the target page is the page of the target program when the test reaches a preset step during the testing process using test cases; or...

[0235] The target page is the page of the target program at any step during the testing process using a random testing method; or,

[0236] The target page is the page whose blocking time exceeds a time threshold during the online operation of the target program.

[0237] In one embodiment, the black-and-white screen detection result includes a black-and-white screen page. When the processor 1004 executes the black-and-white screen detection result for outputting the target page, it specifically performs the following steps:

[0238] If the target page is detected as a black and white screen, then the screenshot image is displayed, as well as the adjacent image sequence of the screenshot image. The adjacent image sequence is an image sequence composed of screenshots of multiple pages belonging to the same path as the target page.

[0239] In one embodiment, the black-and-white screen detection result also includes a normal page, and the processor 1004 further performs the following steps:

[0240] If the screenshot image is identified as a normal image, then the black-and-white screen detection result of the target page is determined to be a normal page.

[0241] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figures 1-13 The description of the image processing method in the corresponding embodiments can also be performed as described above. Figure 14 The description of the image processing apparatus 1 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0242] Furthermore, it should be noted that this application embodiment also provides a computer storage medium, which stores a computer program executed by the aforementioned image processing device 1. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figures 1-13 The description of the image processing method in the corresponding embodiments is already provided and will not be repeated here. Similarly, the beneficial effects of using the same method will not be repeated here either. For technical details not disclosed in the embodiments of the computer storage medium involved in this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed and executed on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network. Multiple computer devices distributed across multiple locations and interconnected via a communication network can be combined to form a blockchain network.

[0243] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figures 1 to 13 The methods described in the corresponding embodiments are therefore not repeated here.

[0244] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0245] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a screenshot image, the screenshot image being an image generated by screenshot processing on a target page; performing black-and-white abnormal image identification on the screenshot image; if the screenshot image is identified as a black-and-white abnormal image, performing false judgment detection on the screenshot image to obtain a false judgment detection result; wherein the false judgment detection result is a detection result determined according to one or more of a first false judgment detection result and a second false judgment detection result, the first false judgment detection result is a detection result based on existence judgment of a screenshot image loading mark, and the second false judgment detection result is a detection result based on matching judgment between the screenshot image and a false judgment image set; the screenshot image loading mark comprises a loading progress bar and a loading icon; determining a black-and-white screen detection result of the target page according to the false judgment detection result, and outputting the black-and-white screen detection result of the target page; wherein the existence of the loading progress bar is determined by: extracting an image corresponding to the loading progress bar region from the screenshot image, and taking the extracted image as a progress bar image; performing overall black-and-white image identification on the progress bar image; if the progress bar image is identified as a black-and-white abnormal image, it is determined that there is no loading progress bar in the screenshot image; if the progress bar image is identified as a normal image, it is determined that there is a loading progress bar in the screenshot image.

2. The method of claim 1, wherein, The black-and-white abnormal image identification on the screenshot image comprises: performing cropping processing on the header region and the footer region of the screenshot image to obtain a screenshot text image; performing overall black-and-white image identification on the screenshot text image; if the screenshot text image is identified as a black-and-white abnormal image, it is determined that the screenshot image is a black-and-white abnormal image.

3. The method of claim 2, wherein, The overall black-and-white image identification on the screenshot text image comprises: performing black screen image identification on the screenshot text image to obtain a black screen identification result; the black screen identification result comprises a black screen image; performing white screen image identification on the screenshot text image to obtain a white screen identification result; the white screen identification result comprises a white screen image; if the black screen identification result is a black screen image, or the white screen identification result is a white screen image, it is determined that the screenshot text image is a black-and-white abnormal image.

4. The method of claim 3, wherein, The black screen image identification on the screenshot text image to obtain a black screen identification result comprises: converting the screenshot text image into a grayscale image; in the grayscale image, the number of pixels with a pixel value not greater than a black pixel threshold value is counted; if the ratio between the number of pixels and the total number of pixels of the grayscale image is greater than a ratio threshold value, it is determined that the black screen identification result is a black screen image.

5. The method of claim 3, wherein, The white screen image identification on the screenshot text image to obtain a white screen identification result comprises: converting the screenshot text image into a grayscale image; determining a binary pixel threshold value according to the pixel value of the grayscale image, and dividing the foreground type and the background type of the grayscale image pixels by the binary pixel threshold value to satisfy the minimum within-class variance; converting the grayscale image into a binary image according to the binary pixel threshold value, and determining a first similarity between the white image and the binary image; If the first similarity is not less than a first similarity threshold, it is determined that the white screen identification result is a white screen image.

6. The method of claim 1, wherein, The black-and-white abnormal image identification on the screenshot image comprises: According to the layout of the screenshot image, the screenshot image is divided into N unit screenshot images, N being a positive integer; Each unit screenshot image is subjected to overall black-and-white image identification; If it is identified that there is at least one unit screenshot image being a black-and-white abnormal image, it is determined that the screenshot image is a black-and-white abnormal image.

7. The method of claim 1, wherein, The misjudgment detection result comprises non-misjudgment or misjudgment, and the black-and-white screen detection result comprises a normal page or a black-and-white screen page; The method further comprises: If the misjudgment detection result is misjudgment, it is determined that the black-and-white screen detection result of the target page is a normal page; If the misjudgment detection result is non-misjudgment, it is determined that the black-and-white screen detection result of the target page is a black-and-white screen page.

8. The method of claim 1, wherein, When the misjudgment detection result is a detection result determined according to a first misjudgment detection result and a second misjudgment detection result, the method further comprises: The method further comprises: The first misjudgment detection result comprises non-misjudgment or misjudgment, and the first misjudgment detection on the screenshot image comprises: If there is a loading progress bar or a loading icon in the screenshot image, it is determined that the first misjudgment detection result is misjudgment; 9. The method of claim 8, wherein, If there is no loading progress bar and no loading icon in the screenshot image, it is determined that the first misjudgment detection result is non-misjudgment. The method further comprises: A loading icon set is obtained, the loading icon set comprising M loading icons, M being a positive integer; 10. The method of claim 9, wherein, A second similarity between the screenshot image and each loading icon is determined; If M second similarities are all less than a second similarity threshold, it is determined that there is no loading icon in the screenshot image; If there is at least one second similarity not less than the second similarity threshold in M second similarities, it is determined that there is a loading icon in the screenshot image. The second misjudgment detection result comprises non-misjudgment or misjudgment, and the second misjudgment detection on the screenshot image comprises: A misjudgment image set is obtained, the misjudgment image set comprising K misjudgment images; 11. The method of claim 8, wherein, A third similarity between the screenshot image and each misjudgment image is determined; If K third similarities are all less than a third similarity threshold, it is determined that the second misjudgment detection result is non-misjudgment; If there is at least one third similarity not less than the third similarity threshold in K third similarities, it is determined that the second misjudgment detection result is misjudgment; The method further comprises: If the second misjudgment detection result is misjudgment, the screenshot image is added to the misjudgment image set as a misjudgment image. ​ ​ 12. The method of claim 1, wherein the target page is a page of the target program when testing to a preset step in a test case manner; or, the target page is a page of the target program when testing to any step in a random test manner; or, the target page is a page of the target program when blocking for a time duration greater than a time duration threshold in an online running manner. The black-and-white screen detection result includes a black-and-white screen page, and the outputting the black-and-white screen detection result of the target page includes: If the black-and-white screen detection result of the target page is a black-and-white screen page, displaying the screenshot image and displaying a sequence of adjacent images of the screenshot image, the sequence of adjacent images being a sequence of images combined from screenshots of a plurality of pages under the same path as the target page. The black-and-white screen detection result further includes a normal page, and the method further includes:

13. The method of claim 1, wherein, If it is identified that the screenshot image is a normal image, determining that the black-and-white screen detection result of the target page is a normal page. The method includes:

14. The method of claim 13, wherein, an acquisition module configured to acquire a screenshot image, the screenshot image being an image generated by performing a screenshot process on a target page; an identification module configured to perform black-and-white abnormal image identification on the screenshot image; 15. An image processing apparatus characterized by comprising: a detection module configured to, if it is identified that the screenshot image is a black-and-white abnormal image, perform a misjudgment detection on the screenshot image to obtain a misjudgment detection result, the misjudgment detection result being a detection result determined according to one or more of a first misjudgment detection result and a second misjudgment detection result, the first misjudgment detection result being a detection result determined based on an existence judgment of a screenshot image loading mark, and the second misjudgment detection result being a detection result determined based on a matching judgment between the screenshot image and a misjudgment image set; the screenshot image loading mark including a loading progress bar and a loading icon; the detection module is further configured to extract an image corresponding to a loading progress bar region from the screenshot image, take the extracted image as a progress bar image, perform overall black-and-white image identification on the progress bar image, if it is identified that the progress bar image is a black-and-white abnormal image, determine that there is no loading progress bar in the screenshot image, and if it is identified that the progress bar image is a normal image, determine that there is a loading progress bar in the screenshot image; an output module configured to determine a black-and-white screen detection result of the target page according to the misjudgment detection result, and output the black-and-white screen detection result of the target page. The computer program is stored in the computer storage medium, and the computer program includes program instructions, the program instructions being executed by the processor to cause the computer device with the processor to perform the steps of the method of any one of claims 1-14. The computer program / instructions are executed by the processor to implement the steps of the method of any one of claims 1-14. ​ 16. A computer device, comprising: ​ 17. A computer storage medium, comprising, ​ 18. A computer program product comprising computer programs / instructions, characterized in that, ​

Images