Anomaly detection method, device, computer device and storage medium

Abstract

The present disclosure provides an anomaly detection method, apparatus, computer device, and storage medium. Among them, the method includes: obtaining a current interface picture generated during the script test; comparing the text content of the current interface picture with each historical interface picture generated during the script test to determine whether the current interface picture is a text-differential picture that has text content differences from each historical interface picture; in the case where the current interface picture is determined to be a text-differential picture, detecting the abnormal text area of the current interface picture to determine the candidate abnormal text area; comparing the candidate abnormal text area of the current interface picture with the target abnormal text area in terms of regional structure. If there is a candidate abnormal text area with a structural difference in the current interface picture relative to any historical interface picture, it is determined that the current interface picture is a structure-differential picture, and each candidate abnormal text area of the current interface picture is used as the target abnormal text area.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to an anomaly detection method, apparatus, computer device, and storage medium. Background Art

[0002] For the game interfaces of various games, the interface usually includes rich text information. Guiding users to play games based on the text information appearing in the interface can improve the user experience. However, for game interfaces including multi-language type texts, text anomalies often occur, such as text missing, text overlapping, etc. The appearance of abnormal texts seriously affects the user experience. To solve the text anomaly problem in game interfaces, a large number of game interface pictures are usually collected in the testing stage, and then each collected game interface picture is subjected to anomaly recognition, and the text information in the game interface is adjusted according to the recognition result.

[0003] However, since there are often a large number of duplicate pictures in the game interface pictures collected in the testing stage, directly performing anomaly recognition on each collected game interface picture will cause the entire anomaly detection process to take a long time. In addition, a large number of game interface pictures collected in the testing stage will also bring huge storage pressure. Therefore, how to perform duplicate removal processing on the collected game interface pictures has become an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide an anomaly detection method, apparatus, computer device, and storage medium.

[0005] In a first aspect, the embodiments of the present disclosure provide an anomaly detection method, including:

[0006] Obtaining a current interface picture generated during a script test;

[0007] Comparing the text content of the current interface picture with each historical interface picture generated during the script test to determine whether the current interface picture is a text-different picture that has text content differences from each historical interface picture;

[0008] When it is determined that the current interface picture is a text-different picture, detecting an abnormal text area of the current interface picture to determine a candidate abnormal text area;

[0009] Compare the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures. If there are candidate abnormal text regions with structural differences in the current interface picture relative to any historical interface picture, determine that the current interface picture is a structurally different picture, and use each of the candidate abnormal text regions of the current interface picture as the target abnormal text region.

[0010] In a possible implementation manner, compare the text content of the current interface picture with each historical interface picture generated during the script test process to determine whether the current interface picture is a textually different picture with text content differences from each historical interface picture, including:

[0011] Compare the text content of each text region of the current interface picture with the text content of each text region of each historical interface picture respectively. If there are text content difference regions in the current interface picture relative to any historical interface picture, determine that the current interface picture is a textually different picture;

[0012] The text content difference region refers to: for each text region in the historical interface picture, its text similarity with the current interface picture is less than the text similarity threshold.

[0013] In a possible implementation manner, comparing the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures includes:

[0014] Based on the region coordinate information of the candidate abnormal text region in the current interface picture and the region coordinate information of the target abnormal text region in the historical interface picture, determine the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region;

[0015] In the case where the current interface picture does not meet the structural consistency condition, determine that there are candidate abnormal text regions with structural differences in the current interface picture relative to the historical interface picture; the structural consistency condition means that the coordinate overlap rates corresponding to each candidate abnormal text region of the current interface picture are all greater than the set overlap rate threshold.

[0016] In a possible implementation manner, after determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, comparing the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures further includes:

[0017] When the coordinate overlap rates corresponding to all candidate abnormal text regions in the current interface picture are greater than the set overlap rate threshold, respectively determine the text similarity between the text content of each candidate abnormal text region and the text content of the matching target abnormal text region.

[0018] Among all the candidate abnormal text regions, when there is a candidate abnormal text region where the corresponding text similarity is less than the text similarity threshold, determine the candidate abnormal text regions with structural differences in the current interface picture relative to the historical interface picture.

[0019] In a possible implementation manner, determine the text similarity threshold according to the following steps:

[0020] Based on the text length of each text region in the current interface picture, determine the text similarity threshold corresponding to this text region.

[0021] In a possible implementation manner, after determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, it further includes:

[0022] When the coordinate overlap rates corresponding to all candidate abnormal text regions in the current interface picture are greater than the set overlap rate threshold, determine the picture feature similarity between the picture features of the candidate abnormal text region and the matching target abnormal text region.

[0023] When the picture feature similarity corresponding to the candidate abnormal text region is less than the picture feature similarity threshold, determine the candidate abnormal text regions with structural differences in the current interface picture relative to the historical interface picture.

[0024] In a possible implementation manner, the method further includes:

[0025] Based on the determined various structurally different pictures generated during the script test process and the target abnormal text regions in the structurally different pictures, generate an anomaly detection report.

[0026] In a second aspect, an embodiment of the present disclosure further provides an anomaly detection device, including:

[0027] An acquisition module, configured to acquire the current interface picture generated during the script test process;

[0028] A first comparison module, configured to compare the text content of the current interface picture with each historical interface picture generated during the script test process, and determine whether the current interface picture is a text-different picture that has text content differences from all historical interface pictures;

[0029] A detection module, which is configured to, when determining that the current interface picture is a text difference picture, detect abnormal text areas in the current interface picture and determine candidate abnormal text areas;

[0030] A second comparison module, which is configured to compare the regional structures of the candidate abnormal text areas of the current interface picture with the target abnormal text areas of the detected historical interface pictures. If there are candidate abnormal text areas with structural differences in the current interface picture relative to any historical interface picture, it is determined that the current interface picture is a structural difference picture, and each of the candidate abnormal text areas of the current interface picture is used as the target abnormal text area.

[0031] In a possible implementation manner, when the first comparison module compares the text content of the current interface picture with each historical interface picture generated during the script test process to determine whether the current interface picture is a text difference picture with text content differences from each historical interface picture, it is configured to:

[0032] Compare the text content of each text area of the current interface picture with the text content of each text area of each historical interface picture respectively. If there are text content difference areas in the current interface picture relative to any historical interface picture, it is determined that the current interface picture is a text difference picture;

[0033] The text content difference area refers to: for each text area in the historical interface picture, its text similarity with the text area in the current interface picture is less than the text similarity threshold.

[0034] In a possible implementation manner, when the second comparison module compares the regional structures of the candidate abnormal text areas of the current interface picture with the target abnormal text areas of the detected historical interface pictures, it is configured to:

[0035] Based on the regional coordinate information of the candidate abnormal text area in the current interface picture and the regional coordinate information of the target abnormal text area in the historical interface picture, determine the coordinate overlap rate between the candidate abnormal text area and the target abnormal text area;

[0036] In the case where the current interface picture does not meet the structural consistency condition, determine that there are candidate abnormal text areas with structural differences in the current interface picture relative to the historical interface picture; the structural consistency condition means that the coordinate overlap rates corresponding to each candidate abnormal text area of the current interface picture are all greater than the set overlap rate threshold.

[0037] In a possible implementation, after determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, when the second comparison module compares the candidate abnormal text region of the current interface picture with the target abnormal text region of the detected historical interface picture, it is further configured to:

[0038] When the current interface picture satisfies that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, respectively determine the text similarity between the text content of each candidate abnormal text region and the text content of the matching target abnormal text region;

[0039] In the case that there is a candidate abnormal text region with the corresponding text similarity less than the text similarity threshold among each candidate abnormal text region, determine the candidate abnormal text region with a structural difference in the current interface picture relative to the historical interface picture.

[0040] In a possible implementation, the device further includes:

[0041] A determination module, configured to determine the text similarity threshold according to the following steps:

[0042] Based on the text length of each text region of the current interface picture, determine the text similarity threshold corresponding to this text region.

[0043] In a possible implementation, after determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, the second comparison module is further configured to:

[0044] When the current interface picture satisfies that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, determine the picture feature similarity between the picture features of the candidate abnormal text region and the picture features of the matching target abnormal text region;

[0045] In the case that the picture feature similarity corresponding to the candidate abnormal text region is less than the picture feature similarity threshold, determine the candidate abnormal text region with a structural difference in the current interface picture relative to the historical interface picture.

[0046] In a possible implementation, the device further includes:

[0047] A generation module, configured to generate an anomaly detection report based on each structural difference picture generated during the determined script test process and the target abnormal text region in the structural difference picture.

[0048] In a third aspect, an optional implementation of the present disclosure further provides a computer device, including a processor and a memory. The memory stores machine-readable instructions executable by the processor. The processor is configured to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the machine-readable instructions execute the steps in the above first aspect or any possible implementation manner in the first aspect.

[0049] In a fourth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run, it executes the steps in the above first aspect or any possible implementation manner in the first aspect.

[0050] For the effect descriptions of the above anomaly detection device, computer device, and computer-readable storage medium, refer to the descriptions of the above anomaly detection method, which will not be elaborated here.

[0051] The anomaly detection method, device, computer device, and storage medium provided by the embodiments of the present disclosure, after obtaining the current interface picture, first compare the text content of the current interface picture with each historical interface picture generated during the script test process. According to the text content comparison result, it can be accurately determined whether the current interface picture is a text-difference picture. When it is determined that the current interface picture is a text-difference picture, then perform anomaly text region detection on the current interface picture to determine the candidate anomaly text regions; in this way, the repeated detection of some repeated pictures can be reduced, thereby reducing the workload of anomaly text region detection. When the overall text content of the interface pictures is not repeated, it is possible that the structural content of some anomaly text regions is itself repeated. To further reduce the workload of anomaly processing, the candidate anomaly text regions of the current interface picture can be compared with the target anomaly text regions of the detected historical interface pictures in terms of region structure. When there are anomaly text regions with structural differences in the current interface picture relative to any historical interface picture, the current interface picture is considered a structure-difference picture, and finally, anomaly processing can be performed only on these structure-difference pictures. Therefore, the embodiments of the present disclosure can reduce the pressure of anomaly detection and processing and reduce the data storage pressure during the detection and processing process.

[0052] To make the above objects, features, and advantages of the present disclosure more obvious and understandable, the following specific preferred embodiments are given and described in detail in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings required for use in the embodiments. The drawings here are incorporated into the specification and constitute a part of this specification. These drawings show embodiments that conform to the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure and should not be regarded as a limitation on the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0054] Figure 1 Shows a flowchart of an anomaly detection method provided by an embodiment of the present disclosure;

[0055] Figure 2 Shows a specific flowchart of a method for determining whether the current interface picture is a text difference picture provided by an embodiment of the present disclosure;

[0056] Figure 3 Shows a specific implementation flowchart of a method for comparing the regional structures of the current interface picture provided by an embodiment of the present disclosure;

[0057] Figure 4 Shows a schematic diagram of an anomaly detection device provided by an embodiment of the present disclosure;

[0058] Figure 5 Shows a schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure. Detailed implementation manners

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all of them. Usually, the components of the embodiments of the present disclosure described and illustrated here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed, but only represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present disclosure.

[0060] In addition, the terms "first", "second", etc. in the specification, claims, and the above drawings of the embodiments of the present disclosure are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order other than that illustrated or described here.

[0061] As used herein, "a plurality of" or "several" means two or more. "And / or" describes the relationship between associated objects and indicates that three relationships can exist. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates an "or" relationship between the associated objects before and after.

[0062] It has been found through research that for games released both at home and abroad, abnormal text information often appears in the game interface. To avoid the appearance of abnormal text information, it is usually necessary to use irregular test tasks to detect the abnormal text information in the game interface and make timely adjustments and improvements to the game interface based on the detection results. However, for a single test task, a large number of game interface pictures will be collected each time. For example, a single test device can generate 3,600 game interface pictures per hour, and the running test is generally not less than 10 hours. If 10 devices are used for picture collection, a single test task will generate at least 600 * 10 * 10 = 360,000 game interface pictures. The large number of collected game interface pictures not only brings huge storage pressure but also affects the efficiency of abnormal recognition and processing. Therefore, how to deduplicate the game interface pictures collected in the game has become an urgent problem to be solved.

[0063] Based on the above research, the embodiments of the present disclosure perform deduplication processing on the interface pictures generated during a single script test process immediately. Here, the deduplication is for the current script test process, that is, to deduplicate the repeated pictures generated during a single test process. The process of this deduplication processing involves two stages. In the first stage, text content comparison and deduplication are performed on the entire picture. After completing the deduplication in the first stage, the pictures with repeated text content can be removed. At this time, the detection of abnormal text regions can be performed, which can greatly reduce the workload of abnormal detection. After completing the deduplication in the first stage, when the overall text content of the interface pictures is not repeated, there are still some abnormal text regions whose structural content itself is repeated. To reduce the workload of subsequent abnormal processing, the embodiments of the present disclosure also compare the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures in terms of regional structure. When there are abnormal text regions with structural differences in the current interface picture relative to any historical interface picture, the current interface picture is considered a structurally different picture, and finally, only these structurally different pictures are processed for abnormalities.

[0064] Regarding the problems and solutions addressed by the above solutions, they are all the results obtained by the inventors through practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by the embodiments of the present disclosure for the above problems should all be the contributions made by the inventors to the present disclosure during the process of the present disclosure.

[0065] It should be noted that like reference numerals and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.

[0066] For ease of understanding of this embodiment, first, a detailed introduction is given to an anomaly detection method disclosed in the embodiments of the present disclosure. The execution subject of the anomaly detection method provided in the embodiments of the present disclosure is generally a terminal device or other processing device with certain computing capabilities. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a personal digital assistant device (PDA), a handheld device, a computer device, etc. In some possible implementation manners, the anomaly detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0067] The anomaly detection method provided in the embodiments of the present disclosure is described below by taking a computer device as the execution subject as an example.

[0068] As Figure 1 shown, a flowchart of an anomaly detection method provided in the embodiments of the present disclosure may include the following steps:

[0069] S101: Obtain a current interface picture generated during the script test.

[0070] It should be noted that the embodiments of the present disclosure may be applicable to any script test scenario that generates interface pictures. The following mainly takes a game script test scenario as an example. During the game script test, a series of game interface pictures are generated. For each game interface picture generated in the embodiments of the present disclosure, it is automatically collected and immediately de-duplicated. Compared with the batch collection and processing method, the processing is more timely and efficient. The interface picture may include information such as text and animation related to the game. The script test process is the process of running the script, and the current interface picture generated during the script test is the game interface picture currently collected by the running script.

[0071] Specifically, during the script test, the running automation script may continuously collect game interface pictures of the game. For each currently collected game interface picture, it is used as the current interface picture, and then it can be subjected to anomaly detection and processing according to the steps described below. After the current interface picture is processed, the next game interface picture collected automatically can be obtained, and this game interface picture is used as the new current interface picture and anomaly detection is performed. Based on this, anomaly detection can be sequentially performed on each game interface picture generated during the script test.

[0072] S102: Compare the text content of the current interface picture with each historical interface picture generated during the script test process to determine whether the current interface picture is a text-differentiated picture that has text content differences from each historical interface picture.

[0073] Here, the historical interface pictures can be each game interface picture generated during the current script test process before the current interface picture is generated.

[0074] For each script test, after the first current interface picture is generated, the historical interface pictures in this script test are empty. At this time, it can be directly determined that the first current interface picture is a text-differentiated picture. After that, the first current picture can be used as a historical interface picture during this script test process. Then, for each subsequent generated current interface picture, after using the above S102 to determine that the current interface picture is a text-differentiated picture, it can be used as a historical interface picture during the current script test process and stored.

[0075] In specific implementation, the text content of the current interface picture can be compared with each historical interface picture in turn to determine whether there are text content differences (i.e., whether they are repeated) between the text content in the current interface picture and the text content of the historical interface picture being compared currently. If so, it can be determined that the current interface picture is a text-repeated picture, and no subsequent anomaly detection and processing are performed; if not, the text content of the current interface picture and the next historical interface picture can be compared to determine whether there are text content differences between the text content in the current interface picture and the text content of the next historical interface picture, and so on. Based on the comparison of the text content of the current interface picture and each historical interface picture, it can be accurately determined whether the current interface picture is a text-differentiated picture.

[0076] In one embodiment, for S102, it can be implemented according to the following steps:

[0077] Compare the text content of each text region of the current interface picture with the text content of each text region of each historical interface picture. If there are text content difference regions in the current interface picture relative to any historical interface picture, determine that the current interface picture is a text-differentiated picture.

[0078] Among them, the text content difference region refers to: for each text region in the historical interface picture, its text similarity with other text regions is less than the text similarity threshold.

[0079] Here, the text region can be understood as the interface region where consecutive characters are located in the interface. For example, a text line in the interface can correspond to a text region, that is, the text region can be the sub-picture region where the text line is located in the current interface picture.

[0080] In specific implementation, the Optical Character Recognition (OCR) technology can be used to recognize the text in the current interface picture, and determine the text content of each text area in the current interface picture. After that, for each text area in the current interface picture, it can be used as the current text area to be compared respectively, and the current historical interface picture to be compared can be selected from the historical interface pictures. Then, determine the similarity between the text content of the current text area and the text content of each text area in the current historical interface picture in turn. Determine whether there is a similarity not less than the text similarity threshold among the similarities of each picture. If so, it can be determined that the current text area is a repeated text area relative to the current historical interface picture; if not, it can be determined that the current text area is a text content difference area relative to the current historical interface picture.

[0081] In the case where it is determined that the current text area is a repeated text area, the next text area in the current interface picture can be used as the new current text area, and return to the step of determining the similarity between the text content of the current text area and the text content of each text area in the current historical interface picture in turn, until it is determined that the current text area is a text content difference area relative to the current historical interface picture, or until it is determined that each text area in the current interface picture is a repeated text area relative to the current historical picture.

[0082] Furthermore, if it is determined that each text area in the current interface picture is a repeated text area relative to the current historical picture, it can be determined that the current interface picture is a repeated picture, that is, there is already an interface picture in the historical interface pictures that is the same as the current interface picture. Furthermore, the current interface picture can be deleted, and a new current interface picture can be obtained and an anomaly detection can be performed on the new current interface picture.

[0083] If it is determined that the current text area is a text content difference area relative to the current historical picture, it can be determined that there is a text content difference between the current interface picture and the current historical picture. After that, a new current historical picture can be selected from the historical interface pictures, and for each text area in the current interface picture, it can be used as the current text area to be compared respectively. Then, return to execute the step of determining the similarity between the text content of the current text area and the text content of each text area in the current historical picture in turn, until it is determined that the current text area is a text content difference area relative to the current historical picture, or until it is determined that each text area in the current interface picture is a repeated text area relative to the current historical picture.

[0084] Based on the above steps, it can be determined whether there are text content difference regions in the current interface picture relative to each historical interface picture. If so, the current interface picture can be determined as a text-differentiated picture. If not, in the case where each text region in the current interface picture is a repeated text region relative to each text region in any historical interface picture, the current interface picture can be determined as a repeated picture. For example, when the current interface picture has text region 1 and text region 2, and the historical interface picture 1 also has the same text region 1 and text region 2, the current interface picture can be determined as a repeated picture.

[0085] By comparing each text content in the current interface picture with the text content of each text region in each historical interface picture respectively, it can be accurately determined whether there are repeated text contents for each text content in the current interface picture in any historical interface picture. That is, it can be accurately determined whether the current interface picture is different from any historical interface picture, so as to achieve accurate duplicate removal of the current interface picture based on text content. In addition, this method of comparing each text region one by one conforms to the characteristics of text layout in game interfaces (generally, each text region has a corresponding complete semantics). Duplicate removal based on text region comparison will ultimately result in a duplicate removal result that is more in line with actual requirements.

[0086] Exemplarily, taking the current interface picture including text region A and text region B, the historical interface pictures including historical interface picture 1, historical interface picture 2, and historical interface picture 3, historical interface picture 1 including text region 1 and text region 3, historical interface picture 2 including text region 2 and text region 3, historical interface picture 3 including text region 1, text region A being repeated with text region 1, and text region B being repeated with text region 2 as an example, after comparing the text content of the current interface picture with each historical interface picture, it can be determined that text region B in the current interface picture is a text content difference region relative to historical interface picture 1; text region A in the current interface picture is a text content difference region relative to historical interface picture 2; text region B in the current interface picture is a text content difference region relative to historical interface picture 3. Subsequently, it can be determined that there are text content difference regions in the current interface picture relative to any historical interface picture, and then the current interface picture is determined as a text-differentiated picture.

[0087] Continuing with the above example, if the historical interface picture 3 includes text area 1 and text area 2, after comparing the text content of the current interface picture with each historical interface picture, it can be determined that text area A in the current interface picture and text area 1 in the historical interface picture 3 are duplicate text areas, and text area B in the current interface picture and text area 2 in the historical interface picture 3 are duplicate text areas. Subsequently, it can be determined that each text area of the current interface picture is a duplicate text area relative to each text area in the historical interface picture 3, that is, it can be determined that the current interface picture is a duplicate picture.

[0088] In one embodiment, the similarity can be determined based on the edit distance. The greater the edit distance, the lower the similarity. The embodiments of the present disclosure can adopt a length-adaptive edit distance calculation method, that is, based on the size of the text length, the edit distance threshold is adaptively adjusted, that is, the text similarity threshold is adjusted. Specifically, based on the text length of each text area of the current interface picture, the text similarity threshold corresponding to this text area can be determined.

[0089] Here, the text length can be the character length in the text area. For different text lengths, different text similarity thresholds can be preset. For example, when the text length is not greater than the first preset length, the text similarity threshold can be the first preset value; when the text length is greater than the first preset length and less than the second preset length, the text similarity threshold can be the second preset value. Among them, the first preset value is less than the second preset value, and the first preset length is less than the second preset length.

[0090] For example, when the text length is 3 (unit character count) and below, the edit distance threshold can be set to 0, and correspondingly, the text similarity threshold is 100%; for another example, when the text length is 4 - 6 (unit character count) and below, the edit distance threshold can be set to 1, and correspondingly, the text similarity threshold can be the value after converting this edit distance threshold (such as the ratio of the edit distance threshold to the text length).

[0091] In specific implementation, for the current text area that needs to calculate the similarity currently and the text area in the historical interface picture, the text similarity threshold can be determined according to the text length of this current text area.

[0092] Alternatively, it is also possible to first determine whether the text length of the current text area is the same as the text length of the text area in the historical interface image. If so, the preset text similarity threshold corresponding to the text length of the current text area can be used as the text similarity threshold corresponding to the current text area. If the lengths are not the same, the smaller text length among the text length of the current text area and the text length of the text area in the historical interface image can be determined, and the preset text similarity threshold corresponding to the smaller text length can be used as the text similarity threshold corresponding to the current text area.

[0093] In this way, by setting different text similarity thresholds for text areas with different text lengths, the recognition error caused by OCR recognition can be reduced, and the accuracy of the determined text content comparison result can be improved. Among them, the text content comparison result is the result of determining whether the current interface image is a text difference image.

[0094] As Figure 2 shown, it is a schematic flowchart of a specific process for determining whether the current interface image is a text difference image provided by an embodiment of the present disclosure, which may include the following steps: 1. Obtain the current interface image. 2. Determine the text sequence in the current interface image. Among them, the text sequence includes the text content of each text area in the current interface image. 3. Select a current historical image to be compared from the historical interface images. 4. Calculate the similarity between the text sequence in the current interface image and the text sequence in the current historical image. 5. Determine whether there is a text content difference area in the current interface image relative to the current historical image. 6. If not, it can be determined that the current interface image is a duplicate image, feedback the first indication information (such as true) indicating that the current interface image is a duplicate image, and delete the current interface image; 7. If so, determine whether all historical interface images have been traversed. If not, return to the step of selecting a current historical image to be compared from the historical interface images. If so, execute the following step 8. Among them, traversing all historical interface images means that all historical interface images have been selected as the current historical images to be compared. And, during the loop, if it is determined that the current interface image is a duplicate image, the first indication information (such as TURE) can be feedback, and the current interface image can be deleted. 8. After traversing all historical interface images, it can be determined that the current interface image is a text difference image, and at the same time, the second indication information (such as false) indicating that the current interface image is a text difference image can be feedback, store the current interface image as a new historical interface image, and store the text sequence of the current interface image. Further, the following S103 can be used to perform anomaly detection on the current interface image.

[0095] Regarding the specific implementation steps of the above steps 1 to 8, reference can be made to the above embodiments, and details are not described herein again.

[0096] Using the embodiments of the present disclosure, for a single test device that generates 3,600 game interface pictures per hour, when 10 test devices are running simultaneously for no less than 10 hours, at least 360,000 game interface pictures will be generated. After text deduplication using the above steps S101 and S102, the quantity scale of the generated game interface pictures will be reduced from 360,000 to about 10,000, effectively reducing the data scale.

[0097] S103: When it is determined that the current interface picture is a text difference picture, perform abnormal text area detection on the current interface picture to determine the candidate abnormal text area.

[0098] Here, the candidate abnormal text area can be the text area in the current interface picture that meets the preset abnormal conditions. Among them, the preset abnormal conditions can include at least one of text missing, text out-of-frame, text overlap, text translation error, etc. Among them, regarding text out-of-frame: for different texts, there may be corresponding text boxes in the current interface picture. If the text is not completely located within the text box, it can be determined that the text belongs to text out-of-frame, and further, the text area corresponding to the text can be determined as the candidate abnormal text area. Regarding text translation error: since the text language type in the current interface picture can include any one of multiple language types, there may be a problem of text translation error in the text of the current interface picture. For the text area where the text with translation error is located, it can be determined as the candidate abnormal text area.

[0099] Specifically, when it is determined that the current interface picture is a text difference picture, for each text area in the current interface picture, abnormal text area detection can be performed on the text area to determine whether the text area meets the preset abnormal conditions. If so, it can be determined that the text area is the candidate abnormal text area. If not, the candidate abnormal text area can be ignored.

[0100] In a possible implementation manner, if there is no candidate abnormal text area in the current interface picture, the current interface picture can be directly saved for generating an abnormal detection report based on the current interface picture later and facilitating the user to perform abnormal identification based on the current interface picture.

[0101] After abnormal identification, subsequent processing of the abnormal text area can be performed according to the abnormal detection result. Since in the case where the overall text content of the interface picture is not repeated, it is possible that the structural content of some abnormal text areas themselves is repeated. To further reduce the workload of abnormal processing, the following structural deduplication process is performed.

[0102] S104: Compare the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures in terms of regional structure. If there are candidate abnormal text regions with structural differences in the current interface picture compared to any of the historical interface pictures, determine that the current interface picture is a picture with structural differences, and use each candidate abnormal text region of the current interface picture as the target abnormal text region.

[0103] Here, the target abnormal text region is each candidate abnormal text region detected from the historical interface picture. The regional structure can specifically include the picture features of the picture region corresponding to the text region, the text content in the text region, and the regional coordinate information of the text region.

[0104] Exemplarily, the picture feature can be the Perceptual Hash algorithm (Phash) feature value of the current interface picture. Among them, the Phash algorithm is used to calculate the mean hash based on low frequencies, and can generate a fingerprint string for each picture. By comparing the fingerprint strings, the similarity between pictures can be judged.

[0105] Exemplarily, the text region is usually a rectangular region, and the regional coordinate information of the text region can be the coordinate information of each vertex corresponding to the text region.

[0106] In specific implementation, S103 can compare the current interface picture with each historical interface picture in turn according to the candidate abnormal text regions of the current interface picture and the target abnormal text regions of each historical interface picture, and determine whether the candidate abnormal text regions in the current interface picture are structurally repeated with the target abnormal text regions of the historical interface picture being compared currently. If so, it can be determined that the current interface picture is a picture with structural repetition; if not, the current interface picture and the next historical interface picture can be compared in terms of regional structure to determine whether the candidate abnormal text regions in the current interface picture are repeated with the target abnormal text regions of the next historical interface picture, and so on. Based on the regional structure comparison of the current interface picture and each historical interface picture, it can be accurately determined whether the current interface picture is a picture with structural differences.

[0107] Exemplarily, in the case where there are multiple candidate abnormal text regions, such as including candidate abnormal text region 1 and candidate abnormal text region 2, and the historical interface pictures include historical interface picture 1, historical interface picture 2, and historical interface picture 3, the region structure comparison between the current interface picture and historical interface picture 1 can be performed first. Specifically, candidate abnormal text region 1 can be compared with each target abnormal text region of historical interface picture 1 in sequence to determine whether candidate abnormal text region 1 has a repeated structure with each target abnormal text region of historical interface picture 1. If not, the candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 1 can be determined. After that, the region structure comparison between the current interface picture and historical interface picture 2 can be performed. If so, candidate abnormal text region 2 can be compared with each target abnormal text region of historical interface picture 1 in sequence to determine whether candidate abnormal text region 2 has a repeated structure with each target abnormal text region of historical interface picture 1.

[0108] If so, it indicates that there is no candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 1, and the current interface picture and historical interface picture 1 are pictures with a repeated structure, that is, it can be explained that the current interface picture is a picture with a repeated structure, and thus this current interface picture can be ignored; if not, the candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 1 can be determined. After that, the region structure comparison between the current interface picture and historical interface picture 2 can be performed. If it is determined that there is no candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 2, it can be determined that the current interface picture and historical interface picture 2 are pictures with a repeated structure, that is, it can be explained that the current interface picture is a picture with a repeated structure, and thus this current interface picture can be ignored; if it is determined that there is a candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 2, the region structure comparison between the current interface picture and historical interface picture 3 can be performed.

[0109] Further, if it is determined that there is no candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 3, it can be determined that the current interface picture and historical interface picture 3 are pictures with a repeated structure, that is, it can be explained that the current interface picture is a picture with a repeated structure, and thus this current interface picture can be ignored; if it is determined that there is a candidate abnormal text region with a structural difference in the current interface picture relative to historical interface picture 3, it can be explained that there are candidate abnormal text regions with structural differences in the current interface picture relative to historical interface pictures 1, 2, and 3, that is, it can be determined that the current interface picture is a picture with structural differences. Furthermore, each candidate abnormal text region of the current interface picture can be used as a target abnormal text region.

[0110] Here, since the comparison is between pictures (that is, the current interface picture is compared with each historical interface picture respectively), considering the influence of different picture scenarios on text differences and structural differences, once the current interface picture is determined to be a picture with structural differences, each candidate abnormal text region of the current interface picture will be regarded as the target abnormal text region, which may include candidate abnormal text regions with structural differences and / or candidate abnormal text regions without text structural differences.

[0111] Exemplarily, the target abnormal text regions in the historical interface picture 1 are regions 1, 2, and 3, and the candidate abnormal text regions in the current interface picture 1 are regions 2, 4, and 5. When using the historical interface picture 1 to deduplicate the current interface picture 1, even though region 2 is a structurally repeated region of the historical interface picture 1, since regions 4 and 5 are structurally different regions of the historical interface picture 1, it can be determined that the current interface picture 1 is a picture with structural differences. At this time, regions 2, 4, and 5 can all be regarded as the target abnormal text regions. After that, if a new current interface picture 2 including regions 4 and 5 is obtained, when using the historical interface picture 1 and the current interface picture 1 to deduplicate, it can be known that the current interface picture 2 and the combination of the historical interface picture 1 and the current interface picture 1 (which has region 2 more than the current interface picture 2) are both non-repeated. Furthermore, regions 4 and 5 of the current interface picture 2 will be regarded as the new target abnormal text regions and stored. However, for the current interface picture 1, if it stores regions 4 and 5 but not regions 4 and 5 when storing the target abnormal text regions, when using the current interface picture 1 to deduplicate the current interface picture 2, the result will be that the current interface picture 2 and the current interface picture 1 are repeated. Thus, the current interface picture 2 will be deleted, resulting in the filtering of the picture scenario consisting only of regions 4 and 5, affecting the deduplication effect and accuracy.

[0112] In one embodiment, for the step of comparing the regional structures of the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the historical interface picture in S103, during the process of comparing the regional structures of the current interface picture and each historical interface picture, the following steps can be adopted to determine whether there are candidate abnormal text regions with structural differences in the current interface picture relative to the historical interface picture:

[0113] Step 1: Based on the regional coordinate information of the candidate abnormal text region in the current interface picture and the regional coordinate information of the target abnormal text region in the historical interface picture, determine the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region.

[0114] Here, the coordinate overlap rate can specifically refer to the area overlap rate between the candidate abnormal text region and the target abnormal text region. And based on the above, it can be known that each candidate abnormal text region in the current interface picture has corresponding area coordinate information.

[0115] In specific implementation, when comparing the area structures of the current interface picture and any historical interface picture, in the case where the candidate text region includes only one, the coordinate overlap rate between this candidate text region and each target abnormal text region of this historical interface picture can be determined sequentially.

[0116] In the case where the candidate abnormal text regions include multiple ones, one candidate abnormal text region to be verified can be selected at a time, and the coordinate overlap rate between this candidate abnormal text region to be verified and each target abnormal text region of this historical interface picture can be determined. Then, a new candidate abnormal text region to be verified is selected from the candidate abnormal text regions, and the coordinate overlap rate between this new candidate abnormal text region to be verified and each target abnormal text region of this historical interface picture is determined. By analogy, the coordinate overlap rate between each candidate abnormal text region and each target abnormal text region of this historical interface picture can be determined.

[0117] Step 2: In the case where the current interface picture does not meet the structural consistency condition, determine the candidate abnormal text regions with structural differences in the current interface picture relative to the historical interface picture; the structural consistency condition means that the coordinate overlap rates corresponding to each candidate abnormal text region of the current interface picture are all greater than the set overlap rate threshold.

[0118] Here, the coordinate overlap rate corresponding to each candidate abnormal text region is the coordinate overlap rate between this candidate abnormal text region and each target abnormal text region in the historical interface picture.

[0119] Exemplarily, when there is one candidate abnormal text area in the current interface picture, if the coordinate overlap rate between this candidate abnormal text area and each target abnormal text area is less than the set overlap rate threshold, it can be determined that this candidate abnormal text area is a candidate abnormal text area with structural differences relative to this current interface picture. That is, it can be determined that the current interface picture does not meet the structural consistency condition and there is a candidate abnormal text area with structural differences relative to the historical interface picture. Further, the text similarity between the candidate abnormal text area of the current interface picture and the target abnormal text areas in each of the other historical interface pictures can be determined. According to the text similarity, it can be determined whether there is a candidate abnormal text area with structural differences in the current interface picture relative to each historical interface picture. If there are candidate abnormal text areas with structural differences in the current interface picture relative to each historical interface picture, it can be determined that the current interface picture is a structurally different picture. Otherwise, it can be determined that the current interface picture is a structurally repeated picture and the current interface picture can be ignored.

[0120] When there are multiple candidate abnormal text areas in the current interface picture, if there is at least one candidate abnormal text area whose coordinate overlap rate with each target abnormal text area is less than the set overlap rate threshold, it can be determined that the above at least one candidate abnormal text area is a candidate abnormal text area with structural differences relative to this current interface picture. That is, it can be determined that the current interface picture does not meet the structural consistency condition and there is a candidate abnormal text area with structural differences relative to the historical interface picture. Further, the text similarity between the multiple candidate abnormal text areas of the current interface picture and the target abnormal text areas in each of the other historical interface pictures can be determined. According to the text similarity, it can be determined whether there is a candidate abnormal text area with structural differences in the current interface picture relative to each historical interface picture. If there are candidate abnormal text areas with structural differences in the current interface picture relative to each historical interface picture, it can be determined that the current interface picture is a structurally different picture. Otherwise, it can be determined that the current interface picture is a structurally repeated picture, and then the current interface picture can be ignored.

[0121] It can be understood that in specific implementation, when comparing the regional structures of the current interface picture and any historical interface picture, the coordinate overlap rate between each candidate abnormal text area and each target abnormal text area in the historical interface picture can be determined first using Step 1. After obtaining the used coordinate overlap rate, it is then determined whether the current interface picture does not meet the condition that the coordinate overlap rate corresponding to each candidate abnormal text area is greater than the set overlap rate threshold.

[0122] Alternatively, in the case where there are multiple candidate abnormal text regions, one candidate abnormal text region can be determined, and the coordinate overlap rates between it and each target abnormal text region can be determined. Then, it can be determined whether the corresponding coordinate overlap rates are all less than the set overlap rate threshold. If so, it can be determined that there is a candidate abnormal text region with a structural difference in the current interface image relative to the historical interface image. If not, a new candidate abnormal text region can be selected, and the coordinate overlap rates between this new candidate abnormal text region and each target abnormal text region can be determined. Then, it can be determined whether the corresponding coordinate overlap rates are all less than the set overlap rate threshold, and then a judgment result on whether there is a candidate abnormal text region with a structural difference in the current interface image relative to the historical interface image can be obtained. Through continuous looping, until a judgment result on a candidate abnormal text region with a structural difference in the current interface image relative to the historical interface image is determined; or until each candidate abnormal text region in the multiple candidate abnormal text regions is traversed, and a judgment result that there is no candidate abnormal text region with a structural difference in the current interface image relative to the historical interface image is obtained. After that, when comparing the regional structures of the current interface image and the new historical interface image, until a judgment result on a candidate abnormal text region with a structural difference in the current interface image relative to the historical interface image is determined, or until each historical interface image is traversed, and a judgment result that there is no candidate abnormal text region with a structural difference in the current interface image relative to each historical interface image is obtained.

[0123] In this way, through the coordinate overlap rates between text regions, it can be determined whether the layout positions of the text regions in the current interface image match those in the historical interface image. If the layout positions of the text regions do not match themselves, it can be considered that there is a structural difference between the current interface image and the historical interface image, so there is no need to perform subsequent further text content comparison or image feature comparison.

[0124] In one embodiment, to further improve the accuracy of the judgment result of "whether there is a candidate abnormal text region with a structural difference in the current interface image relative to the historical interface image", after a preliminary judgment using the coordinate overlap rate, text similarity can be used for further judgment.

[0125] Specifically, after determining the coordinate overlap rates between the candidate abnormal text regions and the target abnormal text regions, if it is determined that the current interface image satisfies that the coordinate overlap rates corresponding to each candidate abnormal text region are all greater than the set overlap rate threshold, the text similarities between the text contents of each candidate abnormal text region and the text contents of the matching target abnormal text regions can be determined respectively.

[0126] Here, regarding the text content of the candidate abnormal text region, the text content of the candidate abnormal text region recognized by using the OCR technology can be directly used as the text content for region structure comparison. Alternatively, it is also possible to first determine the text similarity between the text content of the candidate abnormal text region recognized by using the OCR technology and the text content of each target abnormal text region, and use the text content of the target abnormal text region corresponding to the maximum text similarity as the final text content corresponding to the candidate abnormal text region. After that, the final text content can be used for region structure comparison.

[0127] The matching target abnormal text region can be a target abnormal text region in the historical interface picture currently used for region structure comparison, and the coordinate overlap rate between this target abnormal text region and the candidate abnormal text region for which text similarity needs to be calculated currently is greater than the set overlap rate threshold. Exemplarily, the coordinate overlap rate between candidate abnormal text region 1 and target abnormal text region A in historical interface picture 1 is greater than the set overlap rate threshold, and the coordinate overlap rate between candidate abnormal text region 2 and target abnormal text region B in historical interface picture 1 is greater than the set overlap rate threshold. Then, the target abnormal text region matching candidate abnormal text region 1 is target abnormal text region A, and the target abnormal text region matching candidate abnormal text region 2 is target abnormal text region B.

[0128] In specific implementation, for the historical interface picture and the current interface picture currently used for region structure comparison, when the coordinate overlap rate corresponding to each candidate abnormal text region in the current interface picture is greater than the set overlap rate threshold, for each candidate abnormal text region in the current interface picture, the length adaptive edit distance calculation method (see the above process for calculating the text similarity of the interface picture based on the edit distance) can be used to determine the text similarity between the text content of the candidate abnormal text region and the text content of the matching target abnormal text region.

[0129] Furthermore, in the case that there is a candidate abnormal text region with a corresponding text similarity less than the text similarity threshold among the candidate abnormal text regions, determine the candidate abnormal text regions where the current interface picture has a structural difference relative to the historical interface picture.

[0130] In specific implementation, after determining the text similarity between the text content of the candidate abnormal text region of the current interface picture and the text content of the matching target abnormal text region, it can be determined whether the text similarity is less than the text similarity threshold corresponding to the candidate abnormal text region. If so, it can be determined that the candidate abnormal text region has a structural difference relative to the historical interface picture, and further, it can be determined that the candidate abnormal text region with a structural difference exists in the current interface picture relative to the historical interface picture. Among them, the text similarity threshold corresponding to the candidate abnormal text region can be determined according to the text length of the text content of the candidate abnormal text region.

[0131] If not, it can be determined whether the text similarity between the text content of the next candidate abnormal text region in the current interface picture and the text content of the matching target abnormal text region is less than the text similarity threshold corresponding to the next candidate abnormal text region. Through continuous loop judgment, it can be determined whether there is a candidate abnormal text region with a corresponding text similarity less than the text similarity threshold among all candidate abnormal text regions. If not, it can be determined that there is no candidate abnormal text region with a structural difference in the current interface picture relative to each historical interface picture, and the current interface picture can be deleted. If so, it can be determined that the candidate abnormal text region with a structural difference exists in the current interface picture relative to the historical interface picture.

[0132] Exemplarily, when the coordinate overlap rate between the candidate abnormal text region 1 and the target abnormal text region A in the historical interface picture 1 is greater than the set overlap rate threshold, and the coordinate overlap rate between the candidate abnormal text region 2 and the target abnormal text region B in the historical interface picture 1 is greater than the set overlap rate threshold, the text similarity 1 between the text content of the current candidate abnormal text region 1 and the text content of the target abnormal text region A can be determined, and the text similarity 2 between the text content of the candidate abnormal text region 2 and the text content of the target abnormal text region B can be determined. When it is determined that the text similarity 1 is less than the text similarity threshold 1 corresponding to the candidate abnormal text region 1, or when it is determined that the text similarity 2 is less than the text similarity threshold 2 corresponding to the candidate abnormal text region 2, it can be determined that the candidate abnormal text region with a structural difference exists in the current interface picture relative to the historical interface picture 1. When it is determined that the text similarity 1 is not less than the text similarity threshold 1 and the text similarity 2 is not less than the text similarity threshold 2, it is determined that there is no candidate abnormal text region with a structural difference in the current interface picture relative to the historical interface picture 1.

[0133] In this way, by using the coordinate overlap rate first and then the text similarity to compare the regional structures, the accuracy of the judgment result of whether the current interface picture is a structurally different picture can be improved.

[0134] In another embodiment, in order to further improve the accuracy of the judgment result of the "candidate abnormal text region where there is a structural difference between the current interface picture and the historical interface picture", after the preliminary judgment using the coordinate overlap rate, the picture feature similarity can be further used for re - judgment.

[0135] Specifically, after determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, if it is determined that the current interface picture satisfies that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, the picture feature similarity between the picture features of the candidate abnormal text region and the matching target abnormal text region can be determined.

[0136] Exemplarily, for the historical interface picture and the current interface picture currently used for regional structure comparison, when the current interface picture satisfies that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, for each candidate abnormal text region of the current interface picture, the picture feature similarity between the image feature corresponding to the candidate abnormal text region and the picture features of the matching target abnormal text region can be determined by a preset image feature similarity calculation method (such as the Hamming distance calculation method).

[0137] Furthermore, in the case where the picture feature similarity corresponding to the candidate abnormal text region is less than the picture feature similarity threshold, it is determined that the candidate abnormal text region where there is a structural difference between the current interface picture and the historical interface picture.

[0138] Here, the picture feature similarity threshold can be the picture feature similarity threshold corresponding to the candidate abnormal text region. The picture feature similarity threshold corresponding to the candidate abnormal text region can be the same as the text similarity threshold corresponding to the candidate abnormal text region, or can be a preset threshold. The embodiments of the present disclosure do not make limitations.

[0139] In specific implementation, after determining the picture feature similarity between the picture features of the candidate abnormal text region and the matching target abnormal text region, it can be determined whether the picture feature similarity is less than the picture feature similarity threshold corresponding to the candidate abnormal text region. If so, it can be determined that the candidate abnormal text region is a candidate abnormal text region with a structural difference relative to the historical interface picture. Furthermore, that is, it can be determined that the candidate abnormal text region where there is a structural difference between the current interface picture and the historical interface picture.

[0140] If not, it is possible to determine whether the similarity of the image features between the image features of the next candidate abnormal text region in the current interface image and the image features of the target abnormal text region that matches it is less than the image feature similarity threshold corresponding to the next candidate abnormal text region. Through continuous loop judgment, it is possible to determine whether there is a candidate abnormal text region in each candidate abnormal text region whose corresponding image feature similarity is less than the image feature similarity threshold. If not, it can be determined that there is no candidate abnormal text region with structural differences in the current interface image relative to each historical interface image, and the current interface image can be deleted. If so, it can be determined that there is a candidate abnormal text region with structural differences in the current interface image relative to each historical interface image.

[0141] Exemplarily, when the coordinate overlap rate between the candidate abnormal text region 1 and the target abnormal text region A in the historical interface image 1 is greater than the set overlap rate threshold, and the coordinate overlap rate between the candidate abnormal text region 2 and the target abnormal text region B in the historical interface image 1 is greater than the set overlap rate threshold, it is possible to determine the image feature similarity 1 between the image features of the current candidate abnormal text region 1 and the image features of the target abnormal text region A, and determine the image feature similarity 2 between the image features of the candidate abnormal text region 2 and the image features of the target abnormal text region B. When it is determined that the image feature similarity 1 is less than the image feature similarity threshold 1 corresponding to the candidate abnormal text region 1, or when it is determined that the image feature similarity 2 is less than the image feature similarity threshold 2 corresponding to the candidate abnormal text region 2, it can be determined that there is a candidate abnormal text region with structural differences in the current interface image relative to the historical interface image 1. When it is determined that the image feature similarity 1 is not less than the image feature similarity threshold 1 and the image feature similarity 2 is not less than the image feature similarity threshold 2, it is determined that there is no candidate abnormal text region with structural differences in the current interface image relative to the historical interface image 1.

[0142] In this way, by first using the coordinate overlap rate and then using the image feature similarity to compare the regional structures, the accuracy of the judgment result of whether the current interface image is a structurally different image can be improved.

[0143] Of course, when determining "whether there is a candidate abnormal text region with structural differences in the current interface image relative to the historical interface image", it is also possible to first use the coordinate overlap rate and then use the text similarity and image feature similarity to determine. In this way, the accuracy of the judgment result can be further improved.

[0144] Such as Figure 3As shown in the figure, it is a specific implementation flowchart of a method for comparing the regional structures of pictures on the current interface provided by an embodiment of the present disclosure, which may include the following steps: a. Obtain the regional coordinate information corresponding to each candidate abnormal text region in the picture of the current interface. b. Obtain the text content corresponding to each candidate abnormal text region. c. Obtain the picture features corresponding to each candidate abnormal text region. d. Select a historical interface picture to be compared from the historical interface pictures. Here, determine the regional coordinate information corresponding to each target abnormal text region in the historical interface picture, the text content corresponding to each candidate abnormal text region, and the picture features corresponding to each candidate abnormal text region. e. Determine the coordinate overlap rate between each candidate abnormal text region and each target abnormal text region. f. Determine whether the coordinate overlap rate corresponding to each candidate abnormal text region of the picture of the current interface is greater than the set overlap rate threshold (that is, determine whether the structural consistency condition is met). If not, determine whether all historical interface pictures have been traversed. If not, return to the step of selecting a historical interface picture to be compared from the historical interface pictures; if so, output the picture of the current interface as a structurally different picture; where traversing all historical interface pictures means that all historical interface pictures have been selected as the current historical pictures to be compared. If so (that is, all are greater than the set overlap rate threshold), there can be two different judgment methods. Among them, Method 1 includes the following steps g to step j; Method 2 includes the following steps k to step n.

[0145] g. Respectively determine the text similarity between the text content of each candidate abnormal text region and the text content of the matching target abnormal text region. h. Determine whether there is a candidate abnormal text region with a corresponding text similarity less than the text similarity threshold among the candidate abnormal text regions. i. If so, determine whether all historical interface pictures have been traversed. If not, return to the step of selecting a historical interface picture to be compared from the historical interface pictures; if so, output the picture of the current interface as a structurally different picture. j. If not (that is, there is no candidate abnormal text region with a corresponding text similarity less than the text similarity threshold), determine that the picture of the current interface is a structurally repeated picture and delete the picture.

[0146] k. Determine the similarity of image features between the image features of each candidate abnormal text region and the image features of the matching target abnormal text region. l. Determine whether the similarity of image features is less than the threshold of the image feature similarity corresponding to the candidate abnormal text region. m. If so, determine whether all historical interface images have been traversed. If not, return to the step of selecting a historical interface image to be compared from the historical interface images; if so, output the current interface image as a structurally different image. n. If not, that is, when it is determined that the similarity of the image features between the image features of each candidate abnormal text region and the image features of the matching target abnormal text region is greater than the corresponding threshold of the image feature similarity, determine that the current interface image is a structurally repeated image. Among them, the solutions corresponding to steps k to n are shown by a dashed line in Figure 3 as shown in

[0147] For the specific implementation steps of the above steps a to n, reference can be made to the above embodiments and will not be elaborated here.

[0148] Exemplarily, by using the steps of S103 and S104 above to perform structural deduplication on the current interface image, the data scale of the determined structurally different images can be reduced from 500 to 50, achieving effective deduplication of the current interface image and obtaining lighter data.

[0149] In this way, after obtaining the current interface image, through a two-step judgment method of first comparing the text content and then comparing the regional structure, it can be accurately determined whether the current interface image is a duplicate image, and then the deduplication process of the current interface image can be accurately and timely completed. That is, the cleaning of image data is realized, effectively reducing the data scale of the structurally different images that need to be detected for anomalies subsequently, and saving the labor cost of manual inspection.

[0150] Exemplarily, by using the anomaly detection method provided in the embodiments of the present disclosure, by responsively and automatically collecting the current interface image and performing text deduplication and structural deduplication on the image, for a single test task, the workload of 1 day / person can be saved.

[0151] In one embodiment, after using the anomaly detection methods provided in the above embodiments to perform anomaly detection on each current interface image generated during the script test process and determining each structurally different image in each current interface image, an anomaly detection report can also be generated based on the determined structurally different images generated during the script test process and the target abnormal text regions in the structurally different images.

[0152] Exemplarily, the anomaly detection report may include various structural difference diagrams, as well as the target anomaly text regions in each structural difference picture. After generating the anomaly detection report, the anomaly detection report can be fed back to the detector, so that the detector can separately perform anomaly recognition on each structural difference diagram, determine each piece of anomaly text information, and based on the anomaly text information, adjust the anomaly text information in the game interface picture in the game, thereby realizing the update and improvement of the game.

[0153] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order that constitutes any limitation on the implementation process, and the specific execution order of each step should be determined according to its function and possible internal logic.

[0154] Based on the same inventive concept, an anomaly detection device corresponding to the anomaly detection method is also provided in the embodiments of the present disclosure. Since the principle of solving problems by the device in the embodiments of the present disclosure is similar to the above anomaly detection method in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0155] As Figure 4 shown, it is a schematic diagram of an anomaly detection device provided by an embodiment of the present disclosure, including:

[0156] An acquisition module 401, configured to acquire the current interface picture generated during the script test process;

[0157] A first comparison module 402, configured to compare the text content of the current interface picture with each historical interface picture generated during the script test process, and determine whether the current interface picture is a text difference picture that has text content differences from each historical interface picture;

[0158] A detection module 403, configured to, when it is determined that the current interface picture is a text difference picture, perform anomaly text region detection on the current interface picture to determine candidate anomaly text regions;

[0159] A second comparison module 404, configured to perform regional structure comparison on the candidate anomaly text regions of the current interface picture and the target anomaly text regions of the detected historical interface pictures. If there are candidate anomaly text regions with structural differences in the current interface picture relative to any historical interface picture, it is determined that the current interface picture is a structural difference picture, and each of the candidate anomaly text regions of the current interface picture is used as the target anomaly text region.

[0160] In a possible implementation, when the first comparison module 402 compares the text content of the current interface picture with each historical interface picture generated during the script test process to determine whether the current interface picture is a text difference picture with text content differences from each historical interface picture, it is used for:

[0161] Compare the text content of each text region of the current interface picture with the text content of each text region of each of the historical interface pictures respectively. If there are text content difference regions in the current interface picture relative to any historical interface picture, determine that the current interface picture is a text difference picture;

[0162] The text content difference region refers to: for each text region in the historical interface picture, its text similarity with other regions is less than the text similarity threshold.

[0163] In a possible implementation, when the second comparison module 404 compares the candidate abnormal text region of the current interface picture with the target abnormal text region of the detected historical interface picture in terms of region structure, it is used for:

[0164] Based on the region coordinate information of the candidate abnormal text region in the current interface picture and the region coordinate information of the target abnormal text region in the historical interface picture, determine the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region;

[0165] In the case where the current interface picture does not meet the structural consistency condition, determine the candidate abnormal text region with structural differences in the current interface picture relative to the historical interface picture; the structural consistency condition means that the coordinate overlap rate corresponding to each candidate abnormal text region of the current interface picture is greater than the set overlap rate threshold.

[0166] In a possible implementation, after determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, when the second comparison module 404 compares the candidate abnormal text region of the current interface picture with the target abnormal text region of the detected historical interface picture in terms of region structure, it is also used for:

[0167] In the case where the current interface picture meets the condition that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, determine the text similarity between the text content of each candidate abnormal text region and the text content of the matching target abnormal text region respectively;

[0168] In the case that there is a candidate abnormal text area with a corresponding text similarity less than the text similarity threshold in each of the candidate abnormal text areas, determine the candidate abnormal text area where the current interface picture has a structural difference relative to the historical interface picture.

[0169] In a possible implementation manner, the apparatus further includes:

[0170] A determination module 405, configured to determine the text similarity threshold according to the following steps:

[0171] Based on the text length of each text area of the current interface picture, determine the text similarity threshold corresponding to this text area.

[0172] In a possible implementation manner, after determining the coordinate overlap rate between the candidate abnormal text area and the target abnormal text area, the second comparison module 404 is further configured to:

[0173] In the case that the current interface picture satisfies that the coordinate overlap rates corresponding to each candidate abnormal text area are all greater than the set overlap rate threshold, determine the picture feature similarity between the picture features of the candidate abnormal text area and the picture features of the matching target abnormal text area;

[0174] In the case that the picture feature similarity corresponding to the candidate abnormal text area is less than the picture feature similarity threshold, determine the candidate abnormal text area where the current interface picture has a structural difference relative to the historical interface picture.

[0175] In a possible implementation manner, the apparatus further includes:

[0176] A generation module 406, configured to generate an anomaly detection report based on each structural difference picture generated during the determined script test process and the target abnormal text area in the structural difference picture.

[0177] The description of the processing flow of each module in the apparatus and the interaction flow between modules can refer to the relevant descriptions in the above method embodiments and will not be elaborated here.

[0178] Based on the same inventive concept, an embodiment of the present application further provides a computer device. Refer to Figure 5 As shown, it is a schematic structural diagram of a computer device provided by an embodiment of the present application, including:

[0179] A processor 51, a memory 52, and a bus 53. Among them, the memory 52 stores machine-readable instructions executable by the processor 51. The processor 51 is configured to execute the machine-readable instructions stored in the memory 52. When the machine-readable instructions are executed by the processor 51, the processor 51 performs the following steps: S101: Obtain the current interface picture generated during the script test; S102: Compare the text content of the current interface picture with each historical interface picture generated during the script test to determine whether the current interface picture is a text-differential picture that has text content differences from each historical interface picture; S103: In the case where it is determined that the current interface picture is a text-differential picture, perform abnormal text area detection on the current interface picture to determine the candidate abnormal text area, and S104: Compare the candidate abnormal text area of the current interface picture with the target abnormal text area of the detected historical interface picture. If there is a candidate abnormal text area with a structural difference in the current interface picture relative to any historical interface picture, determine that the current interface picture is a structural-differential picture, and use each candidate abnormal text area of the current interface picture as the target abnormal text area.

[0180] The above-mentioned memory 52 includes a memory 521 and an external memory 522; the memory 521 here is also called an internal memory, which is used to temporarily store the operation data in the processor 51 and the data exchanged with the external memory 522 such as a hard disk. The processor 51 exchanges data with the external memory 522 through the memory 521. When the computer device is running, the processor 51 communicates with the memory 52 through the bus 53, so that the processor 51 executes the execution instructions mentioned in the above method embodiments.

[0181] The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the abnormal detection method described in the above method embodiments. Among them, the storage medium can be a volatile or non-volatile computer-readable storage medium.

[0182] The computer program product of the abnormal detection method provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the abnormal detection method described in the above method embodiments. For details, please refer to the above method embodiments and will not be elaborated here.

[0183] The computer program product can be specifically implemented in a way of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0184] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working processes of the devices described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined, or some features can be ignored, or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some communication interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, in each embodiment of the present disclosure, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0187] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0188] If the technical solution of this application involves personal information, before the product applying the technical solution of this application processes personal information, it has clearly informed the personal information processing rules and obtained the individual's independent consent. If the technical solution of this application involves sensitive personal information, before the product applying the technical solution of this application processes sensitive personal information, it has obtained the individual's separate consent and at the same time meets the requirements of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If an individual voluntarily enters the collection scope, it is regarded as consenting to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are informed by obvious signs / information, personal authorization is obtained through pop-up messages or by asking the individual to upload their personal information by themselves; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0189] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, rather than limiting them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions recorded in the foregoing embodiments or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. An anomaly detection method, characterized in that, Including: Obtaining the current interface picture generated during the script test; Comparing the text content of the current interface picture with the text content of each historical interface picture generated during the script test to determine whether the current interface picture is a text-differentiated picture that has text content differences from each historical interface picture; When it is determined that the current interface picture is a text-differentiated picture, detecting abnormal text regions in the current interface picture to determine candidate abnormal text regions; Comparing the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures in terms of regional structure. If there are candidate abnormal text regions with structural differences in the current interface picture relative to any historical interface picture, it is determined that the current interface picture is a structurally-differentiated picture, and each of the candidate abnormal text regions of the current interface picture is used as the target abnormal text region.

2. The method according to claim 1, wherein Comparing the text content of the current interface picture with the text content of each historical interface picture generated during the script test to determine whether the current interface picture is a text-differentiated picture that has text content differences from each historical interface picture, including: Comparing the text content of each text region of the current interface picture with the text content of each text region of each historical interface picture respectively. If there are text content difference regions in the current interface picture relative to any historical interface picture, it is determined that the current interface picture is a text-differentiated picture; The text content difference region refers to: for each text region in the historical interface picture, its text similarity with other text regions is less than the text similarity threshold.

3. The method according to claim 1, characterized in that, Comparing the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures in terms of regional structure, including: Based on the regional coordinate information of the candidate abnormal text region in the current interface picture and the regional coordinate information of the target abnormal text region in the historical interface picture, determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region; When the current interface picture does not meet the structural consistency condition, determining that there are candidate abnormal text regions with structural differences in the current interface picture relative to the historical interface picture; the structural consistency condition means that the coordinate overlap rate corresponding to each candidate abnormal text region of the current interface picture is greater than the set overlap rate threshold.

4. The method according to claim 3, characterized in that, After determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, comparing the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures in terms of regional structure, further including: When the current interface picture meets the condition that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, respectively determining the text similarity between the text content of each candidate abnormal text region and the text content of the matching target abnormal text region. In the case that there is a candidate abnormal text region with the corresponding text similarity less than the text similarity threshold in each of the candidate abnormal text regions, determine the candidate abnormal text region where the current interface picture has a structural difference relative to the historical interface picture.

5. The method according to claim 2 or 4, characterized in that, Determine the text similarity threshold according to the following steps: Based on the text length of each text region of the current interface picture, determine the text similarity threshold corresponding to this text region.

6. The method according to claim 3, wherein After determining the coordinate overlap rate between the candidate abnormal text region and the target abnormal text region, it further includes: In the case that the current interface picture satisfies that the coordinate overlap rate corresponding to each candidate abnormal text region is greater than the set overlap rate threshold, determine the picture feature similarity between the picture features of the candidate abnormal text region and the matching target abnormal text region. In the case that the picture feature similarity corresponding to the candidate abnormal text region is less than the picture feature similarity threshold, determine the candidate abnormal text region where the current interface picture has a structural difference relative to the historical interface picture.

7. The method according to claim 1, wherein The method further includes: Based on the determined various structural difference pictures generated during the script test process and the target abnormal text regions in the structural difference pictures, generate an anomaly detection report.

8. An anomaly detection device, characterized in that, It includes: An acquisition module, configured to acquire the current interface picture generated during the script test process. A first comparison module, configured to compare the text content of the current interface picture with each historical interface picture generated during the script test process, and determine whether the current interface picture is a text difference picture that has text content differences with each historical interface picture. A detection module, configured to, in the case that it is determined that the current interface picture is a text difference picture, perform anomaly text region detection on the current interface picture to determine candidate abnormal text regions. A second comparison module, configured to compare the candidate abnormal text regions of the current interface picture with the target abnormal text regions of the detected historical interface pictures in terms of regional structure. If there are candidate abnormal text regions with structural differences in the current interface picture relative to any historical interface picture, determine that the current interface picture is a structural difference picture, and use each of the candidate abnormal text regions of the current interface picture as the target abnormal text region.

9. A computer device, characterized in that, It includes: A processor and a memory. The memory stores machine-readable instructions executable by the processor. The processor is configured to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the steps of the anomaly detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by a computer device, the computer device executes the steps of the anomaly detection method according to any one of claims 1 to 7.

Images