Automatic testing method, device and equipment, readable storage medium and program product

Through image recognition and text recognition processing, the accuracy and efficiency of UI automation testing are improved, and the problem of low accuracy of test results in the prior art is solved.

CN118939523APending Publication Date: 2024-11-12TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310526683.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The test results of existing UI automation tests are low in accuracy, resulting in reduced efficiency.

Method used

By obtaining screenshots of the running results of the test cases, using image recognition processing to determine the page type, perform element analysis and text recognition, calculate the similarity between the running results and the expected results, and determine the test results based on the similarity threshold.

Benefits of technology

It improves the accuracy and efficiency of test results and improves the overall effectiveness of automated testing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939523A_ABST
    Figure CN118939523A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an automatic testing method and device, equipment, a readable storage medium and a program product, and relates to the fields of artificial intelligence, maps and the like, and application scenes include but are not limited to automatic testing scenes. The method comprises the steps of determining a page type corresponding to a running result screenshot through image recognition processing based on the running result screenshot of a test case; based on the page type corresponding to the running result screenshot, element analysis processing is conducted on the running result screenshot, image features corresponding to the running result screenshot are determined, text recognition processing is conducted on the running result screenshot, and text features corresponding to the running result screenshot are determined; based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot and a preset expected running result of the test case, determining the similarity between the running result screenshot and the expected running result of the test case; and based on the similarity and a preset similarity threshold, performing corresponding processing on the test case.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an automated testing method, device, equipment, readable storage medium, and program product. Background Art

[0002] In the prior art, automated testing generally refers to the automation of software testing. Software testing is to run a system or application under preset conditions and evaluate the running results. The preset conditions include normal conditions and abnormal conditions. Automated testing, for example, UI (User Interface) automated testing. UI automated testing can be a test for WEB (World Wide Web) pages and APP (Application). Since UI automated testing determines the test results of UI automated testing through assertions based on traditional algorithms, the accuracy of the test results of UI automated testing is often low, thereby reducing the efficiency of UI automated testing. Summary of the invention

[0003] In view of the shortcomings of the existing methods, the present application proposes an automated testing method, device, equipment, computer-readable storage medium and computer program product to solve the problem of how to improve the efficiency of UI automated testing.

[0004] In a first aspect, the present application provides an automated testing method, comprising:

[0005] Get a screenshot of the test case's running results;

[0006] Based on the test case running result screenshot, determine the page type corresponding to the running result screenshot through image recognition processing;

[0007] Based on the page type corresponding to the running result screenshot, perform element analysis processing on the running result screenshot to determine the image features corresponding to the running result screenshot, and perform text recognition processing on the running result screenshot to determine the text features corresponding to the running result screenshot;

[0008] Determine the similarity between the running result screenshot and the expected running result of the test case based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, and the preset expected running result of the test case;

[0009] Based on the similarity and the preset similarity threshold, the test case is processed accordingly.

[0010] In one embodiment, based on the screenshot of the running result of the test case, determining the page type corresponding to the screenshot of the running result through image recognition processing includes:

[0011] Based on the pre-trained image recognition model, an image recognition model is obtained through transfer learning;

[0012] The running result screenshot of the test case is input into the image recognition model. Through image recognition processing, the page type corresponding to the running result screenshot is determined. The page type corresponding to the running result screenshot is any one of the pop-up page type, semi-pop-up page type, FAQ page type, face page type, and form page type.

[0013] In one embodiment, based on the pre-trained image recognition model, an image recognition model is obtained through transfer learning, including:

[0014] Update the fully connected layer in the pre-trained image recognition model to obtain an updated model;

[0015] The updated model is fine-tuned to obtain an image recognition model.

[0016] In one embodiment, based on the page type corresponding to the operation result screenshot, element parsing processing is performed on the operation result screenshot to determine the image features corresponding to the operation result screenshot, including:

[0017] Based on the page type corresponding to the running result screenshot, each element in the running result screenshot is detected and segmented to determine the image features corresponding to the running result screenshot. The image features include the type of each element, the position information of each element and the number of elements corresponding to the type of each element.

[0018] In one embodiment, based on the page type corresponding to the operation result screenshot, each element in the operation result screenshot is detected and segmented to determine the image features corresponding to the operation result screenshot, including:

[0019] Through adaptive thresholding, the page background of the page type corresponding to the running result screenshot is denoised to obtain a denoised image;

[0020] Through rectangle detection and Hough transform, detect and segment the elements in the running result screenshot from the denoised image;

[0021] Based on the elements in the segmented running result screenshot, image features corresponding to the running result screenshot are determined.

[0022] In one embodiment, based on the page type corresponding to the operation result screenshot, element parsing processing is performed on the operation result screenshot to determine the image features corresponding to the operation result screenshot, including:

[0023] Based on the page type corresponding to the running result screenshot and the expected running result of the preset test case, the image features corresponding to the running result screenshot are determined through scale-invariant feature transformation feature matching. The image features include the element type of each element, the element position information of each element, and the number of elements corresponding to the element type of each element.

[0024] In one embodiment, based on the page type corresponding to the operation result screenshot, text recognition processing is performed on the operation result screenshot to determine the text features corresponding to the operation result screenshot, including:

[0025] Based on the page type corresponding to the running result screenshot, text recognition processing is performed on the text information in each element of the running result screenshot to determine the text features corresponding to the running result screenshot. The text features corresponding to the running result screenshot include the text information of each element, and each element is any one of a pop-up window, button, check box, icon, and text box.

[0026] In one embodiment, the preset expected running result of the test case includes expected image features and expected text features, and based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, and the preset expected running result of the test case, determining the similarity between the running result screenshot and the expected running result of the test case includes:

[0027] Perform feature comparison processing on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, the expected image features and the expected text features to determine the similarity between the running result screenshot and the expected running result of the test case,

[0028] Among them, the image features include the element type of each element in the running result screenshot, the element position information of each element, and the number of elements corresponding to the element type of each element; the text features include the text information of each element; the expected image features include the expected element type, the expected element position information, and the number of elements corresponding to the expected element type; and the expected text features include the expected text information.

[0029] In one embodiment, based on the similarity and a preset similarity threshold, corresponding processing is performed on the test case, including any of the following:

[0030] If the similarity is greater than or equal to the preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be passed, and other pages are tested by running the test case;

[0031] If the similarity is less than a preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be failed, and the running of the test case is stopped.

[0032] In a second aspect, the present application provides an automated testing device, comprising:

[0033] The first processing module is used to obtain a screenshot of the running result of the test case;

[0034] The second processing module is used to determine the page type corresponding to the running result screenshot based on the running result screenshot of the test case through image recognition processing;

[0035] A third processing module is used to perform element analysis processing on the operation result screenshot based on the page type corresponding to the operation result screenshot, determine the image features corresponding to the operation result screenshot, and perform text recognition processing on the operation result screenshot to determine the text features corresponding to the operation result screenshot;

[0036] A fourth processing module, for determining the similarity between the running result screenshot and the expected running result of the test case based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot and the preset expected running result of the test case;

[0037] The fifth processing module is used to perform corresponding processing on the test case based on the similarity and a preset similarity threshold.

[0038] In a third aspect, the present application provides an electronic device, including: a processor, a memory, and a bus;

[0039] A bus, used to connect the processor and memory;

[0040] A memory, used for storing operation instructions;

[0041] The processor is used to execute the automated testing method of the first aspect of the present application by calling operation instructions.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which is used to execute the automated testing method of the first aspect of the present application.

[0043] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the automated testing method in the first aspect of the present application when the computer program is executed by a processor.

[0044] The technical solution provided in the embodiments of the present application has at least the following beneficial effects:

[0045] Obtain a screenshot of the running result of the test case; based on the screenshot of the running result of the test case, determine the page type corresponding to the screenshot of the running result through image recognition processing; based on the page type corresponding to the screenshot of the running result, perform element analysis processing on the screenshot of the running result to determine the image features corresponding to the screenshot of the running result, and perform text recognition processing on the screenshot of the running result to determine the text features corresponding to the screenshot of the running result; based on the image features corresponding to the screenshot of the running result, the text features corresponding to the screenshot of the running result and the preset expected running results of the test case, determine the similarity between the screenshot of the running result and the expected running results of the test case; in this way, the accuracy of the similarity between the screenshot of the running result and the expected running results of the test case is improved, so that based on the high-accuracy similarity and the preset similarity threshold, corresponding processing is performed on the test case, thereby improving the accuracy of the test results of the automated test and improving the efficiency of the automated test. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0047] Figure 1 A schematic diagram of the architecture of an automated testing system provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a flow chart of an automated testing method provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of an automated test provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of an automated test provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of an automated test provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of an automated test provided in an embodiment of the present application;

[0053] Figure 7 A schematic diagram of a flow chart of an automated testing method provided in an embodiment of the present application;

[0054] Figure 8 A schematic diagram of the structure of an automated testing device provided in an embodiment of the present application;

[0055] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0057] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates that it is implemented as "A", or implemented as "B", or implemented as "A and B".

[0058] It is understandable that in the specific implementation of this application, data related to automated testing is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0059] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0060] The embodiment of the present application is an automated testing method provided by an identification system, which relates to fields such as artificial intelligence and maps.

[0061] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0062] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0063] Intelligent Traffic System (ITS), also known as Intelligent Transportation System, effectively and comprehensively applies advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) to transportation, service control and vehicle manufacturing, strengthens the connection between vehicles, roads and users, and thus forms a comprehensive transportation system that ensures safety, improves efficiency, improves the environment and saves energy.

[0064] In order to better understand and illustrate the solutions of the embodiments of the present application, some technical terms involved in the embodiments of the present application are briefly explained below.

[0065] Assertion: Assertion is a first-order logic in a program; assertion is a debugging method. If the assertion is not true, the program will terminate execution and give an error message.

[0066] Smoke testing: Smoke testing verifies from an abstract level whether the basic functions of the software have been implemented to determine whether more testing is needed; if the test fails, the software will not be tested further and will be returned directly to the developer.

[0067] Regression testing: Regression testing refers to retesting the old code after modifying it to confirm that the modification has not introduced new errors or caused errors in other codes.

[0068] SIFT feature matching: SIFT (Scale-Invariant Feature Transform) feature matching is an algorithm used for image registration. SIFT feature matching can find key points at different image scales and extract local descriptors of key points for image matching.

[0069] Transfer learning: Transfer learning is a machine learning method that uses the model developed for task A as a starting point and reuses it in the process of developing a model for task B.

[0070] Hough transform: Hough transform is a feature extraction method that is widely used in image analysis, computer vision and digital image processing. Hough transform is used to identify features in objects, such as lines.

[0071] ResNet18 model: The ResNet18 model is a deep learning model. The ResNet18 model has good deep learning performance and is widely used in image classification, natural language processing and other fields.

[0072] Yolo algorithm: The Yolo algorithm uses a separate CNN model to implement end-to-end target detection. The core idea is to use the entire image as the input of the network and directly regress the position of the bounding box and its category in the output layer.

[0073] The solution provided in the embodiments of the present application involves artificial intelligence technology. The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0074] In order to better understand the solution provided by the embodiment of the present application, the solution is described below in conjunction with a specific application scenario.

[0075] In one embodiment, Figure 1 FIG. 1 shows a schematic diagram of the architecture of an automated testing system applicable to an embodiment of the present application. It can be understood that the automated testing method provided in the embodiment of the present application can be applicable to, but not limited to, Figure 1 In the application scenario shown.

[0076] In this example, Figure 1As shown, the architecture of the automated test system in this example may include but is not limited to a simulation terminal, an intelligent recognition module, an experience engine, and a bus. The simulation terminal, the intelligent recognition module, the experience engine, and the bus can be deployed on different servers or on the same server. Among them, the simulation terminal includes functions such as a simulation engine, intelligent interaction, and experience screenshots; the intelligent recognition module includes functions such as image recognition, element parsing, OCR (Optical Character Recognition, text recognition), and feature comparison; the experience engine includes functions such as test case management, parameter management in test cases, test case-related process management, and analysis and display of test results; the bus is a communication channel between the simulation terminal, the intelligent recognition module, and the experience engine, and the bus can send and receive instructions.

[0077] The experience engine sends the test case to the simulation terminal through the bus, the simulation terminal runs the test case, and the simulation terminal sends the screenshot of the running result of the test case to the intelligent recognition module through the bus; the intelligent recognition module receives the screenshot of the running result of the test case through the bus; the intelligent recognition module determines the page type corresponding to the running result screenshot through image recognition processing based on the running result screenshot of the test case; the intelligent recognition module performs element parsing processing on the running result screenshot based on the page type corresponding to the running result screenshot, determines the image features corresponding to the running result screenshot, and performs text recognition processing on the running result screenshot to determine the text features corresponding to the running result screenshot; the intelligent recognition module determines the similarity between the running result screenshot and the expected running result of the test case based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot and the preset expected running result of the test case; the intelligent recognition module determines the test result based on the similarity; the intelligent recognition module sends the test result to the experience engine through the bus; the experience engine analyzes and displays the test result; if the test result is passed, the experience engine sends an instruction through the bus to instruct the simulation terminal to continue running the test case; if the test result is not passed, the experience engine sends an instruction through the bus to instruct the simulation terminal to stop running the test case.

[0078] It is understandable that the above is only an example and this embodiment is not limited here.

[0079] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0080] Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, allowing various application systems to obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0081] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0082] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0083] The so-called artificial intelligence cloud service is generally also called AIaaS (AI as a Service). This is the service mode of a mainstream artificial intelligence platform. Specifically, the AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall: all developers can access and use one or more artificial intelligence services provided by the platform through the API interface. Some senior developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and operate their own cloud artificial intelligence services.

[0084] The above network may include but is not limited to: wired network, wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, Wi-Fi and other networks that realize wireless communication. The specific can also be determined based on the actual application scenario requirements and is not limited here.

[0085] See also Figure 2 , Figure 2 The flowchart of an automated testing method provided by an embodiment of the present application is shown, wherein the method can be executed by any electronic device, such as a server corresponding to an automated testing system; as an optional implementation, the method can be executed by a server corresponding to an automated testing system. For the convenience of description, in the description of some optional embodiments below, the server corresponding to the automated testing system will be used as an example to illustrate the execution subject of the method. Figure 2 As shown, the automated testing method provided in the embodiment of the present application includes the following steps:

[0086] S201, obtaining a screenshot of the running result of the test case.

[0087] Specifically, the test type of the test case can be a smoke test, a regression test, etc. Run the test case to obtain the running result of a certain page; take a screenshot of the running result of the page to obtain a screenshot of the running result of the page, that is, a screenshot of the running result of the test case for the page. Figure 3 The screenshot of the operation result shown in the figure includes three elements, namely a text box and two buttons. The text in the text box reads "The current behavior is suspected to be not operated by me. To ensure the safety of funds, the transaction cannot be completed temporarily. If it is confirmed that it is used by me, you can click "View Solution" to submit the information to remove the restriction"; the two buttons are "Close" and "View Solution". For another example, Figure 1 As shown, a screenshot of the running results of the test case is obtained through the intelligent recognition module.

[0088] S202, based on the running result screenshot of the test case, determine the page type corresponding to the running result screenshot through image recognition processing.

[0089] Specifically, the page type corresponding to the running result screenshot may be a pop-up page type, a semi-pop-up page type, a FAQ (Frequently-Asked Questions) page type, a face page type, a form page type, and the like.

[0090] The purpose of image recognition processing (image recognition) is to identify and distinguish the page type corresponding to the running result screenshot, so that you can focus only on the relevant information of the elements corresponding to the page type. For example, for a pop-up page type, you can only focus on the relevant information of the pop-up window (for example, the text information in the text box of the pop-up window, the button of the pop-up window, and the text information in the button), and ignore the information outside the pop-up window. For another example, for an FAQ page type, you can only focus on the text information in the text area. For another example, for a semi-pop-up page type, you can only focus on the risk warning text information, the button, and the text information in the button. For another example, for a semi-pop-up page type, you can only focus on the text information in the check box, the button, and the text information in the button. For another example, Figure 1 As shown, the page type corresponding to the running result screenshot can be determined through the image recognition function in the intelligent recognition module.

[0091] S203, based on the page type corresponding to the operation result screenshot, perform element analysis processing on the operation result screenshot to determine the image features corresponding to the operation result screenshot, and perform text recognition processing on the operation result screenshot to determine the text features corresponding to the operation result screenshot.

[0092] Specifically, the image features corresponding to the running result screenshot include the type of element, the location information of the element, and the number of elements corresponding to the type of element. The text features corresponding to the running result screenshot include the text information of the element. Elements such as pop-up windows, buttons, check boxes, icons, text boxes, etc. For example, Figure 3 In the screenshot of the running results shown, the element types are text boxes and buttons; the position information of the text box is the coordinates of the text box, and the position information of the button is the coordinates of the button; the number of elements corresponding to the text box is 1, that is, the number of text boxes is 1; the number of elements corresponding to the button is 2, that is, the number of buttons is 2.

[0093] For example, Figure 1 As shown, the running result screenshot can be subjected to element analysis through the analysis and processing function in the intelligent recognition module to determine the image features corresponding to the running result screenshot; the running result screenshot can be subjected to text recognition through the OCR function in the intelligent recognition module to determine the text features corresponding to the running result screenshot.

[0094] S204, determining the similarity between the running result screenshot and the expected running result of the test case based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot and the preset expected running result of the test case.

[0095] Specifically, the expected running result of the preset test case is the user's expectation. If the running result screenshot meets the user's expectation, it means that the test case has passed the test. Similarity is, for example, cosine similarity; for example, Figure 1As shown, the cosine similarity between the running result screenshot and the expected running result of the test case can be determined through the feature comparison function in the intelligent recognition module.

[0096] S205: Based on the similarity and a preset similarity threshold, corresponding processing is performed on the test case.

[0097] Specifically, the corresponding processing for the test case includes testing other pages by running the test case, stopping running the test case, etc.

[0098] In an embodiment of the present application, a screenshot of the running result of the test case is obtained; based on the screenshot of the running result of the test case, the page type corresponding to the screenshot of the running result is determined through image recognition processing; based on the page type corresponding to the screenshot of the running result, the screenshot of the running result is subjected to element parsing processing to determine the image features corresponding to the screenshot of the running result, and the screenshot of the running result is subjected to text recognition processing to determine the text features corresponding to the screenshot of the running result; based on the image features corresponding to the screenshot of the running result, the text features corresponding to the screenshot of the running result and the preset expected running result of the test case, the similarity between the screenshot of the running result and the expected running result of the test case is determined; in this way, the accuracy of the similarity between the screenshot of the running result and the expected running result of the test case is improved, so that based on the high-accuracy similarity and the preset similarity threshold, the test case is processed accordingly, the accuracy of the test results of the automated test is improved, and the efficiency of the automated test is improved.

[0099] In one embodiment, based on the screenshot of the running result of the test case, the page type corresponding to the screenshot of the running result is determined through image recognition processing, including steps A1-A2:

[0100] Step A1, based on the pre-trained image recognition model, obtain the image recognition model through transfer learning.

[0101] Specifically, for example, pre-trained models on large data sets such as ImageNet (such as pre-trained image recognition models) already have very good feature extraction capabilities; therefore, it is only necessary to transform the fully connected layers in the pre-trained models to adapt to actual business scenarios; for example, the pre-trained image recognition model is a ResNet18 model. Based on the ResNet18 model, migration training is performed, and the fully connected layers in the ResNet18 model are updated and fine-tuned to obtain an image recognition model.

[0102] It should be noted that for training the image recognition model, the recall rate of the model is about 85% because the amount of data in the early stage may be insufficient. The amount of data can be increased by 6-8 times through data enhancement, such as flipping transformation, random pruning, color jittering, translation transformation, scale transformation, contrast transformation, noise disturbance, rotation transformation, reflection transformation, etc., so that the recall rate of the model can be increased to 90.5%.

[0103] Step A2, input the running result screenshot of the test case into the image recognition model, and determine the page type corresponding to the running result screenshot through image recognition processing. The page type corresponding to the running result screenshot is any one of a pop-up page type, a semi-pop-up page type, a FAQ page type, a face page type, and a form page type.

[0104] Specifically, the role of image recognition processing (image recognition) is to identify and distinguish the page type corresponding to the running result screenshot, so that you can only focus on the relevant information of the elements corresponding to the page type. For example, for a pop-up page type, you can only focus on the relevant information of the pop-up window (for example, the text information in the text box of the pop-up window, the button of the pop-up window, and the text information in the button), and ignore the information outside the pop-up window. For another example, for an FAQ page type, you can only focus on the text information in the text area. For another example, for a semi-pop-up page type, you can only focus on the risk warning text information, buttons, and text information in the buttons. For another example, for a semi-pop-up page type, you can only focus on the text information in the check box, the button, and the text information in the button. For another example, if Figure 1 As shown, the page type corresponding to the running result screenshot can be determined through the image recognition function in the intelligent recognition module.

[0105] In one embodiment, based on the pre-trained image recognition model, an image recognition model is obtained through transfer learning, including:

[0106] Update the fully connected layer in the pre-trained image recognition model to obtain an updated model;

[0107] The updated model is fine-tuned to obtain an image recognition model.

[0108] Specifically, the pre-trained image recognition model, such as a deep model, can be trained by the Yolo algorithm. For example, the pre-trained image recognition model is a ResNet18 model, and the fully connected layer in the ResNet18 model is updated to obtain an updated model; the updated model is fine-tuned to obtain an image recognition model.

[0109] In one embodiment, based on the page type corresponding to the operation result screenshot, element parsing processing is performed on the operation result screenshot to determine the image features corresponding to the operation result screenshot, including:

[0110] Based on the page type corresponding to the running result screenshot, each element in the running result screenshot is detected and segmented to determine the image features corresponding to the running result screenshot. The image features include the type of each element, the position information of each element and the number of elements corresponding to the type of each element.

[0111] Specifically, for example, based on the page type corresponding to the running result screenshot, each element in the running result screenshot is detected and segmented by adaptive thresholding, rectangle detection, Hough transform, etc. to determine the image features corresponding to the running result screenshot.

[0112] In one embodiment, based on the page type corresponding to the operation result screenshot, each element in the operation result screenshot is detected and segmented to determine the image features corresponding to the operation result screenshot, including steps B1-B3:

[0113] Step B1, through adaptive thresholding, the page background of the page type corresponding to the running result screenshot is denoised to obtain a denoised image.

[0114] Specifically, for example, the page type corresponding to the running result screenshot is a pop-up window, and the page background of the pop-up window is denoised through adaptive thresholding, that is, the noise of the page background of the pop-up window is removed to obtain a denoised image.

[0115] Step B2: Detect and segment the elements in the running result screenshot from the denoised image through rectangle detection and Hough transform.

[0116] Specifically, for example, the page type corresponding to the running result screenshot is a pop-up window, such as Figure 4 As shown, the rectangular pop-up window can be detected and split through rectangle detection, and the straight line in the pop-up window can be detected through Hough transform. The horizontal position is used to split the text box and the button, and the vertical straight line is used to split the two buttons. Among them, the text information in the text box is "The current behavior is suspected to be not operated by yourself. In order to ensure the safety of funds, the transaction cannot be completed temporarily. If it is confirmed that it is used by yourself, you can click "View Solution" to submit the information to lift the restriction"; the two buttons are "Close" button and "View Solution" button.

[0117] It should be noted that in the actual detection and segmentation process, the boundary can be cropped to avoid detecting other straight line boundaries.

[0118] Step B3: determining image features corresponding to the operation result screenshot based on the segmented elements in the operation result screenshot.

[0119] Specifically, for example, the page type corresponding to the running result screenshot is a pop-up window, such as Figure 4As shown, a text box and two buttons in the pop-up window are separated, and based on the separated text box and two buttons, the image features corresponding to the running result screenshot are determined. The image features include: the element type is a text box and a button; the position information of the text box is the coordinates of the text box, and the position information of the button is the coordinates of the button; the number of elements corresponding to the text box is 1, that is, the number of text boxes is 1; the number of elements corresponding to the button is 2, that is, the number of buttons is 2.

[0120] In one embodiment, based on the page type corresponding to the operation result screenshot, element parsing processing is performed on the operation result screenshot to determine the image features corresponding to the operation result screenshot, including:

[0121] Based on the page type corresponding to the running result screenshot and the expected running result of the preset test case, the image features corresponding to the running result screenshot are determined through scale-invariant feature transformation feature matching. The image features include the element type of each element, the element position information of each element, and the number of elements corresponding to the element type of each element.

[0122] Specifically, for example, Figure 5 As shown, the page type corresponding to the running result screenshot is a semi-pop-up page type. Based on the page type corresponding to the running result screenshot and the expected running result of the preset test case, SIFT feature matching is performed to obtain a matching result, that is, the image feature corresponding to the running result screenshot; Figure 5 The matching results are marked with dots, and the matching results include two checkboxes and two buttons. The two checkboxes are "I promise that the account is for my own use and will not be used for illegal purposes" and "I agree that if the payment determines that I am suspected of using the account for illegal activities, it has the right to suspend, interrupt or terminate the service", and the two buttons are "Abandon payment" and "Enter identity verification", that is, the element type of the element is checkbox and button, the element position information of the checkbox is the coordinates of the checkbox, the element position information of the button is the coordinates of the button, the number of elements corresponding to the checkbox is 2, and the number of elements corresponding to the button is 2. For another example, Figure 1 As shown, the analysis and processing function in the intelligent recognition module can be used to perform element analysis on the running result screenshot to determine the image features corresponding to the running result screenshot.

[0123] In one embodiment, based on the page type corresponding to the operation result screenshot, text recognition processing is performed on the operation result screenshot to determine the text features corresponding to the operation result screenshot, including:

[0124] Based on the page type corresponding to the running result screenshot, text recognition processing is performed on the text information in each element of the running result screenshot to determine the text features corresponding to the running result screenshot. The text features corresponding to the running result screenshot include the text information of each element, and each element is any one of a pop-up window, button, check box, icon, and text box.

[0125] Specifically, for example, the page type corresponding to the running result screenshot is a pop-up window, such as Figure 4 As shown, based on the page type corresponding to the running result screenshot, the text information in the text box of the running result screenshot is subjected to text recognition processing to determine the text features corresponding to the running result screenshot. The text features corresponding to the running result screenshot include the text information in the text box. The text information in the text box is "The current behavior is suspected to be not operated by the user. To ensure the safety of funds, the transaction cannot be completed temporarily. If it is confirmed that the user is using it himself, click "View Solution" to submit the information to lift the restriction." For another example, Figure 1 As shown, the OCR function in the intelligent recognition module can be used to perform text recognition processing on the running result screenshot to determine the text features corresponding to the running result screenshot.

[0126] In one embodiment, the preset expected running result of the test case includes expected image features and expected text features, and based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, and the preset expected running result of the test case, determining the similarity between the running result screenshot and the expected running result of the test case includes:

[0127] Perform feature comparison processing on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, the expected image features and the expected text features to determine the similarity between the running result screenshot and the expected running result of the test case,

[0128] Among them, the image features include the element type of each element in the running result screenshot, the element position information of each element, and the number of elements corresponding to the element type of each element; the text features include the text information of each element; the expected image features include the expected element type, the expected element position information, and the number of elements corresponding to the expected element type; and the expected text features include the expected text information.

[0129] Specifically, for example, Figure 1 As shown, the feature comparison function in the intelligent recognition module can be used to determine the similarity between the element type of the element in the screenshot, the element position information of the element in the screenshot, the number of elements corresponding to the element type of the element in the screenshot, and the text information of the element in the screenshot and the expected element type, expected element position information, the number of elements corresponding to the expected element type, and the expected text information.

[0130] In one embodiment, based on the similarity and a preset similarity threshold, corresponding processing is performed on the test case, including any of the following:

[0131] If the similarity is greater than or equal to the preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be passed, and other pages are tested by running the test case;

[0132] If the similarity is less than a preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be failed, and the running of the test case is stopped.

[0133] Specifically, the similarity threshold is, for example, 95%. Figure 1 As shown, if the similarity is greater than or equal to the similarity threshold of 95%, the intelligent recognition module determines that the test result of the page corresponding to the running result screenshot is passed, and the intelligent recognition module sends the test result to the experience engine through the bus; the experience engine analyzes and displays the test result; the experience engine sends an instruction through the bus to instruct the simulation terminal to continue running the test case, that is, the simulation terminal tests other pages by running the test case. For another example, if the similarity is less than the similarity threshold of 95%, the intelligent recognition module determines that the test result of the page corresponding to the running result screenshot is not passed, and the intelligent recognition module sends the test result to the experience engine through the bus; the experience engine analyzes and displays the test result; the experience engine sends an instruction through the bus to instruct the simulation terminal to stop running the test case, that is, the simulation terminal stops running the test case.

[0134] The application of the embodiments of the present application has at least the following beneficial effects:

[0135] The accuracy of the similarity between the running result screenshot and the expected running result of the test case is improved. Based on the high-accuracy similarity and the preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be passed or failed, thereby improving the accuracy of the test results of the automated test, thereby improving the confidence corresponding to multiple similarities (the similarities between multiple running result screenshots and the expected running results of the test case) within a certain period of time, that is, improving the accuracy of the assertions in the automated test and improving the efficiency of the automated test.

[0136] In order to better understand the method provided in the embodiment of the present application, the scheme of the embodiment of the present application is further explained below with reference to examples of specific application scenarios.

[0137] In one embodiment, Figure 6As shown, the running result screenshots are the first page, the second page, the third page and the fourth page respectively. The first page displays the risk notification, the second page displays the risk confirmation, the third page displays the identity authentication, and the fourth page displays the verification success; run the test case, if the test result of the page corresponding to the first page (running result screenshot) is passed, then the test of the page corresponding to the second page (running result screenshot) is performed; if the test result of the page corresponding to the second page (running result screenshot) is passed, then the test of the page corresponding to the third page (running result screenshot) is performed; if the test result of the page corresponding to the third page (running result screenshot) is passed, then the test of the page corresponding to the fourth page (running result screenshot) is performed. If the test result of the page corresponding to any of the first, second, third and fourth pages (running result screenshots) is failed, stop running the test case. As shown Figure 6 The middle scenario is interception (releasable), which is used to indicate that the user can remove the interception by filling out a survey form, security verification, etc. In addition, there are other scenarios such as light reminder, interception (such as FAQ), and interception (such as unreleasable); among them, unreleasable is used to indicate that the user is only informed of the risk and the intercepted transaction cannot be removed.

[0138] In a specific application scenario embodiment, such as an automated testing scenario, see Figure 7 , shows a processing flow of an automated testing method, such as Figure 7 As shown, the processing flow of the automated testing method provided in the embodiment of the present application includes the following steps:

[0139] S701, the experience engine in the automated test system sends the test case to the simulation terminal via the bus.

[0140] Specifically, for example, Figure 1 As shown, the experience engine includes functions such as test case management, parameter management in test cases, test case-related process management, and analysis and display of test results. Among them, test case management is used to provide test cases, parameter management in test cases, for example, abstractly encapsulates user real-name authentication information into parameters for unified management, test case-related process management is used to manage test cases in parallel or serially, and analysis and display of test results is used to analyze the test results and display the analysis results.

[0141] The simulation terminal provides a running carrier for automated testing. The simulation terminal includes functions such as simulation engine, intelligent interaction, and experience screenshots. Among them, the simulation engine provides a virtual device for the automated testing system to control; intelligent interaction simulates user behavior, such as clicking, sliding, inputting, and taking screenshots on the simulation terminal; and experience screenshots are used to provide screenshots of the running results of the test case.

[0142] The bus is the communication channel between the simulation terminal, intelligent recognition module, and experience engine. The bus can send and receive commands.

[0143] S702, the simulation terminal in the automated test system receives the test case and runs the test case, and sends a screenshot of the running result of the test case to the intelligent recognition module in the automated test system through the bus.

[0144] Specifically, for example, Figure 1 As shown, the intelligent recognition module includes functions such as image recognition, element analysis, OCR, and feature comparison; wherein, through image recognition in the intelligent recognition module, the page type corresponding to the running result screenshot is determined; through the analysis processing function in the intelligent recognition module, the running result screenshot is subjected to element analysis to determine the image features corresponding to the running result screenshot; through the OCR in the intelligent recognition module, the running result screenshot is subjected to text recognition processing to determine the text features corresponding to the running result screenshot; through feature comparison in the intelligent recognition module, the element type of the element in the screenshot, the element position information of the element in the screenshot, the number of elements corresponding to the element type of the element in the screenshot, and the text information of the element in the screenshot are determined to be similar to the expected element type, the expected element position information, the number of elements corresponding to the expected element type, and the expected text information.

[0145] S703, the intelligent recognition module in the automated test system receives the running result screenshot of the test case through the bus; the intelligent recognition module determines the page type corresponding to the running result screenshot through image recognition processing based on the running result screenshot of the test case.

[0146] Specifically, a screenshot of the running result of the test case is input into the image recognition model, and through image recognition processing, the page type corresponding to the running result screenshot is determined. The page type corresponding to the running result screenshot is any one of a pop-up page type, a semi-pop-up page type, a FAQ page type, a face page type, and a form page type.

[0147] S704, the intelligent recognition module in the automated testing system performs element analysis on the running result screenshot based on the page type corresponding to the running result screenshot, determines the image features corresponding to the running result screenshot, and performs text recognition on the running result screenshot to determine the text features corresponding to the running result screenshot.

[0148] Specifically, for example, based on the page type corresponding to the running result screenshot, each element in the running result screenshot is detected and segmented by adaptive thresholding, rectangle detection, Hough transform, etc., to determine the image features corresponding to the running result screenshot. For another example, based on the page type corresponding to the running result screenshot and the expected running result of the preset test case, the image features corresponding to the running result screenshot are determined by SIFT feature matching.

[0149] For example, Figure 1 As shown, the OCR function in the intelligent recognition module can be used to perform text recognition processing on the running result screenshot to determine the text features corresponding to the running result screenshot.

[0150] S705, the intelligent recognition module in the automated testing system determines the similarity between the running result screenshot and the expected running result of the test case based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot and the preset expected running result of the test case.

[0151] Specifically, the image features include the element type, element position information, and the number of elements corresponding to the element type of the element in the run result screenshot, the text features include the text information of the element, the expected image features include the expected element type, the expected element position information, and the number of elements corresponding to the expected element type, and the expected text features include the expected text information. Figure 1 As shown, the feature comparison function in the intelligent recognition module can be used to determine the similarity between the element type of the element in the screenshot, the element position information of the element in the screenshot, the number of elements corresponding to the element type of the element in the screenshot, and the text information of the element in the screenshot and the expected element type, expected element position information, the number of elements corresponding to the expected element type, and the expected text information.

[0152] S706, the intelligent recognition module in the automated testing system determines the test result of the page corresponding to the running result screenshot based on the similarity and the similarity threshold.

[0153] Specifically, if the similarity is greater than or equal to the similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be passed; if the similarity is less than the similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be failed.

[0154] S707, the intelligent identification module in the automated testing system sends the test result to the experience engine in the automated testing system through the bus.

[0155] S708, the experience engine in the automated testing system receives the test results, and analyzes and displays the test results.

[0156] Specifically, the experience engine analyzes the reasons why the test case passed the test, the reasons why the test case failed the test, etc.

[0157] S709: The experience engine in the automated test system instructs the simulation terminal to perform corresponding operations through the bus based on the test results.

[0158] Specifically, if the test result is passed, the experience engine sends an instruction through the bus to instruct the simulation terminal to continue running the test case; if the test result is failed, the experience engine sends an instruction through the bus to instruct the simulation terminal to stop running the test case.

[0159] The application of the embodiments of the present application has at least the following beneficial effects:

[0160] The accuracy of the similarity between the running result screenshot and the expected running result of the test case is improved. Based on the high-accuracy similarity and the preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be passed or failed, thereby improving the accuracy of the test results of the automated test, thereby improving the confidence corresponding to multiple similarities (the similarities between multiple running result screenshots and the expected running results of the test case) within a certain period of time, that is, improving the accuracy of the assertions in the automated test and improving the efficiency of the automated test.

[0161] The present application also provides an automated testing device. The structure diagram of the automated testing device is as follows: Figure 8 As shown, the automated testing device 80 includes a first processing module 801 , a second processing module 802 , a third processing module 803 , a fourth processing module 804 and a fifth processing module 805 .

[0162] The first processing module 801 is used to obtain a screenshot of the running result of the test case;

[0163] The second processing module 802 is used to determine the page type corresponding to the running result screenshot through image recognition processing based on the running result screenshot of the test case;

[0164] The third processing module 803 is used to perform element analysis processing on the operation result screenshot based on the page type corresponding to the operation result screenshot, determine the image features corresponding to the operation result screenshot, and perform text recognition processing on the operation result screenshot to determine the text features corresponding to the operation result screenshot;

[0165] The fourth processing module 804 is used to determine the similarity between the running result screenshot and the expected running result of the test case based on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot and the preset expected running result of the test case;

[0166] The fifth processing module 805 is used to perform corresponding processing on the test case based on the similarity and a preset similarity threshold.

[0167] In one embodiment, the second processing module 802 is specifically configured to:

[0168] Based on the pre-trained image recognition model, an image recognition model is obtained through transfer learning;

[0169] The running result screenshot of the test case is input into the image recognition model. Through image recognition processing, the page type corresponding to the running result screenshot is determined. The page type corresponding to the running result screenshot is any one of the pop-up page type, semi-pop-up page type, FAQ page type, face page type, and form page type.

[0170] In one embodiment, the second processing module 802 is specifically configured to:

[0171] Update the fully connected layer in the pre-trained image recognition model to obtain an updated model;

[0172] The updated model is fine-tuned to obtain an image recognition model.

[0173] In one embodiment, the third processing module 803 is specifically configured to:

[0174] Based on the page type corresponding to the running result screenshot, each element in the running result screenshot is detected and segmented to determine the image features corresponding to the running result screenshot. The image features include the type of each element, the position information of each element and the number of elements corresponding to the type of each element.

[0175] In one embodiment, the third processing module 803 is specifically configured to:

[0176] Through adaptive thresholding, the page background of the page type corresponding to the running result screenshot is denoised to obtain a denoised image;

[0177] Through rectangle detection and Hough transform, detect and segment the elements in the running result screenshot from the denoised image;

[0178] Based on the elements in the segmented running result screenshot, image features corresponding to the running result screenshot are determined.

[0179] In one embodiment, the third processing module 803 is specifically configured to:

[0180] Based on the page type corresponding to the running result screenshot and the expected running result of the preset test case, the image features corresponding to the running result screenshot are determined through scale-invariant feature transformation feature matching. The image features include the element type of each element, the element position information of each element, and the number of elements corresponding to the element type of each element.

[0181] In one embodiment, the third processing module 803 is specifically configured to:

[0182] Based on the page type corresponding to the running result screenshot, text recognition processing is performed on the text information in each element of the running result screenshot to determine the text features corresponding to the running result screenshot. The text features corresponding to the running result screenshot include the text information of each element, and each element is any one of a pop-up window, button, check box, icon, and text box.

[0183] In one embodiment, the preset expected running result of the test case includes expected image features and expected text features. The fourth processing module 804 is specifically used to:

[0184] Perform feature comparison processing on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, the expected image features and the expected text features to determine the similarity between the running result screenshot and the expected running result of the test case,

[0185] Among them, the image features include the element type of each element in the running result screenshot, the element position information of each element, and the number of elements corresponding to the element type of each element; the text features include the text information of each element; the expected image features include the expected element type, the expected element position information, and the number of elements corresponding to the expected element type; and the expected text features include the expected text information.

[0186] In one embodiment, the fifth processing module 805 is specifically configured to perform any one of the following:

[0187] If the similarity is greater than or equal to the preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be passed, and other pages are tested by running the test case;

[0188] If the similarity is less than a preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be failed, and the running of the test case is stopped.

[0189] The application of the embodiments of the present application has at least the following beneficial effects:

[0190] Obtain a screenshot of the running result of the test case; based on the screenshot of the running result of the test case, determine the page type corresponding to the screenshot of the running result through image recognition processing; based on the page type corresponding to the screenshot of the running result, perform element analysis processing on the screenshot of the running result to determine the image features corresponding to the screenshot of the running result, and perform text recognition processing on the screenshot of the running result to determine the text features corresponding to the screenshot of the running result; based on the image features corresponding to the screenshot of the running result, the text features corresponding to the screenshot of the running result and the preset expected running results of the test case, determine the similarity between the screenshot of the running result and the expected running results of the test case; in this way, the accuracy of the similarity between the screenshot of the running result and the expected running results of the test case is improved, so that based on the high-accuracy similarity and the preset similarity threshold, corresponding processing is performed on the test case, thereby improving the accuracy of the test results of the automated test and improving the efficiency of the automated test.

[0191] The present application also provides an electronic device. The structural diagram of the electronic device is as follows: Fig. 9 As shown, Fig. 9 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0192] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0193] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0194] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0195] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0196] Among them, electronic equipment includes but is not limited to: servers, etc.

[0197] The application of the embodiments of the present application has at least the following beneficial effects:

[0198] Obtain a screenshot of the running result of the test case; based on the screenshot of the running result of the test case, determine the page type corresponding to the screenshot of the running result through image recognition processing; based on the page type corresponding to the screenshot of the running result, perform element analysis processing on the screenshot of the running result to determine the image features corresponding to the screenshot of the running result, and perform text recognition processing on the screenshot of the running result to determine the text features corresponding to the screenshot of the running result; based on the image features corresponding to the screenshot of the running result, the text features corresponding to the screenshot of the running result and the preset expected running results of the test case, determine the similarity between the screenshot of the running result and the expected running results of the test case; in this way, the accuracy of the similarity between the screenshot of the running result and the expected running results of the test case is improved, so that based on the high-accuracy similarity and the preset similarity threshold, corresponding processing is performed on the test case, thereby improving the accuracy of the test results of the automated test and improving the efficiency of the automated test.

[0199] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0200] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0201] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application further provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in any optional embodiment of the present application.

[0202] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0203] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. An automated testing method, characterized in that: include: Get a screenshot of the test case's running results; Based on the screenshot of the running result of the test case, determine the page type corresponding to the screenshot of the running result through image recognition processing; Based on the page type corresponding to the operation result screenshot, perform element analysis processing on the operation result screenshot to determine the image features corresponding to the operation result screenshot, and perform text recognition processing on the operation result screenshot to determine the text features corresponding to the operation result screenshot; Determine the similarity between the screenshot of the running result and the expected running result of the test case based on the image features corresponding to the screenshot of the running result, the text features corresponding to the screenshot of the running result and the expected running result of the preset test case; Based on the similarity and a preset similarity threshold, corresponding processing is performed on the test case.

2. The method according to claim 1, characterized in that The step of determining the page type corresponding to the screenshot of the running result based on the test case through image recognition processing includes: Based on the pre-trained image recognition model, an image recognition model is obtained through transfer learning; The running result screenshot of the test case is input into the image recognition model, and the page type corresponding to the running result screenshot is determined through image recognition processing. The page type corresponding to the running result screenshot is any one of a pop-up page type, a semi-pop-up page type, a FAQ page type, a face page type, and a form page type.

3. The method according to claim 2, characterized in that The image recognition model is obtained by transfer learning based on the pre-trained image recognition model, including: Update the fully connected layer in the pre-trained image recognition model to obtain an updated model; The updated model is fine-tuned to obtain an image recognition model.

4. The method according to claim 1, characterized in that The performing element parsing processing on the operation result screenshot based on the page type corresponding to the operation result screenshot to determine the image feature corresponding to the operation result screenshot includes: Based on the page type corresponding to the operation result screenshot, each element in the operation result screenshot is detected and segmented to determine the image features corresponding to the operation result screenshot, wherein the image features include the type of each element, the position information of each element and the number of elements corresponding to the type of each element.

5. The method according to claim 4, characterized in that The detecting and segmenting each element in the operation result screenshot based on the page type corresponding to the operation result screenshot to determine the image features corresponding to the operation result screenshot includes: By adaptive thresholding, the page background of the page type corresponding to the operation result screenshot is denoised to obtain a denoised image; Detect and segment each element in the operation result screenshot from the denoised image through rectangle detection and Hough transform; Based on the segmented elements in the operation result screenshot, image features corresponding to the operation result screenshot are determined.

6. The method according to claim 1, characterized in that The performing element parsing processing on the operation result screenshot based on the page type corresponding to the operation result screenshot to determine the image feature corresponding to the operation result screenshot includes: Based on the page type corresponding to the running result screenshot and the expected running result of the preset test case, the image features corresponding to the running result screenshot are determined through scale-invariant feature transformation feature matching, and the image features include the element type of each element, the element position information of each element and the number of elements corresponding to the element type of each element.

7. The method according to claim 1, characterized in that The performing text recognition processing on the operation result screenshot based on the page type corresponding to the operation result screenshot to determine the text feature corresponding to the operation result screenshot includes: Based on the page type corresponding to the operation result screenshot, text recognition processing is performed on the text information in each element of the operation result screenshot to determine the text features corresponding to the operation result screenshot, and the text features corresponding to the operation result screenshot include the text information of each element, and each element is any one of a pop-up window, a button, a check box, an icon, and a text box.

8. The method according to claim 1, characterized in that The preset expected running result of the test case includes an expected image feature and an expected text feature, and determining the similarity between the running result screenshot and the expected running result of the test case based on the image feature corresponding to the running result screenshot, the text feature corresponding to the running result screenshot, and the preset expected running result of the test case includes: Perform feature comparison processing on the image features corresponding to the running result screenshot, the text features corresponding to the running result screenshot, the expected image features and the expected text features to determine the similarity between the running result screenshot and the expected running result of the test case, Among them, the image features include the element type of each element in the running result screenshot, the element position information of each element and the number of elements corresponding to the element type of each element, the text features include the text information of each element, the expected image features include the expected element type, the expected element position information and the number of elements corresponding to the expected element type, and the expected text features include the expected text information.

9. The method according to claim 1, characterized in that: The performing corresponding processing on the test case based on the similarity and a preset similarity threshold includes any one of the following: If the similarity is greater than or equal to a preset similarity threshold, determining that the test result of the page corresponding to the running result screenshot is passed, and testing other pages by running the test case; If the similarity is less than a preset similarity threshold, the test result of the page corresponding to the running result screenshot is determined to be failed, and the running of the test case is stopped.

10. An automated testing device, characterized in that: include: The first processing module is used to obtain a screenshot of the running result of the test case; A second processing module is used to determine the page type corresponding to the running result screenshot based on the running result screenshot of the test case through image recognition processing; A third processing module is used to perform element parsing processing on the operation result screenshot based on the page type corresponding to the operation result screenshot to determine the image features corresponding to the operation result screenshot, and perform text recognition processing on the operation result screenshot to determine the text features corresponding to the operation result screenshot; A fourth processing module, configured to determine the similarity between the operation result screenshot and the expected operation result of the test case based on the image features corresponding to the operation result screenshot, the text features corresponding to the operation result screenshot and the preset expected operation result of the test case; The fifth processing module is used to perform corresponding processing on the test case based on the similarity and a preset similarity threshold.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Citation Information

Cited By

  • Automatic testing method, device and equipment and storage medium

    CN120179568A