Operation point-oriented automatic test evaluation quantification method and device

Abstract

The invention provides an operation point-oriented automatic test evaluation quantification method and device, and relates to the technical field of automatic test evaluation, and the method comprises the steps: obtaining a function case text and a mapping table of keywords and operation point types; extracting function operation point data from the function case text according to a mapping table of keywords and operation point types, and generating a function operation point table; analyzing the received operation point message sent by the automatic program, and updating the automatic operation point table; matching the automatic operation points with the function operation points according to the function operation point table and the automatic operation point table, and generating a mapping table of the automatic operation points and the function operation points; and according to the function operation point table, the automatic operation point table and the mapping table of the automatic operation points and the function operation points, generating an evaluation quantification result of the automatic test. According to the method and the device, objective quantification and real-time updating of the automatic test coverage rate and accurate output of the evaluation result are realized, and the efficiency and the reliability of test effect evaluation are greatly improved.

Description

Technical Field

[0001] This application relates to the field of automated test and evaluation technology, and in particular to an automated test and evaluation quantification method and apparatus oriented towards operation points. Background Technology

[0002] In the field of software testing, automated testing has become a mainstream method for improving testing efficiency and reducing labor costs. Current methods for quantifying the effectiveness of automated testing primarily rely on the number of automated test cases; generally, a higher number of automated test cases is considered to indicate greater coverage. However, this method, primarily based on the number of test cases, has significant limitations: it fails to reflect the true coverage of functional test cases by automated testing. In actual testing, each functional test case typically includes preconditions, execution steps, and multiple operation points. Simply counting the number of automated test cases cannot determine whether the automated script has actually executed all the key operations within the functional test case. Therefore, current technology struggles to objectively assess the coverage of automated testing for each operation (including preconditions, test case execution operations, and operation points) of functional test cases, and cannot accurately identify missing or redundant operation points, resulting in inaccurate evaluations of automated testing effectiveness.

[0003] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0004] To address the problems in the prior art, this application provides an automated test evaluation quantification method and apparatus oriented towards operation points, which can solve the problems that existing automated tests cannot accurately determine whether automated cases truly cover all operation points of functional cases, and lack real-time monitoring and quantitative evaluation of the automated execution process.

[0005] One aspect of the present invention provides an automated test evaluation quantification method oriented towards operation points, the method comprising:

[0006] Obtain a mapping table of functional case texts and keywords to operation point types;

[0007] Based on the mapping table between keywords and operation point types, extract functional operation point data from the functional case text to generate a functional operation point table;

[0008] The received operation point messages sent by the automation program are parsed, and the automation operation point table is updated.

[0009] Based on the functional operation point table and the automated operation point table, the automated operation points and functional operation points are matched to generate a mapping table between automated operation points and functional operation points.

[0010] The evaluation and quantitative results of the automated test are generated based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

[0011] Further, the step of extracting functional operation point data from the functional case text based on the mapping table between the keywords and operation point types, and generating a functional operation point table, includes:

[0012] Read the mapping table between keywords and operation point types. The mapping table between keywords and operation point types includes keyword number, keyword name, keyword hit rule and operation point type number.

[0013] Based on the keyword hitting rules, text recognition is performed on the functional case text to obtain functional operation point data;

[0014] A function operation point table is generated based on the function operation point data; the function operation point table includes a unique function operation point number, a case number, an operation point type number, and detailed operation point information.

[0015] Further, the step of parsing the received operation point messages sent by the automation program and updating the automation operation point table includes:

[0016] Read the automated operation point table; the automated operation point table includes: case number, operation point type number, unique automated operation point number, detailed operation point content, encrypted text of automated operation point, generation time and update time;

[0017] Write the encrypted text of the automated operation point into the first cache;

[0018] Receive the operation point message sent by the automation program, and determine whether the encrypted text of the automation operation point in the operation point message exists in the first cache;

[0019] If it exists, the generation time of the operation point message is written into the first cache as an attribute of the automated operation point encrypted text;

[0020] If it does not exist, determine whether the unique automation operation point number of the operation point message exists in the automation operation point table;

[0021] If it exists, update the automated operation point table according to the operation point message;

[0022] If it does not exist, insert the operation point message as new data into the automated operation point table, write the corresponding automated operation point encrypted text into the first cache, and write the case number and generation time into the second cache.

[0023] Further, the step of matching automated operation points with functional operation points according to the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points includes:

[0024] Based on the case number and operation point type number of the automation operation point data in the automation operation point table, obtain the corresponding functional operation point data from the functional operation point table;

[0025] Based on preset matching rules, the detailed operation point content of the automated operation point data, and the detailed operation point content of the corresponding functional operation point data, the automated operation points and functional operation points are matched to generate a mapping table of automated operation points and functional operation points; the mapping table of automated operation points and functional operation points includes a unique number of automated operation points, a unique number of functional operation points, and a case number.

[0026] Further, the step of generating the quantitative evaluation result of the automated test based on the function operation point table, the automation operation point table, and the mapping table between automation operation points and function operation points includes:

[0027] Extract the mapping relationship between the automated operation points and the functional operation points from the mapping table of the automated operation points and the functional operation points;

[0028] Determine whether the automated operation point exists in the automated operation point table and whether the corresponding functional operation point exists in the functional operation point table;

[0029] If the automated operation point exists in the automated operation point table but the corresponding functional operation point does not exist in the functional operation point table, then operation point redundancy is determined.

[0030] If the function operation point exists in the function operation point table but the corresponding automation operation point does not exist in the automation operation point table, then the operation point is determined to be missing.

[0031] Furthermore, it also includes:

[0032] Based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache, discarded automated operation point data is deleted.

[0033] Further, the deletion of obsolete automated operation point data based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache includes:

[0034] At preset time intervals, the system determines whether the automated case has been completed based on the case number and generation time read from the second cache.

[0035] If the automation case has been completed, filter the automation operation points that have not been updated within a preset time range from the automation operation point table;

[0036] Read the attributes of the encrypted text of the automation operation point corresponding to the automation operation point that has not been updated from the first cache;

[0037] Based on the generation time in the second cache and the attribute, it is determined whether the automation operation point that has not been updated in this automation case has been generated;

[0038] If not generated, the corresponding automated operation point data is deleted from the automated operation point table and the mapping table between automated operation points and functional operation points.

[0039] In another aspect, the present invention provides an automated test evaluation quantification device oriented towards operation points, the device comprising:

[0040] The data acquisition unit is used to acquire the mapping table between functional case text and keywords and operation point types;

[0041] The function operation point table generation unit is used to extract function operation point data from the function case text according to the mapping table between the keywords and operation point types, and generate a function operation point table.

[0042] The automated operation point table update unit is used to parse the operation point messages sent by the received automated program and update the automated operation point table.

[0043] The mapping table generation unit is used to match automated operation points with functional operation points according to the functional operation point table and the automated operation point table, and generate a mapping table between automated operation points and functional operation points.

[0044] The evaluation quantification result generation unit is used to generate the evaluation quantification result of the automated test based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

[0045] Furthermore, the function operation point table generation unit includes:

[0046] The mapping table reading module is used to read the mapping table between the keywords and operation point types. The mapping table between the keywords and operation point types includes keyword number, keyword name, keyword hitting rule and operation point type number.

[0047] The function operation point data generation module is used to perform text recognition on the function case text based on the keyword hitting rules to obtain function operation point data.

[0048] The function operation point table generation module is used to generate a function operation point table based on the function operation point data; the function operation point table includes a unique function operation point number, a case number, an operation point type number, and detailed operation point information.

[0049] Furthermore, the automated operation point table update unit includes:

[0050] The automated operation point table reading module is used to read the automated operation point table; the automated operation point table includes: case number, operation point type number, unique automated operation point number, detailed operation point content, encrypted text of automated operation point, generation time and update time.

[0051] The first writing module is used to write the encrypted text of the automated operation point into the first cache;

[0052] The first judgment module is used to receive the operation point message sent by the automation program and determine whether the encrypted text of the automation operation point in the operation point message exists in the first cache.

[0053] The second writing module is used to write the generation time of the operation point message as an attribute of the automated operation point encrypted text into the first cache, if it exists.

[0054] The second judgment module is used to determine whether the unique number of the automated operation point of the operation point message exists in the automated operation point table if it does not exist.

[0055] An automated operation point table update module is used to update the automated operation point table according to the operation point message if it exists.

[0056] The automated operation point table writing module is used to insert the operation point message as new data into the automated operation point table if it does not exist, write the corresponding automated operation point encrypted text into the first cache, and write the case number and generation time into the second cache.

[0057] Furthermore, the mapping table generation unit includes:

[0058] The function operation point data acquisition module is used to obtain the corresponding function operation point data from the function operation point table based on the case number and operation point type number of the automation operation point data in the automation operation point table.

[0059] The matching module is used to match automated operation points and functional operation points according to preset matching rules, the detailed operation point content of the automated operation point data, and the detailed operation point content of the corresponding functional operation point data, and generate a mapping table between automated operation points and functional operation points; the mapping table between automated operation points and functional operation points includes a unique number of automated operation point, a unique number of functional operation point, and a case number.

[0060] Furthermore, the evaluation quantification result generation unit includes:

[0061] The mapping relationship extraction module is used to extract the mapping relationship between the automated operation points and the functional operation points from the mapping table between the automated operation points and the functional operation points.

[0062] The third judgment module is used to determine whether the automated operation point exists in the automated operation point table and whether the corresponding functional operation point exists in the functional operation point table;

[0063] The first evaluation module is used to determine operation point redundancy if the automated operation point exists in the automated operation point table but the corresponding functional operation point does not exist in the functional operation point table.

[0064] The second evaluation module is used to determine that an operation point is missing if the function operation point exists in the function operation point table but the corresponding automation operation point does not exist in the automation operation point table.

[0065] Furthermore, it also includes:

[0066] The operation point deletion unit is used to delete obsolete automated operation point data based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache.

[0067] Furthermore, the operation point deletion unit includes:

[0068] The fourth judgment module is used to determine whether the automated case has been completed at preset time intervals based on the case number and generation time read from the second cache.

[0069] The operation point filtering module is used to filter automated operation points that have not been updated within a preset time range from the automated operation point table if the automated case has been completed.

[0070] The attribute reading module is used to read the attributes of the encrypted text of the automated operation point corresponding to the automated operation point that has not been updated from the first cache;

[0071] The fifth judgment module is used to determine whether the automation operation point that has not been updated in this automation case has been generated based on the generation time in the second cache and the attribute.

[0072] The operation point deletion module is used to delete the corresponding automated operation point data from the automated operation point table and the mapping table between automated operation points and functional operation points if the operation point has not been generated.

[0073] To achieve the above objectives, according to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described automated test evaluation quantification method for operation points.

[0074] To achieve the above objectives, according to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program / instructions are stored, which, when executed by a processor, implement the steps of the above-described automated test evaluation quantification method oriented towards operation points.

[0075] To achieve the above objectives, according to another aspect of the present invention, a computer program product is also provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the above-described operation-point-oriented automated test evaluation quantification method.

[0076] The beneficial effects of this invention are as follows:

[0077] This invention receives operation point messages sent in real time during automated test execution and performs rapid comparison based on the encrypted text of the operation points to determine whether the operation point content has changed. The automated operation point table is updated only when changes occur, thus avoiding the performance overhead of repeated writes. Furthermore, combined with a discarded automated operation point deletion mechanism, historical operation point data not generated in the current execution cycle can be automatically removed, achieving continuous real-time updates of operation point data. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0079] Figure 1 This is a schematic diagram of the first process of the automated test evaluation quantification method for operation points provided in this embodiment of the invention;

[0080] Figure 2 This is a schematic diagram of the second process of the automated test evaluation quantification method for operation points provided in the embodiments of the present invention;

[0081] Figure 3 This is a schematic diagram of the third process of the automated test evaluation quantification method for operation points provided in the embodiments of the present invention;

[0082] Figure 4 This is a schematic diagram of the fourth process of the automated test evaluation quantification method for operation points provided in this embodiment of the invention;

[0083] Figure 5 This is a schematic diagram of the fifth process of the automated test evaluation quantification method for operation points provided in the embodiments of the present invention;

[0084] Figure 6 This is a schematic diagram of the sixth process of the automated test evaluation and quantification method for operation points provided in this embodiment of the invention;

[0085] Figure 7 This is a first structural schematic block diagram of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention;

[0086] Figure 8 This is a schematic block diagram of the second structure of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention;

[0087] Figure 9 This is a schematic block diagram of the third structure of the automated test and evaluation quantification device for operation points provided in the embodiments of the present invention;

[0088] Figure 10 This is a schematic block diagram of the fourth structure of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention;

[0089] Figure 11 This is a fifth structural schematic block diagram of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention;

[0090] Figure 12 This is a sixth structural schematic block diagram of the automated test evaluation quantification device for operation points provided in this embodiment of the invention;

[0091] Figure 13 This is a seventh structural schematic block diagram of the automated test and evaluation quantification device for operation points provided in the embodiments of the present invention;

[0092] Figure 14 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0093] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0094] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0096] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0097] The following describes the specific implementation process of the automated test evaluation quantification method for operation points provided in this application embodiment, taking the server as the execution subject as an example.

[0098] Figure 1 This is a schematic diagram of the first process of the automated test evaluation quantification method for operation points provided in this embodiment of the invention, as shown below. Figure 1 As shown, in one embodiment of the present invention, the automated test evaluation quantification method for operation points of the present invention includes:

[0099] S101: Obtain the mapping table between functional case text and keywords and operation point types;

[0100] S102: Extract functional operation point data from the functional case text according to the mapping table between the keywords and operation point types, and generate a functional operation point table;

[0101] S103: Parse the operation point messages sent by the received automation program and update the automation operation point table;

[0102] S104: Match the automated operation points with the functional operation points according to the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points.

[0103] S105: Generate the evaluation and quantitative results of the automated test based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

[0104] from Figure 1 As shown in the flowchart, the automated test evaluation quantification method for operation points provided by this invention obtains functional case text and a mapping table between keywords and operation point types; extracts functional operation point data from the functional case text according to the mapping table to generate a functional operation point table; parses the operation point messages sent by the received automated program and updates the automated operation point table; matches automated operation points with functional operation points according to the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points; and generates the evaluation quantification result of automated testing based on the functional operation point table, the automated operation point table, and the mapping table between automated operation points and functional operation points. This achieves objective quantification, real-time updating, and accurate output of evaluation results for automated test coverage, significantly improving the efficiency and reliability of test effectiveness evaluation.

[0105] Each step is explained in detail below.

[0106] S101: Obtain the mapping table between functional case text and keywords and operation point types;

[0107] Specifically, the server first obtains the functional case text and a pre-configured mapping table of keywords and operation point types. This mapping table establishes the correspondence between keywords in the functional text description and different operation point types. By loading the mapping table, the server can identify different types of operation points, thus laying the foundation for subsequent identification and parsing of operation points from the functional case text. The content of the functional case text includes semantic sentences describing the test execution process and verification logic, while the mapping table acts as a knowledge base to guide subsequent automated identification of functional operation points.

[0108] S102: Extract functional operation point data from the functional case text according to the mapping table between the keywords and operation point types, and generate a functional operation point table;

[0109] Specifically, the server parses the functional case text based on the aforementioned mapping table, automatically extracts multiple functional operation points involved in the functional cases, and structures and stores them in the functional operation point table according to case number, operation point type, and operation behavior semantics. This transforms the text description into a data format that can be used for matching calculations. In this process, the server not only identifies the operation point type but also extracts the operation behavior content related to the operation point to ensure that the functional operation point table can fully reflect the actual execution behavior of the functional cases. The transformation of natural language descriptions into structured operation point information guides the comparison and evaluation of subsequent automated testing behaviors.

[0110] Figure 2 This is a schematic diagram of the second process of the automated test evaluation quantification method for operation points provided in this embodiment of the invention, as shown below. Figure 2 As shown, in one embodiment of the present invention, S102 includes:

[0111] S201: Read the mapping table between the keyword and the operation point type. The mapping table between the keyword and the operation point type includes the keyword number, the keyword name, the keyword hitting rule, and the operation point type number.

[0112] Specifically, the server first reads a mapping table between keywords and operation point types. This mapping table is pre-constructed based on the functional characteristics of the business system and describes the correspondence between keywords in the functional copy and different operation point types. The mapping table not only contains the basic text of the keywords but also the keyword number, keyword matching rules, and their corresponding operation point type number. This enables the server to identify operation point types from natural language descriptions, ensuring that the server has a knowledge base that can be used to parse functional case text, providing an accurate source of rules for the automated identification of functional operation points.

[0113] S202: Based on the keyword hitting rules, perform text recognition on the functional case text to obtain functional operation point data;

[0114] Specifically, after successfully loading the keyword-operation point type mapping table, the server uses the matching rules in the mapping table to perform semantic-level text recognition on the functional case text. It scans each statement, phrase, and keyword in the case description to extract semantic fragments related to the functional execution behavior. Matching rules can include string matching, regular expressions, and business behavior qualifiers to determine whether keywords corresponding to a certain type of operation point exist in the case text. Through this parsing process, the server extracts the type identifier and operation behavior text for each functional operation point, forming functional operation point data for subsequent structured storage and matching calculations.

[0115] S203: Generate a function operation point table based on the function operation point data; the function operation point table includes a unique function operation point number, a case number, an operation point type number, and detailed operation point information.

[0116] Specifically, the server constructs a function operation point table based on the parsed function operation point data, achieving structured modeling of function case behaviors. The function operation point table assigns a unique function operation point number to each operation point, while also recording the case number to which the operation point belongs, the corresponding operation point type number, and the detailed content of the operation point. By converting the case behaviors originally described in natural language into a structured data format, not only do the function case operation points become searchable and indexable, but the foundation is also laid for establishing a one-to-one mapping relationship with the execution behavior of automated programs, thus avoiding the inefficiency and easy omissions of traditional manual comparison.

[0117] In one embodiment, the server first reads the mapping relationship between keywords and operation point types to guide the parsing of operation point types in the functional case text. The server preloads an operation point type data table, which includes operation point type numbers and operation point names, to define all identifiable functional operation point types. Simultaneously, the server reads an operation point type-keyword mapping table, which includes keyword numbers, keyword names, keyword matching rules, and operation point type numbers. The keyword number is a unique index field in this table to ensure the distinguishability of keyword configurations. The relationship between operation point types and keywords can be one-to-one or one-to-many. For example, a database operation point may correspond to multiple keywords, such as "database," "data table," and "data source," while an interface parameter reading operation point may correspond to keywords such as "interface return" and "output parameters."

[0118] During the parsing of functional case text, keyword matching rules are used to resolve ambiguity where multiple keywords may point to the same operation point type. For example, the text "read fields from data table a in database A" contains both the keywords "database" and "data table," but only represents one database read operation point. To accurately identify this operation point type, the server uses keyword matching rules to comprehensively judge the text, such as determining it as the same operation point based on the relationship between "database" and "data table," language structure, or the presence of English names. Furthermore, keyword matching rules can also be automatically generated based on historical correction records or trained models and can be stored on the server for dynamic retrieval, continuously improving the accuracy of operation point identification.

[0119] After completing the matching process, the server generates operation point data based on the identified operation point type and its detailed content, and stores it in the functional case operation point table. The functional case operation point table includes four fields: a unique function operation point ID, a case ID, an operation point type ID, and detailed operation point content. These fields are used to structurally record each operation behavior within a functional case. The same operation point type can appear multiple times within the same case; for example, reading multiple data tables will generate multiple operation point records. The detailed operation point content stores specific information related to the operation point, such as the data table name, field names, and updated data values, enabling the server to accurately locate the semantic meaning and execution purpose of each operation point during subsequent evaluation.

[0120] S103: Parse the operation point messages sent by the received automation program and update the automation operation point table;

[0121] Specifically, the server parses the operation point messages sent during the execution of the automation program to obtain the operation point behaviors during the actual execution of the automated test. Operation point messages are actively sent by the automation program when performing interface calls, database operations, or critical business processing. After parsing the messages, the server updates the automation operation point table to reflect all the actual operation points involved in the latest automated test execution. By continuously receiving and processing operation point messages, the server can maintain the operation point-level execution trajectory of the automated execution behavior in real time, providing basic data for subsequent comparison and difference analysis with functional operation points.

[0122] Figure 3 This is a schematic diagram of the third process of the automated test evaluation quantification method for operation points provided in this embodiment of the invention, as shown below. Figure 3 As shown, in one embodiment of the present invention, S103 includes:

[0123] S301: Read the automated operation point table; the automated operation point table includes: case number, operation point type number, unique automated operation point number, detailed operation point content, encrypted text of automated operation point, generation time and update time;

[0124] Specifically, the server first reads the automated operation point table to retrieve operation point data recorded during previous executions of the automated test. The automated operation point table uses the unique identifier of each automated operation point as its core index and also stores runtime behavior fields such as case number, operation point type number, detailed operation point content, encrypted text of the automated operation point, generation time, and update time, reflecting the actual operation trajectory of the automated test in different execution cycles. By preloading this table, the server can accurately determine in subsequent steps whether newly received operation point messages are new or duplicate data.

[0125] S302: Write the encrypted text of the automated operation point into the first cache;

[0126] Specifically, the server writes the encrypted text from the automated operation point message into a first cached encrypted text to indicate whether the behavior content of the automated operation point has changed. Since the encrypted text is generated based on the detailed content of the operation point calculated using a preset encryption algorithm, it is not necessary to parse the entire text to determine whether the operation point content has changed, which can significantly improve the efficiency of change detection and avoid the complexity of field-by-field comparison.

[0127] S303: Receive the operation point message sent by the automation program, and determine whether the automated operation point encrypted text of the operation point message exists in the first cache;

[0128] Specifically, when the automated program executes the test script and sends an operation point message to the server, the server parses the encrypted text of the automated operation point contained in the message and checks if the same encrypted text exists in the first cache. If the same encrypted text already exists, it means that the content of the automated operation point has not changed during this test execution. This mode uses "encrypted text hit" as a fast change judgment condition, significantly reducing the processing cost of storage updates and supporting high-frequency automated test scenarios.

[0129] S304: If it exists, write the generation time of the operation point message as an attribute of the automated operation point encrypted text into the first cache;

[0130] Specifically, if the encrypted text of the received operation point message is detected to already exist in the first cache, the server updates the corresponding attribute in the first cache based on the generation time carried in the message, to indicate that the automated operation point has been triggered in this test execution. This update does not modify the automated operation point table, but only updates the cached time attribute, thereby reducing unnecessary table write operations and providing the necessary timeline basis for subsequent discard data judgment.

[0131] S305: If it does not exist, determine whether the unique number of the automated operation point of the operation point message exists in the automated operation point table;

[0132] Specifically, if the encrypted text of the received operation point message does not exist in the first cache, it is further determined whether the unique number of the automated operation point in the message exists in the automated operation point table.

[0133] S306: If it exists, update the automated operation point table according to the operation point message;

[0134] Specifically, if a unique number already exists, it means that the behavior content corresponding to the automated operation point has changed. At this time, the server updates the automated operation point table according to the message content, and writes the latest operation point details and encrypted text into the corresponding record to truly reflect the changes in the automated test execution behavior.

[0135] S307: If it does not exist, insert the operation point message as new data into the automated operation point table, write the corresponding automated operation point encrypted text into the first cache, and write the case number and generation time into the second cache.

[0136] Specifically, if the encrypted text in the operation point message does not exist in the first cache and the corresponding unique automation operation point number does not exist in the automation operation point table, it indicates that the automation operation point is newly added data. In this case, the server inserts the new operation point message into the automation operation point table, writes the encrypted text of the operation point to the first cache, and simultaneously writes the case number and generation time to the second cache. The second cache is used to track the latest execution time of automation cases, ensuring that subsequent deletion of obsolete data can accurately determine whether an operation point has been generated in the current test execution.

[0137] In one embodiment, the automation framework is modified to output operation point messages in real time during test script execution, recording every critical operation actually performed by the automated test. When the automated program performs actions such as API calls, database operations, log analysis, or input / output parameter validation, the framework automatically generates operation point messages through embedded encapsulation methods and sends them to the server in the form of a message queue (MQ). The operation point message includes at least the case number, operation point type number, unique automation operation point number, operation point content, encrypted automation operation point text, and generation time. The encrypted automation operation point text is generated by the case number, operation point type number, unique automation operation point number, and operation point content according to a preset encryption algorithm. Therefore, if any information in the above fields changes, the encrypted text changes accordingly, thus enabling the identification of whether the content of the same operation point has been updated at different execution stages. Meanwhile, the unique number of the automated operation point is generated by combining the case number, operation point type number and key features of the operation point, such as "case number + operation point type number + data table name + data field" or "case number + operation point type number + interface name + input / output parameter field node", thereby ensuring that the automated operation point has stable uniqueness in the business semantic sense, does not change with the adjustment of the automation code, and provides a basis for subsequent anti-duplicate processing.

[0138] After the server starts, it automatically loads the automated operation point table and writes the encrypted text of all automated operation points in the table to cache A to quickly determine whether the received message is a duplicate operation point. The automated operation point table stores automated operation point data recorded during historical execution. Fields include case number, operation point type number, unique automated operation point number, operation point details, encrypted text of the automated operation point, generation time, and update time. After receiving an operation point message from the automated program, the server first checks whether the encrypted text of the automated operation point exists in cache A: if it exists, it means that the operation point has been received and entered into the database in the previous execution. At this time, the server does not need to update the automated operation point table, but only writes the generation time in the current message as an attribute of the encrypted text to cache A to mark that the operation point has been triggered in the current execution cycle; if the encrypted text does not exist, it continues to check whether the operation point already has a database record based on the unique automated operation point number.

[0139] If the unique ID of an automated operation point already exists in the automated operation point table, the server determines that the operation point has undergone content changes in the current execution cycle and updates the corresponding record in the automated operation point table with the operation point details, encrypted text, and generation time information from the current message. If the unique ID of an automated operation point does not exist in the automated operation point table, the server directly inserts the current message as new data into the automated operation point table and writes the encrypted text of the automated operation point to cache A. Simultaneously, the server writes the case number as the key and the generation time as the value to cache B to record the latest execution time of the automated case, providing a basis for subsequent determination of discarded automated operation points.

[0140] S104: Match the automated operation points with the functional operation points according to the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points.

[0141] Specifically, the server performs matching calculations between automated operation points and functional operation points based on the functional operation point table and the automated operation point table to determine whether the automated test has executed each operation point defined in the functional test case. The matching process aligns functional operation points and automated operation points according to the test case number and operation point type, and then compares the detailed content of the operation points in both tables, ultimately generating a mapping table between automated operation points and functional operation points. This mapping table describes the correspondence between automated test execution behavior and functional requirement operations, and is a key intermediate result for the quantitative evaluation of automation.

[0142] Figure 4 This is a schematic diagram of the fourth process of the automated test evaluation quantification method for operation points provided in this embodiment of the invention, as shown below. Figure 4 As shown, in one embodiment of the present invention, S104 includes:

[0143] S401: Obtain the corresponding functional operation point data from the functional operation point table based on the case number and operation point type number of the automated operation point data in the automated operation point table;

[0144] Specifically, after updating the automated operation point table, the server retrieves the corresponding functional operation point data from the functional operation point table based on the case number and operation point type number of the automated operation point data in that table, for subsequent matching. Both automated and functional operation points use the case number and operation point type number as association fields, thus enabling rapid filtering of the candidate matching set and avoiding mismatches and excessive computation caused by cross-case comparisons. Through this process, the server can locate the actual operational behaviors that occurred during the automation execution to the corresponding functional requirement definition space, thereby providing an accurate range of candidate operation points for subsequent consistency comparisons.

[0145] S402: Match automated operation points and functional operation points according to preset matching rules, the detailed operation point content of the automated operation point data, and the detailed operation point content of the corresponding functional operation point data to generate a mapping table of automated operation points and functional operation points; the mapping table of automated operation points and functional operation points includes a unique number of automated operation point, a unique number of functional operation point, and a case number.

[0146] Specifically, based on preset matching rules, the server performs a one-to-one matching of automated operation points and functional operation points, combining the detailed operation point information of the automated operation point data and the functional operation point data. The matching rules can be set according to business scenarios, such as field consistency comparison, key parameter comparison, semantic similarity comparison, or multi-field combination condition judgment, to identify whether the automated operation point actually executed the corresponding operation point in the functional case. When the matching conditions are met, the server generates a mapping relationship between automated operation points and functional operation points, and records the unique ID of the automated operation point, the unique ID of the functional operation point, and the case number in the mapping table. This mapping result clearly identifies the correspondence between the automated test execution behavior and the functional requirement definition, serving as a key data source for subsequent evaluation and quantification.

[0147] In one embodiment, after updating the automated operation point table, the server performs a matching calculation for each automated operation point to determine whether the corresponding functional requirements are actually met during the automated test execution. The server first retrieves candidate functional operation point data from the functional case operation point table based on the case number and operation point type number to which the automated operation point belongs. Since multiple operation points of the same type may exist in the same case, such as multiple data table read operations or multiple field validation operations, the server may obtain multiple candidate functional operation points as comparison objects. If no candidate data is obtained, it indicates that the automated operation point does not have a corresponding definition in the functional case, meaning that the automated operation point lacks functional basis, and the server no longer processes the automated operation point.

[0148] After selecting candidate functional operation points, the server compares the automated operation points with the functional operation points based on the detailed content of the operation points to determine whether they belong to the same business semantics. The comparison rules are controlled by preset matching rules, which can be flexibly changed according to the differences in operation point types and the comparison of different nodes in JSON or other structured fields. For example, database operation points can be compared based on the data table name and field name, interface parameter validation operation points can be compared based on the return node path or key field value, and log retrieval operation points can be compared based on the log file name and keywords. Only when the detailed content of the automated operation point and the detailed content of the candidate functional operation point meet the corresponding matching rules is the server considered that the two have successfully matched, thus confirming that the automated test has covered the expected behavior of the functional case at that operation point.

[0149] For successfully matched automated operation points and functional operation points, the server writes the matching record into an automated operation point-functional operation point mapping table for subsequent evaluation, quantification, and backtracking analysis. The mapping table records the unique ID of the automated operation point, the unique ID of the functional operation point, and the case number. This structured mapping relationship fully demonstrates the correspondence between automated test execution behaviors and functional requirement definitions. This mechanism not only supports the server in identifying missing and redundant operation points in subsequent steps but also enables testers to quickly locate uncovered or repeatedly executed operation points, thus providing data support for automated script optimization.

[0150] S105: Generate the evaluation and quantitative results of the automated test based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

[0151] Specifically, after mapping automated operation points to functional operation points, the server generates quantitative evaluation results for automated testing based on the functional operation point table, the automated operation point table, and the mapping table. The quantitative evaluation process identifies missing and redundant operation points by determining whether all operation points in the functional test cases are executed automatically and whether there are any redundant operations outside the functional test cases during the automated execution process. Based on this, the degree of consistency between the automated test and the functional requirements is calculated. This quantitative evaluation result can be used to intuitively reflect the true effectiveness of automated testing in meeting functional requirements, offering significantly greater refinement and objectivity compared to the traditional method of simply counting the number of automated test cases.

[0152] Figure 5 This is a schematic diagram of the fifth process of the automated test evaluation and quantification method for operation points provided in this embodiment of the invention, as shown below. Figure 5 As shown, in one embodiment of the present invention, S105 includes:

[0153] S501: Extract the mapping relationship between the automated operation points and the functional operation points from the mapping table between the automated operation points and the functional operation points;

[0154] Specifically, the server first extracts the correspondence between automated operation points and functional operation points from the mapping table, including paired matching records between the unique IDs of automated operation points and functional operation points. This mapping reflects which actually executed operation points successfully correspond to the operation points expected to be defined in the functional test cases during the automated test execution, providing a basic basis for judging the completeness of the automated test execution. By reading the mapping records, the server can quickly locate the successfully matched operation points and the set of operation points that may have missing or redundant risks.

[0155] S502: Determine whether the automated operation point exists in the automated operation point table and whether the corresponding functional operation point exists in the functional operation point table;

[0156] Specifically, the server then checks whether each automated operation point exists in the automated operation point table and whether each functional operation point exists in the functional operation point table to confirm the relationship between the automated test execution behavior and the functional requirements within the current execution cycle. Through this check, it can identify whether the predefined operation points in the corresponding functional cases exist during automated execution, and whether the operation points defined in the functional cases are actually executed by the automated tests. This two-way existence check ensures that the evaluation process does not miss or misjudge any operation points.

[0157] S503: If the automated operation point exists in the automated operation point table but the corresponding functional operation point does not exist in the functional operation point table, then operation point redundancy is determined;

[0158] Specifically, if an automated operation point exists in the automated operation point table, but its corresponding functional operation point in the mapping table does not exist in the functional operation point table, the server determines that the automated operation point is redundant, meaning that the automated test executed an additional operation point not required by the functional test case. This redundancy usually reflects that the automated script has extra logic or non-functional execution steps, which may lead to deviations between the test program and the requirements. Therefore, it is recorded as an inconsistency in the evaluation results.

[0159] S504: If the function operation point exists in the function operation point table but the corresponding automation operation point does not exist in the automation operation point table, then it is determined that the operation point is missing.

[0160] Specifically, if a functional operation point exists in the functional operation point table, but its corresponding automated operation point in the mapping table does not exist in the automated operation point table, the server determines that the functional operation point is missing, meaning that the automated test did not execute the necessary operation specified in the functional case. Such a deficiency usually indicates that the functional logic has not been verified, which may lead to insufficient test coverage or unexposed business risks. Therefore, this missing item is recorded as a high-risk indicator in the evaluation results. By identifying missing operation points, the server can clearly point out the functional aspects not covered by automated testing, providing a basis for improving subsequent testing strategies.

[0161] In one embodiment, after completing the operation point matching, the server generates a quantitative evaluation result for each test case to measure the consistency between the automated test execution behavior and the functional requirements. First, the server retrieves all functional operation points and automated operation point data corresponding to the case from the functional case operation point table and the automated operation point table, respectively, based on the case number, to establish the basic data set for evaluation. By splitting the data at the case dimension, the server can generate evaluation conclusions one by one for each test case, thereby ensuring that the analysis of automated test effectiveness has clear positioning and traceability.

[0162] Subsequently, the server further retrieves all mapping records corresponding to the case number from the automated operation point to functional operation point mapping table to confirm which automated operation points have successfully covered the operation points defined in the functional case. This mapping relationship reflects the reach of the automated test for each functional operation point during this execution and is a key criterion for evaluating the effectiveness of the automated test.

[0163] Based on the above data, the server performs a consistency analysis on each functional operation point and automation operation point. If a functional operation point exists in the functional case operation point table but no corresponding record is found in the automation operation point table, and there is no matching relationship in the mapping table, the server determines that the functional operation point was not triggered in this automation execution and records it as an operation point missing. This missing case indicates that the automation script does not cover the functional step, which may lead to insufficient business logic verification.

[0164] Conversely, if an automated operation point exists in the automated operation point table but has no corresponding record in the functional case operation point table, and there is no matching relationship in the mapping table, the server determines that the automated operation point is redundant. This type of redundancy indicates that the automated script executes additional logic not defined in the functional requirements, which may lead to abnormal data processing, invalid resource calls, or unexpected business path execution.

[0165] In one embodiment of the present invention, the automated test evaluation quantification method oriented towards operation points further includes:

[0166] Based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache, discarded automated operation point data is deleted.

[0167] Specifically, the server deletes obsolete automated operation point data based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache. This ensures that the automated operation point table only retains operation point data that actually occurred during the current execution. The second cache stores the generation time of automated test cases, marking the time period of the automated test in the current execution; while the first cache stores the attributes of the encrypted text of the automated operation point, indicating whether each automated operation point has been triggered within the current execution period. Therefore, by comparing the time information in the second and first caches, the server can determine whether a certain automated operation point was actually generated during the current automated test execution. If it is determined that a certain automated operation point was not triggered in the current test execution, it is determined that the automated operation point has expired or is invalid, and it is deleted from the automated operation point table to avoid historical legacy data affecting subsequent evaluation results, thereby ensuring that the quantitative results of the automated test evaluation are generated based on the latest execution behavior and maintain high reliability.

[0168] Figure 6 This is a schematic diagram of the sixth process of the automated test evaluation and quantification method for operation points provided in this embodiment of the invention, as shown below. Figure 6 As shown, in one embodiment of the present invention, deleting obsolete automated operation point data based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache includes:

[0169] S601: At preset time intervals, determine whether the automated case has been completed based on the case number and generation time read from the second cache;

[0170] Specifically, the server triggers a discarded data detection process at preset time intervals, periodically reading the latest generation time of each automated test case from the second cache and determining whether the automated test case has been completed. The generation time serves as an identifier for the execution cycle of the automated test, distinguishing between the current execution round and historical execution rounds. Therefore, when the server detects that the generation time of an automated test case has not been updated for a certain period, it considers that the automated test case has completed its current execution process and enters the discarded data determination phase. This mechanism ensures that discarded data cleanup is performed at appropriate times, avoiding premature deletion of valid operation point data while the automated test is still in progress.

[0171] S602: If the automation case has been completed, filter the automation operation points that have not been updated within a preset time range from the automation operation point table;

[0172] Specifically, after confirming the completion of automated test execution, the server filters automated operation point data from the automated operation point table that has not been updated within a preset time range. These are considered potential discarded data for further verification. The update time field in the automated operation point table records the time when the automated program last triggered the same operation point. Therefore, if an automated operation point is not updated within the current execution cycle, it is preliminarily determined that the operation point was not triggered in this test execution. This filtering process can quickly narrow down the scope of discarded data and improve the efficiency of subsequent calculations.

[0173] S603: Read the attributes of the encrypted text of the automation operation point corresponding to the automation operation point that has not been updated from the first cache;

[0174] Specifically, for the automated operation points that have not been updated as selected above, the server continues to read the attributes of the encrypted text of the automated operation point from the first cache and compares it with the generation time recorded in the second cache to determine whether the operation point was generated during the current automated test execution.

[0175] S604: Based on the generation time in the second cache and the attribute, determine whether the automation operation point that has not been updated in this automation case has been generated;

[0176] Specifically, since the first cache writes encrypted text attributes when the automated program triggers an operation point, if the encrypted text attributes are not updated within the current execution cycle, it indicates that the operation point was not triggered in this test execution. This step can avoid misjudging historical operation points as valid data, thereby ensuring the accuracy of discard data decisions.

[0177] S605: If not generated, delete the corresponding automated operation point data from the automated operation point table and the mapping table between automated operation points and functional operation points.

[0178] Specifically, if it is determined that a certain automated operation point was not generated in this automation case, the server deletes that automated operation point from the automated operation point table and simultaneously deletes the corresponding mapping record from the mapping table between automated operation points and functional operation points. This is to avoid invalid mapping relationships during subsequent evaluation. Through this deletion mechanism, the server can effectively remove historical legacy data that no longer belongs to the latest automated execution behavior, ensuring that the automated operation point table and mapping table always reflect the actual operation situation of the current execution cycle, providing a stable and accurate data foundation for the quantitative results of automated test evaluation.

[0179] In one embodiment, the server periodically monitors cached records to automatically identify and delete expired or non-generated automated operation points, ensuring that the automated operation point table only reflects the actual test behavior of the most recent execution cycle. The server triggers a cleanup task at fixed time intervals. When the task starts, it first reads all records in cache B, which uses the case number as the key and the generation time as the value to identify the time node of the most recent execution of the automated case.

[0180] Subsequently, the server calculates the difference between the current time and the generation time of each record in cache B. If the difference exceeds a preset threshold, it is determined that the corresponding automated case has completed its current round of automated execution, and there is no need to wait for the test script to run further, thus allowing the automated operation points related to that case to be judged as obsolete data. The server then reads all data belonging to that case number in the automated operation point table and filters out automated operation points whose update time is earlier than a preset time range. These operation points did not receive an update time in this round of execution, possibly because the content has not changed and therefore was not updated, or because they are no longer being executed and are therefore obsolete, so further judgment is needed to determine whether they belong to obsolete data.

[0181] For the aforementioned candidate automated operation points, the server continues to read the corresponding encrypted text attribute of the automated operation point from cache A to determine whether the operation point is triggered within the current automated execution cycle. If the generation time attribute corresponding to the encrypted text is not found in cache A, it means that the automated operation point was not generated in the current execution. The server deletes the automated operation point data from the automated operation point table and simultaneously deletes the corresponding mapping record in the automated operation point and function operation point mapping table.

[0182] If a generation time attribute exists, the server further determines whether the difference between this generation time and the case generation time recorded in cache B exceeds a preset time range:

[0183] If the difference exceeds the preset range, it means that the time when the automated operation point was last triggered was earlier than the current round of automated execution cycle, so it is still judged as discarded data and the table entry is deleted;

[0184] If the difference does not exceed the preset range, it means that the automated operation point has been generated in the current automated execution cycle and no deletion operation will be performed.

[0185] In one embodiment, taking the functional case "Query bank-merchant transfer instruction - status 2 transaction successful" as an example, its case number is A_1234. The case steps are described as follows: the input parameter status value is 2, and the corresponding data exists in data table a1; after execution, the interface return result should only contain data with status 2, and the values ​​of the returned data fields out1 and out2 should be equal to the data in col1 and col2 in data table a1, respectively. The server automatically identifies the four operation points involved in the functional case based on the functional case text and keyword mapping relationship, and generates a functional case operation point table record for the identified operation points, including the unique number of the functional operation point, the case number, the operation point type number, and the detailed content of the operation point, as follows:

[0186] Operation point type number 1 (interface input parameter operation), operation point details are {"status":2};

[0187] Operation point type number 2 (data table update operation), operation point details are {"table":"a1","status":"2"};

[0188] Operation point type number 3 (interface output parameter verification operation), the detailed content of operation point is {"status":2,"out1":"col1","out2":"col2"};

[0189] Operation point type number 4 (data table read operation), the operation point details are {"table":"a1","data":["col1","col2"]}.

[0190] The above records constitute a set of functional case operation points, which serve as the target basis for subsequent automated operation point matching.

[0191] During the execution of this automation case, the automation framework uses a wrapper method to send automation operation point messages in real time via message queues (MQ) at key operation nodes: message mq1 is sent when assembling interface input parameters, message mq2 is sent when updating the data table, message mq3 is sent when validating interface output parameters, and message mq4 is sent when reading from the database. The server parses the content of each message and updates the automation operation point table accordingly. For example, if the message content of mq1 is {“status”:2}, its encrypted text already exists in cache A, and cache A records the generation time attribute of this encrypted text, indicating that this automation operation point has been generated in this execution, therefore the server does not update the automation operation point table. For example, if the message content of MQ3 is {"status":2, "out1":"col1", "out2":"col2"}, which is different from the original record {"status":2, "out1":"col1"} in the automated operation point table, the server determines that the operation point content has changed based on the fact that the encrypted text does not exist in cache A. Therefore, the server updates the operation point details in the automated operation point table and refreshes the encrypted text and update time attributes.

[0192] In addition, there is an automated operation point with operation point type number 5 (log file operation) in the history. Since the corresponding encrypted text in cache A does not record the generation time attribute, it indicates that this operation point was not triggered in this automated execution. Therefore, the server deletes the automated operation point data from the automated operation point table and the automated operation point to function operation point mapping table according to the discard judgment rule.

[0193] After the automated execution behavior is updated, the server sequentially checks whether the detailed content of each automated operation point and functional operation point meets the preset matching rules, such as comparing based on field consistency, field semantic mapping, or JSON node correspondence. When the comparison result meets the matching conditions, the server considers the automated operation point and functional operation point as a match and generates a corresponding mapping relationship record, including a unique number for the automated operation point, a unique number for the functional operation point, and a case number, thereby establishing the correspondence between the automated execution behavior and the functional requirement definition.

[0194] This invention provides a quantitative method for automated test evaluation oriented towards operation points. It involves: acquiring functional case text and a mapping table between keywords and operation point types; extracting functional operation point data from the functional case text based on the mapping table to generate a functional operation point table; parsing operation point messages sent by the automated program and updating the automated operation point table; matching automated operation points with functional operation points based on the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points; and generating quantitative evaluation results for automated tests based on the functional operation point table, the automated operation point table, and the mapping table. This method achieves objective quantification, real-time updating, and accurate output of evaluation results for automated test coverage, significantly improving the efficiency and reliability of test effectiveness evaluation.

[0195] This invention receives operation point messages sent in real time during automated test execution and performs rapid comparison based on the encrypted text of the operation points to determine whether the operation point content has changed. The automated operation point table is updated only when changes occur, thus avoiding the performance overhead of repeated writes. Furthermore, combined with a discarded automated operation point deletion mechanism, historical operation point data not generated in the current execution cycle can be automatically removed, achieving continuous real-time updates of operation point data.

[0196] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0197] Based on the same inventive concept, embodiments of the present invention also provide an automated test evaluation quantization apparatus oriented towards operation points, which can be used to implement the automated test evaluation quantization method oriented towards operation points described in the above embodiments, as described in the following embodiments. Since the principle of the automated test evaluation quantization apparatus oriented towards operation points is similar to that of the automated test evaluation quantization method oriented towards operation points, embodiments of the automated test evaluation quantization apparatus oriented towards operation points can refer to embodiments of the automated test evaluation quantization method oriented towards operation points, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0198] Figure 7 This is a schematic block diagram of the first structure of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention, as shown below. Figure 7As shown, in one embodiment of the present invention, the automated test evaluation and quantification device for operation points of the present invention includes:

[0199] Data acquisition unit 701 is used to acquire the mapping table between functional case text and keywords and operation point types;

[0200] The function operation point table generation unit 702 is used to extract function operation point data from the function case text according to the mapping table between keywords and operation point types, and generate a function operation point table.

[0201] The automated operation point table update unit 703 is used to parse the operation point messages sent by the received automated program and update the automated operation point table.

[0202] The mapping table generation unit 704 is used to match automated operation points with functional operation points according to the functional operation point table and the automated operation point table, and generate a mapping table between automated operation points and functional operation points.

[0203] The evaluation quantification result generation unit 705 is used to generate the evaluation quantification result of the automated test based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

[0204] Figure 8 This is a schematic block diagram of the second structure of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention. Figure 7 Based on the embodiments, further, such as Figure 8 As shown, in one embodiment of the present invention, the function operation point table generation unit 702 includes:

[0205] The mapping table reading module 801 is used to read the mapping table between keywords and operation point types. The mapping table between keywords and operation point types includes keyword number, keyword name, keyword hitting rule and operation point type number.

[0206] The function operation point data generation module 802 is used to perform text recognition on the function case text based on the keyword hitting rules to obtain function operation point data.

[0207] The function operation point table generation module 803 is used to generate a function operation point table based on the function operation point data; the function operation point table includes a unique function operation point number, a case number, an operation point type number, and detailed operation point information.

[0208] Figure 9 This is a schematic block diagram of the third structure of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention. Figure 7 Based on the embodiments, further, such as Figure 9 As shown, in one embodiment of the present invention, the automated operation point table update unit 703 includes:

[0209] The automated operation point table reading module 901 is used to read the automated operation point table; the automated operation point table includes: case number, operation point type number, unique automated operation point number, detailed operation point content, encrypted text of automated operation point, generation time and update time.

[0210] The first writing module 902 is used to write the encrypted text of the automated operation point into the first cache;

[0211] The first judgment module 903 is used to receive the operation point message sent by the automation program and determine whether the automated operation point encrypted text of the operation point message exists in the first cache.

[0212] The second writing module 904 is used to write the generation time of the operation point message as an attribute of the automated operation point encrypted text into the first cache, if it exists.

[0213] The second judgment module 905 is used to determine whether the unique number of the automated operation point of the operation point message exists in the automated operation point table if it does not exist.

[0214] The automated operation point table update module 906 is used to update the automated operation point table according to the operation point message if it exists.

[0215] The automated operation point table writing module 907 is used to insert the operation point message as new data into the automated operation point table if it does not exist, write the corresponding automated operation point encrypted text into the first cache, and write the case number and generation time into the second cache.

[0216] Figure 10 This is a schematic block diagram of the fourth structure of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention. Figure 7 Based on the embodiments, further, such as Figure 10 As shown, in one embodiment of the present invention, the mapping table generation unit 704 includes:

[0217] The function operation point data acquisition module 1001 is used to acquire the corresponding function operation point data from the function operation point table according to the case number and operation point type number of the automation operation point data in the automation operation point table.

[0218] The matching module 1002 is used to match automated operation points and functional operation points according to preset matching rules, the detailed operation point content of the automated operation point data and the detailed operation point content of the corresponding functional operation point data, and generate a mapping table between automated operation points and functional operation points; the mapping table between automated operation points and functional operation points includes a unique number of automated operation points, a unique number of functional operation points and a case number.

[0219] Figure 11 This is a fifth structural schematic block diagram of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention. Figure 7 Based on the embodiments, further, such as Figure 11 As shown, in one embodiment of the present invention, the evaluation quantification result generation unit 705 includes:

[0220] The mapping relationship extraction module 1101 is used to extract the mapping relationship between the automated operation point and the functional operation point from the mapping table between the automated operation point and the functional operation point.

[0221] The third judgment module 1102 is used to determine whether the automated operation point exists in the automated operation point table and whether the corresponding functional operation point exists in the functional operation point table;

[0222] The first evaluation module 1103 is used to determine operation point redundancy if the automated operation point exists in the automated operation point table but the corresponding functional operation point does not exist in the functional operation point table.

[0223] The second evaluation module 1104 is used to determine that an operation point is missing if the function operation point exists in the function operation point table but the corresponding automation operation point does not exist in the automation operation point table.

[0224] Figure 12 This is a sixth structural schematic block diagram of the automated test and evaluation quantification device for operation points provided in this embodiment of the invention. Figure 7 Based on the embodiments, further, such as Figure 12 As shown, in one embodiment of the present invention, the automated test evaluation and quantification device for operation points of the present invention further includes:

[0225] Operation point deletion unit 1201 is used to delete obsolete automated operation point data based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache.

[0226] Figure 13 This is the seventh structural schematic block diagram of the automated test and evaluation quantification device for operation points provided in the embodiments of the present invention. Figure 12 Based on the embodiments, further, such as Figure 13 As shown, in one embodiment of the present invention, the operation point deletion unit 1201 includes:

[0227] The fourth judgment module 1301 is used to determine whether the automated case has been executed at preset time intervals based on the case number and generation time read from the second cache.

[0228] The operation point filtering module 1302 is used to filter automated operation points that have not been updated within a preset time range from the automated operation point table if the automated case has been completed.

[0229] The attribute reading module 1303 is used to read the attributes of the encrypted text of the automated operation point corresponding to the automated operation point that has not been updated from the first cache;

[0230] The fifth judgment module 1304 is used to determine whether the automation operation point that has not been updated in this automation case has been generated based on the generation time in the second cache and the attribute.

[0231] The operation point deletion module 1305 is used to delete the corresponding automated operation point data from the automated operation point table and the mapping table between automated operation points and functional operation points if the operation point is not generated.

[0232] This invention provides a method and apparatus for quantifying automated test evaluation based on operation points. The method involves: acquiring functional case text and a mapping table between keywords and operation point types; extracting functional operation point data from the functional case text based on the mapping table to generate a functional operation point table; parsing operation point messages sent by the automated program and updating the automated operation point table; matching automated operation points with functional operation points based on the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points; and generating a quantitative evaluation result for automated testing based on the functional operation point table, the automated operation point table, and the mapping table. This method achieves objective quantification, real-time updating, and accurate output of evaluation results for automated test coverage, significantly improving the efficiency and reliability of test effectiveness evaluation.

[0233] This invention receives operation point messages sent in real time during automated test execution and performs rapid comparison based on the encrypted text of the operation points to determine whether the operation point content has changed. The automated operation point table is updated only when changes occur, thus avoiding the performance overhead of repeated writes. Furthermore, combined with a discarded automated operation point deletion mechanism, historical operation point data not generated in the current execution cycle can be automatically removed, achieving continuous real-time updates of operation point data.

[0234] Figure 14This is a schematic diagram of the structure of the computer device provided in an embodiment of the present invention, such as... Figure 14 As shown, the electronic device may include a processor 1401, a communications interface 1402, a memory 1403, and a communication bus 1404, wherein the processor 1401, the communications interface 1402, and the memory 1403 communicate with each other via the communication bus 1404. The processor 1401 can call logical instructions in the memory 1403 to execute the following methods: obtaining functional case text and a mapping table of keywords and operation point types; extracting functional operation point data from the functional case text according to the mapping table of keywords and operation point types to generate a functional operation point table; parsing operation point messages sent by the received automation program and updating the automation operation point table; matching automation operation points with functional operation points according to the functional operation point table and the automation operation point table to generate a mapping table of automation operation points and functional operation points; and generating an evaluation quantification result of the automation test based on the functional operation point table, the automation operation point table, and the mapping table of automation operation points and functional operation points.

[0235] Furthermore, the logical instructions in the aforementioned memory 1403 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a top-drive control center server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0236] This embodiment discloses a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided in the above-described method embodiments, such as: obtaining functional case text and a mapping table between keywords and operation point types; extracting functional operation point data from the functional case text according to the mapping table between keywords and operation point types to generate a functional operation point table; parsing operation point messages sent by the received automation program and updating the automation operation point table; matching automation operation points with functional operation points according to the functional operation point table and the automation operation point table to generate a mapping table between automation operation points and functional operation points; and generating an evaluation quantification result of the automation test according to the functional operation point table, the automation operation point table, and the mapping table between automation operation points and functional operation points.

[0237] This embodiment provides a computer-readable storage medium storing a computer program that causes a computer to execute the methods provided in the above-described method embodiments. For example, the methods include: obtaining functional case text and a mapping table between keywords and operation point types; extracting functional operation point data from the functional case text according to the mapping table to generate a functional operation point table; parsing operation point messages sent by a received automation program and updating the automation operation point table; matching automation operation points with functional operation points according to the functional operation point table and the automation operation point table to generate a mapping table between automation operation points and functional operation points; and generating an evaluation quantification result for the automation test based on the functional operation point table, the automation operation point table, and the mapping table between automation operation points and functional operation points.

[0238] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0239] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0240] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0241] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0242] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0243] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A quantitative method for automated testing and evaluation oriented towards operation points, characterized in that, include: Obtain a mapping table of functional case texts and keywords to operation point types; Based on the mapping table between keywords and operation point types, extract functional operation point data from the functional case text to generate a functional operation point table; The received operation point messages from the automation program are parsed, and the automation operation point table is updated. Based on the functional operation point table and the automated operation point table, the automated operation points and functional operation points are matched to generate a mapping table between automated operation points and functional operation points. The evaluation and quantitative results of the automated test are generated based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

2. The automated test evaluation and quantification method for operation points according to claim 1, characterized in that, The step of extracting functional operation point data from the functional case text based on the mapping table between keywords and operation point types, and generating a functional operation point table, includes: Read the mapping table between keywords and operation point types. The mapping table between keywords and operation point types includes keyword number, keyword name, keyword hit rule and operation point type number. Based on the keyword hitting rules, text recognition is performed on the functional case text to obtain functional operation point data; A function operation point table is generated based on the function operation point data; the function operation point table includes a unique function operation point number, a case number, an operation point type number, and detailed operation point information.

3. The automated test evaluation and quantification method for operation points according to claim 1, characterized in that, The step of parsing the received operation point messages sent by the automation program and updating the automation operation point table includes: Read the automated operation point table; the automated operation point table includes: case number, operation point type number, unique automated operation point number, detailed operation point content, encrypted text of automated operation point, generation time and update time; Write the encrypted text of the automated operation point into the first cache; Receive the operation point message sent by the automation program, and determine whether the encrypted text of the automation operation point in the operation point message exists in the first cache; If it exists, the generation time of the operation point message is written into the first cache as an attribute of the automated operation point encrypted text; If it does not exist, determine whether the unique automation operation point number of the operation point message exists in the automation operation point table; If it exists, update the automated operation point table according to the operation point message; If it does not exist, insert the operation point message as new data into the automated operation point table, write the corresponding automated operation point encrypted text into the first cache, and write the case number and generation time into the second cache.

4. The automated test evaluation and quantification method for operation points according to claim 1, characterized in that, The step of matching automated operation points with functional operation points according to the functional operation point table and the automated operation point table to generate a mapping table between automated operation points and functional operation points includes: Based on the case number and operation point type number of the automation operation point data in the automation operation point table, obtain the corresponding functional operation point data from the functional operation point table; Based on preset matching rules, the detailed operation point content of the automated operation point data, and the detailed operation point content of the corresponding functional operation point data, the automated operation points and functional operation points are matched to generate a mapping table of automated operation points and functional operation points; the mapping table of automated operation points and functional operation points includes a unique number of automated operation points, a unique number of functional operation points, and a case number.

5. The automated test evaluation and quantification method for operation points according to claim 1, characterized in that, The step of generating an evaluation quantification result for automated testing based on the functional operation point table, the automated operation point table, and the mapping table between automated operation points and functional operation points includes: Extract the mapping relationship between the automated operation points and the functional operation points from the mapping table of the automated operation points and the functional operation points; Determine whether the automated operation point exists in the automated operation point table and whether the corresponding functional operation point exists in the functional operation point table; If the automated operation point exists in the automated operation point table but the corresponding functional operation point does not exist in the functional operation point table, then operation point redundancy is determined. If the function operation point exists in the function operation point table but the corresponding automation operation point does not exist in the automation operation point table, then the operation point is determined to be missing.

6. The automated test evaluation and quantification method for operation points according to claim 1, characterized in that, Also includes: Based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache, discarded automated operation point data is deleted.

7. The automated test evaluation and quantification method for operation points according to claim 6, characterized in that, The deletion of obsolete automated operation point data based on the generation time recorded in the second cache and the attributes of the encrypted text of the automated operation point recorded in the first cache includes: At preset time intervals, the system determines whether the automated case has been completed based on the case number and generation time read from the second cache. If the automation case has been completed, filter the automation operation points that have not been updated within a preset time range from the automation operation point table; Read the attributes of the encrypted text of the automation operation point corresponding to the automation operation point that has not been updated from the first cache; Based on the generation time in the second cache and the attribute, it is determined whether the automation operation point that has not been updated in this automation case has been generated; If not generated, the corresponding automated operation point data is deleted from the automated operation point table and the mapping table between automated operation points and functional operation points.

8. An automated testing and evaluation quantification device oriented towards operation points, characterized in that, include: The data acquisition unit is used to acquire the mapping table between functional case text and keywords and operation point types; The function operation point table generation unit is used to extract function operation point data from the function case text according to the mapping table between the keywords and operation point types, and generate a function operation point table. The automated operation point table update unit is used to parse the operation point messages sent by the received automated program and update the automated operation point table. The mapping table generation unit is used to match automated operation points with functional operation points according to the functional operation point table and the automated operation point table, and generate a mapping table between automated operation points and functional operation points. The evaluation quantification result generation unit is used to generate the evaluation quantification result of the automated test based on the functional operation point table, the automated operation point table, and the mapping table between the automated operation points and the functional operation points.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images