A method and apparatus for verifying an expected result

Abstract

The application discloses a method and device for verifying expected results, and relates to the technical field of automatic testing, which comprises the following steps: obtaining an expected result string of a test case; performing format identification on the expected result string to determine the format category of the expected result string; determining a target verification mode corresponding to the expected result string from a plurality of preset verification modes based on the format category; and performing a differential verification operation corresponding to the target verification mode on an actual result string according to the target verification mode to obtain a verification result.

Description

Technical Field

[0001] This application relates to the field of automated testing technology, and in particular to a method and apparatus for verifying expected results. Background Technology

[0002] Existing automated testing frameworks typically use pre-defined, fixed verification logic to compare expected and actual results during the test case execution verification phase. This makes the verification process incompatible with different formats of expected results, requiring testers to manually configure verification rules or write additional scripts for different formats, thus limiting the efficiency and accuracy of automated testing. Therefore, a new method for verifying expected results is urgently needed to address the aforementioned technical problems. Summary of the Invention

[0003] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solutions, nor is it intended to determine the scope of protection of the claimed technical solutions.

[0004] Firstly, this application provides a method for verifying expected results, comprising: Get the expected result string of the test case; The expected result string is formatted to determine its format category. Based on the format category, a target verification mode corresponding to the expected result string is determined from a variety of preset verification modes; Based on the target verification pattern, perform a differential verification operation corresponding to the target verification pattern on the actual result string to obtain the verification result.

[0005] In some implementations, the step of performing format recognition on the expected result string to determine the format category of the expected result string includes: Based on a preset set of regular expressions, a format feature matching operation is performed on the expected result string to obtain the matching result; Based on the matching results, the format category of the expected result string is determined, wherein the format category includes multi-condition fuzzy comparison format, exact match triggered format, or single-condition fuzzy comparison format.

[0006] In some implementations, the preset regular expression set includes a first regular expression and a second regular expression. The step of performing format feature matching on the expected result string based on the preset regular expression set to obtain the matching result includes: Based on the first regular expression, a first matching operation is performed on the expected result string to obtain a first matching sub-result, wherein the first matching sub-result is used to indicate whether the expected result string contains a numeric sequence number; Based on the second regular expression, a second matching operation is performed on the expected result string to obtain a second matching sub-result, wherein the second matching sub-result is used to indicate whether the expected result string contains a preset exact matching keyword; The matching result is generated based on the first matching sub-result and the second matching sub-result.

[0007] In some implementations, determining the format category of the expected result string based on the matching result includes: When the first matching sub-result indicates that the expected result string contains at least two numeric indices, the format category is determined to be the multi-condition fuzzy comparison format; When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string contains a preset exact match keyword, the format category is determined to be the exact match trigger format; When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string does not contain a preset exact matching keyword, the format category is determined to be the single-condition fuzzy comparison format.

[0008] In some implementations, the target verification pattern includes a multi-condition fuzzy verification pattern. The step of performing a differential verification operation corresponding to the target verification pattern on the actual result string to obtain the verification result includes: When the target verification mode is the multi-condition fuzzy verification mode, a condition extraction operation is performed on the expected result string to obtain multiple strings to be verified. The actual result string is subjected to text normalization to obtain a normalized actual result string; Iterate through the multiple strings to be verified, and determine in turn whether each string to be verified exists in the normalized actual result string; The verification result is determined to be successful when each string to be verified exists in the normalized actual result string; or, the verification result is determined to be unsuccessful when any string to be verified does not exist in the normalized actual result string.

[0009] In some implementations, the conditional extraction operation on the expected result string yields multiple strings to be verified, including: Based on preset multi-condition extraction rules, the expected result string is subjected to conditional segmentation to obtain multiple candidate text lines starting with a number and the corresponding number for each candidate text line. For each candidate text line, a keyword location operation is performed, and the first character position after the end position of the preset keyword in the candidate text line is determined as the content start position; Based on the starting position of the content, a content truncation operation is performed on each candidate text line to obtain multiple strings to be verified.

[0010] In some implementations, the target verification pattern includes an exact match verification pattern. The step of performing a differential verification operation on the actual result string corresponding to the target verification pattern, based on the target verification pattern, to obtain the verification result, includes: When the target verification mode is the exact match verification mode, the exact content extraction operation is performed on the expected result string to obtain the exact match string; The actual result string is subjected to text normalization to obtain a normalized actual result string; Determine whether the exact matching string is exactly equal to the normalized actual result string; The verification result is determined to be successful when the exact matching string is exactly equal to the normalized actual result string; or, the verification result is determined to be unsuccessful when the exact matching string is not exactly equal to the normalized actual result string.

[0011] In some implementations, the target verification pattern includes a single-condition fuzzy verification pattern. The step of performing a differential verification operation corresponding to the target verification pattern on the actual result string to obtain the verification result includes: When the target verification mode is the single-condition fuzzy verification mode, the expected result string is determined as the string to be verified; The actual result string is subjected to text normalization to obtain a normalized actual result string; Determine whether the string to be verified exists in the normalized actual result string; When the string to be verified exists in the normalized actual result string, the verification result is determined to be verified successfully; or, when the string to be verified does not exist in the normalized actual result string, the verification result is determined to be verified unsuccessfully.

[0012] In some implementations, the text normalization process performed on the actual result string to obtain a normalized actual result string includes: Perform a special character removal operation on the actual result string to obtain the first intermediate string; Perform a case-consistent conversion on the first intermediate string to obtain the second intermediate string; Perform a whitespace removal operation on the second intermediate string to obtain the normalized actual result string.

[0013] Secondly, this application provides a verification device for expected results, comprising: The result acquisition unit is used to obtain the expected result string of the test case; A format recognition unit is used to recognize the format of the expected result string and determine the format category of the expected result string; The pattern determination unit is used to determine the target verification pattern corresponding to the expected result string from a variety of preset verification patterns based on the format category. The verification execution unit is used to perform a differential verification operation on the actual result string according to the target verification pattern, and obtain the verification result.

[0014] In summary, the expected result verification method provided in this application provides a comparison benchmark for automated verification by obtaining the expected result string of the test case, avoiding operational errors and efficiency losses caused by testers manually inputting or configuring expected results. By recognizing the format of the expected result string and determining its format category, the verification process automatically distinguishes between different formats of expected result input, eliminating the need to pre-write independent verification scripts for each format and improving the compatibility of the verification method with different input formats. Based on the identified format category, the corresponding target verification mode is matched from multiple preset verification modes, realizing an adaptive association between the verification logic and the expected result format. This ensures that each format of expected result is processed using verification rules that are compatible with it, avoiding verification deviations caused by mismatch between verification rules and expected result formats. Based on the determined target verification pattern, a differentiated verification operation corresponding to that pattern is performed on the actual result string. This allows the verification actions to check the actual result item by item according to the format characteristics of the expected result. Different formats of expected results correspond to different verification logics, thus accommodating multiple verification needs within a single verification process. The final verification result accurately reflects the true comparison between the actual result and the expected result, improving the accuracy and reliability of result verification in automated testing scenarios. This application, through the linkage mechanism of format recognition, pattern determination, and differentiated verification, enables the verification method to adaptively handle expected results in multiple formats. This reduces the manual configuration work required during testing due to changes in the format of expected results. At the same time, by employing corresponding verification algorithms for different formats, verification bias caused by format recognition errors or mismatched verification logic is reduced. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic flowchart of a method for verifying expected results provided in an embodiment of this application; Figure 2 This is a schematic diagram of a verification device for expected results provided in an embodiment of this application. Detailed Implementation

[0016] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0017] The method for verifying expected results proposed in this application can be applied to various automated testing scenarios, especially complex testing scenarios such as mobile terminal system testing, server-side interface testing in containerized environments, and functional verification of embedded devices. In mobile terminal automated testing, testers typically need to write test cases, which include the expected output after performing a certain operation, such as text information displayed on the interface, specific fields output by the system log, or string content returned by the command line. However, during actual test execution, the actual results returned by the system under test often differ in format from the expected results. For example, the expected results may contain multiple conditions that need to be met simultaneously, specific strings that require exact matching, or keywords that only require fuzzy matching.

[0018] Traditional testing frameworks typically require testers to pre-write corresponding verification logic for each expected result format. When the number of test cases is large, this manual configuration of verification rules leads to redundant test scripts and high maintenance costs. The method in this application is deployed in a continuous integration pipeline as the verification engine of an automated testing platform. It adaptively compares the actual result strings generated during test execution with the pre-set expected result strings in the test cases, achieving automatic judgment of various test results and improving the overall efficiency and reliability of the automated testing process.

[0019] To clearly illustrate the technical solution of this application, some of the terms used in this application will be explained below.

[0020] The expected result string in this application refers to a text format that testers pre-define in the test case to describe the correct output content that should be obtained after the test is executed. This string may contain one or more verification conditions, and its specific format determines the comparison method used in the verification process.

[0021] The actual result string in this application refers to the text form of the actual output content obtained from the system or device under test during the execution of the test case, and is the target object of the verification operation.

[0022] The format categories in this application refer to the types classified according to the textual structure characteristics of the expected result string, used to distinguish different forms of verification requirements. In this application, the format categories include three types: multi-condition fuzzy comparison format, exact match triggered format, and single-condition fuzzy comparison format. Specifically, the multi-condition fuzzy comparison format refers to an expected result string containing multiple parallel verification conditions, each starting with a numerical sequence number, and verification only requires the actual result to contain the text content corresponding to these conditions, not necessarily identical. The exact match triggered format refers to an expected result string containing a preset exact match keyword, which is used to indicate that the verification process must perform an exact equality judgment on the actual result. The single-condition fuzzy comparison format refers to an expected result string containing only one verification condition, and verification only requires the actual result to contain the text content corresponding to that condition.

[0023] The multiple preset verification modes in this application refer to a pre-defined set of verification logic corresponding to different format categories, with each format category corresponding to one verification mode. Specifically, the verification mode corresponding to the multi-condition fuzzy comparison format is the multi-condition fuzzy verification mode, the verification mode corresponding to the exact match trigger format is the exact match verification mode, and the verification mode corresponding to the single-condition fuzzy comparison format is the single-condition fuzzy verification mode.

[0024] The target verification mode in this application refers to the specific verification mode determined and selected from a variety of preset verification modes based on the format category of the expected result string, which is used to guide subsequent differential verification operations.

[0025] The string to be verified in this application refers to the specific text content extracted from the expected result string and used for comparison with the actual result string. For multi-condition fuzzy verification mode, there are multiple strings to be verified; for single-condition fuzzy verification mode, there is one string to be verified; for exact match verification mode, the string to be verified is the exact match string.

[0026] The normalized actual result string in this application refers to the text obtained after performing text normalization processing on the actual result string. Normalization processing includes operations such as removing special characters, standardizing capitalization, and removing leading and trailing whitespace, aiming to eliminate the impact of format differences or symbol interference on the text comparison results and improve the accuracy of verification.

[0027] The verification result of this application refers to the conclusion output after performing differential verification on the expected result string and the actual result string, which is used to characterize whether the test case passes or fails.

[0028] The preset regular expression set in this application refers to a set of regular expressions used to detect the string format characteristics of the expected results. The first regular expression is used to match lines that start with a number sequence, and the second regular expression is used to match exact match trigger keywords.

[0029] The preset keyword in this application refers to a specific string used to identify the exact match requirement in the exact match trigger format, such as "equal to in the returned result". When the expected result string contains this keyword, it indicates that the use case needs to perform an exact equality verification operation.

[0030] The preset multi-condition extraction rules in this application refer to the rules used to extract each condition to be verified from the expected result string of the multi-condition fuzzy comparison format. Specifically, they include operations such as splitting by line, locating the starting position of the content after the preset keywords, and truncating the condition content.

[0031] Please see Figure 1 This is a schematic flowchart of a method for verifying expected results provided in an embodiment of this application, which may specifically include: S110. Obtain the expected result string of the test case; For example, the expected result string is pre-defined by the tester when writing the test case to describe the correct output content expected after the test is executed. During the actual automated test execution, the string is read from the storage location of the test case and used as the input object for format recognition and differential verification.

[0032] S120. Perform format recognition on the expected result string and determine the format category of the expected result string; For example, after obtaining the expected result string, by analyzing the text structure features of the string, it can be classified into one of the preset multiple format categories. These format categories correspond to the verification mode, so that the format recognition result can be directly used to determine the execution path of the verification operation.

[0033] S130. Based on the format category, determine the target validation pattern corresponding to the expected result string from a variety of preset validation patterns; For example, after determining the format category of the expected result string, a target verification mode that matches the current expected result string can be selected from a variety of preset verification modes based on the correspondence between the format category and the preset verification mode. The target verification mode is used to limit the verification logic type used when performing differentiated verification operations on the actual result string, so that the verification process can automatically adapt to the corresponding processing path according to the format characteristics of the expected result and convert the format recognition result into an executable verification path.

[0034] S140. Based on the target verification pattern, perform a differential verification operation on the actual result string that corresponds to the target verification pattern to obtain the verification result.

[0035] For example, after determining the target verification pattern, a differential verification operation is performed. This operation uses the verification logic corresponding to the target verification pattern as the execution basis and the actual result string as the operation object. Depending on the target verification pattern, different verification logics, including conditional existence judgment or string equality judgment, are performed on the actual result string. Through the operation of the above verification logic, a verification result is generated to represent the comparison between the actual result string and the expected result string.

[0036] In summary, the expected result verification method provided in this application provides a comparison benchmark for automated verification by obtaining the expected result string of the test case, avoiding operational errors and efficiency losses caused by testers manually inputting or configuring expected results. By recognizing the format of the expected result string and determining its format category, the verification process automatically distinguishes between different formats of expected result input, eliminating the need to pre-write independent verification scripts for each format and improving the compatibility of the verification method with different input formats. Based on the identified format category, the corresponding target verification mode is matched from multiple preset verification modes, realizing an adaptive association between the verification logic and the expected result format. This ensures that each format of expected result is processed using verification rules that are compatible with it, avoiding verification deviations caused by mismatch between verification rules and expected result formats. Based on the determined target verification pattern, a differentiated verification operation corresponding to that pattern is performed on the actual result string. This allows the verification actions to check the actual result item by item according to the format characteristics of the expected result. Different formats of expected results correspond to different verification logics, thus accommodating multiple verification needs within a single verification process. The final verification result accurately reflects the true comparison between the actual result and the expected result, improving the accuracy and reliability of result verification in automated testing scenarios. This application, through the linkage mechanism of format recognition, pattern determination, and differentiated verification, enables the verification method to adaptively handle expected results in multiple formats. This reduces the manual configuration work required during testing due to changes in the format of expected results. At the same time, by employing corresponding verification algorithms for different formats, verification bias caused by format recognition errors or mismatched verification logic is reduced.

[0037] In some instances, the expected result string is format-identified to determine its format category, including: Based on a pre-defined set of regular expressions, the expected result string is subjected to format feature matching to obtain the matching result; Based on the matching results, determine the format category of the expected result string, where the format category includes multi-condition fuzzy comparison format, exact match triggered format, or single-condition fuzzy comparison format.

[0038] For example, when performing format recognition on the expected result string, a format feature matching operation is first performed on the string based on a preset set of regular expressions. The preset set of regular expressions includes a first regular expression and a second regular expression. The first regular expression is used to detect whether the expected result string contains a numerical sequence, and the second regular expression is used to detect whether the expected result string contains a preset exact matching keyword. Through the above matching operation, a first matching sub-result and a second matching sub-result are obtained respectively, and the two sub-results are combined to form a complete matching result.

[0039] Based on the combination of the first and second matching sub-results in the matching results, the format category of the expected result string is determined from preset format categories. When the first matching sub-result indicator contains a numeric sequence number and the second matching sub-result indicator does not contain a preset exact match keyword, the format category is determined to be a multi-condition fuzzy comparison format. When the second matching sub-result indicator contains a preset exact match keyword, the format category is determined to be an exact match triggered format. When the first matching sub-result indicator does not contain a numeric sequence number and the second matching sub-result indicator does not contain a preset exact match keyword, the format category is determined to be a single-condition fuzzy comparison format.

[0040] In summary, the format recognition process described in this application uses regular expressions to parse the structural features of the expected result string, transforming different forms of expected result input into a unified format category label. This avoids the burden of manually identifying the expected result format and configuring verification rules. By combining the first and second matching sub-results, three format categories are distinguished, preventing incorrect verification logic matching due to format classification errors. This approach is compatible with multiple expected result formats within a single verification framework, improving the verification method's adaptability to different input formats.

[0041] In some instances, the preset set of regular expressions includes a first regular expression and a second regular expression. Based on the preset set of regular expressions, a format feature matching operation is performed on the expected result string to obtain the matching result, including: Based on the first regular expression, the first matching operation is performed on the expected result string to obtain the first matching sub-result, wherein the first matching sub-result is used to indicate whether the expected result string contains a numeric ordinal number; Based on the second regular expression, a second matching operation is performed on the expected result string to obtain a second matching sub-result. The second matching sub-result is used to indicate whether the expected result string contains a preset exact matching keyword. Based on the first and second matching sub-results, a matching result is generated.

[0042] For example, the first regular expression is pre-configured to detect text lines that begin with a numeric sequence. This regular expression determines whether the expected result string contains multi-conditional formatting by matching the text pattern of a numeric sequence at the beginning of the line followed by a delimiter. When performing the first matching operation, the first regular expression is applied to the entire expected result string, and the regular expression engine scans and matches the string. If at least one line of text in the expected result string matches the starting pattern of a numeric sequence followed by a delimiter, the first matching operation returns a successful match indication, and this indication is taken as the first matching sub-result. If no text line matching the pattern exists in the expected result string, the first matching operation returns an indication of match failure, and this indication is taken as the first matching sub-result. Through the above first matching operation, a first matching sub-result is obtained to characterize whether the expected result string contains a numeric sequence number.

[0043] The second regular expression is pre-configured to detect a preset exact match keyword, which is a specific string, such as "equals in the returned result," used to indicate that the expected result string needs to undergo exact match verification. During the second matching operation, this second regular expression is applied to the entire expected result string. The regular expression engine performs substring matching on the string. If a substring in the expected result string matches the preset exact match keyword, the second matching operation returns a successful match indication, which is taken as the second matching sub-result. If no substring in the expected result string exactly matches the preset exact match keyword, the second matching operation returns a failed match indication, which is also taken as the second matching sub-result. Through the above second matching operation, a second matching sub-result is obtained, representing whether the expected result string contains the preset exact match keyword.

[0044] In summary, after obtaining the first and second matching sub-results, this embodiment generates a matching result based on these two sub-results. This matching result, composed of the first and second matching sub-results, characterizes the format features of the expected result string in two dimensions: the existence of numeric sequences and the existence of preset exact matching keywords. Through the aforementioned format feature matching operation, the original text structure of the expected result string is transformed into a matching result composed of two Boolean indicators. This matching result serves as the direct basis for format category determination, enabling the format recognition process to uniquely determine the format category of the expected result string from multiple preset format categories based on the combined state of the two sub-results in the matching result. This avoids ambiguity in format category determination caused by the inability of a single-dimensional feature to distinguish between multi-condition fuzzy comparison formats and single-condition fuzzy comparison formats. The aforementioned format feature matching operation, through the first and second matching sub-results, achieves two-dimensional decoupling of the format features of the expected result string, allowing the format recognition process to independently detect the existence of numeric sequences and the existence of preset exact matching keywords. This avoids the complexity of matching logic and feature coupling caused by simultaneously matching multiple features in a single regular expression.

[0045] In some instances, the format category of the expected result string is determined based on the matching results, including: When the first matching sub-result indicates that the expected result string contains at least two numeric ordinal numbers, the format category is determined to be a multi-condition fuzzy comparison format. When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string contains a preset exact match keyword, the format category is determined to be an exact match triggered format; When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string does not contain a preset exact matching keyword, the format category is determined to be a single-condition fuzzy comparison format.

[0046] For example, when the first matching sub-result indicates that the expected result string contains at least two numeric indices, the format category is determined to be a multi-condition fuzzy comparison format. Specifically, after obtaining the first matching sub-result, the sub-result is parsed. If the sub-result indicates that there are two or more text lines in the expected result string that begin with numeric indices, it means that the expected result string adopts a multi-line parallel conditional organization form in terms of structure. The numeric indices serve as explicit identifiers for conditional segmentation, and their number reflects the number of independent validation conditions contained in the expected result.

[0047] When the number of numerical sequences reaches two or more, it indicates that the expected result string contains multiple parallel verification conditions, each corresponding to an independent content to be verified. In this case, there is no need to further judge the second matching sub-result, because even if the expected result string also contains a preset exact match keyword, this keyword can only be attached to a specific sequence number line in terms of structure. The multi-condition format itself requires performing inclusion checks on all conditions one by one, which conflicts with the exact match's complete equality verification logic. Through the above judgment method, the format category of the expected result string is determined to be a multi-condition fuzzy comparison format. This format category is used to indicate that the verification process should extract multiple condition contents from the expected result string and perform inclusion checks on each condition one by one, rather than performing an overall equality comparison on the actual result string.

[0048] When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string contains a preset exact match keyword, the format category is determined to be an exact match triggered format. Specifically, after obtaining the first matching sub-result, it is first parsed. If the sub-result indicates that the expected result string contains only one text line starting with a numeric sequence number, it means that the expected result string structurally contains only one independent conditional unit. Based on this, the second matching sub-result is further judged. If the sub-result indicates that the conditional unit contains a substring that is completely identical to the preset exact match keyword, it means that although the expected result string adopts the numeric sequence number format, its semantic essence requires the execution of an exact equality verification. The appearance of the preset exact match keyword plays a decisive role in the format category determination in this scenario. It directs the expected result string, which might originally be classified as a single-condition fuzzy comparison format, to the exact match verification path. Through the above determination method, the format category of the expected result string is determined to be an exact match triggered format. This format category is used to indicate that the verification process should extract exact match content from the expected result string and perform an exact equality verification on the actual result string.

[0049] When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string does not contain a preset exact match keyword, the format category is determined to be a single-condition fuzzy comparison format. Specifically, after obtaining the first matching sub-result, it is first parsed. If the sub-result indicates that the expected result string contains only one text line starting with a numeric sequence number, it means that the expected result string structurally contains only one independent condition unit. Based on this, the second matching sub-result is further judged. If the sub-result indicates that the preset exact match keyword does not appear in the condition unit, it means that the single numeric sequence number is only used as a regular number, rather than as identification information for forced exact matching. At this time, the verification requirement for the expected result string is to perform a single condition inclusion judgment on the actual result string, which is different from the multiple conditions required for verification one by one in the multi-condition fuzzy comparison format, and also different from the exact match trigger format required for complete equality verification. Using the above determination method, the format category of the expected result string is determined to be a single-condition fuzzy comparison format. This format category is used to indicate that the verification process should treat the single condition content extracted from the expected result string as a string to be verified, and perform an inclusion judgment on the actual result string.

[0050] In summary, the format category determination process described in this embodiment transforms the original text structure of the expected result string into structured format category labels through the collaborative determination of the number of numeric serial numbers and the existence of preset exact matching keywords, thus achieving the division into three format categories. The number of numeric serial numbers serves as the first-level determination dimension, first distinguishing the boundary between multi-conditional and single-conditional formats. At least two numeric serial numbers directly correspond to multi-conditional fuzzy comparison formats, while a single numeric serial number enters the second-level determination dimension. The preset exact matching keywords serve as the basis for the second-level determination, further distinguishing the expected result string into exact match triggered formats and single-conditional fuzzy comparison formats in the context of a single numeric serial number. Through the collaboration of these two-level determination mechanisms, the format recognition process accurately distinguishes the three format categories. Multi-conditional fuzzy comparison formats do not rely on the determination of preset exact matching keywords, while exact match triggered formats and single-conditional fuzzy comparison formats are distinguished based on a single numeric serial number using preset exact matching keywords. This ensures that each type of expected result string is classified into its corresponding format category, providing a category basis for differential verification operations.

[0051] In some instances, the target validation pattern includes a multi-condition fuzzy validation pattern. Based on the target validation pattern, a differential validation operation corresponding to the target validation pattern is performed on the actual result string to obtain the validation result, including: When the target verification mode is a multi-condition fuzzy verification mode, condition extraction is performed on the expected result string to obtain multiple strings to be verified. Perform text normalization on the actual result string to obtain the normalized actual result string; Iterate through multiple strings to be verified, and check in turn whether each string to be verified exists in the normalized actual result string; The verification result is determined to be successful if every string to be verified exists in the normalized actual result string; or, the verification result is determined to be unsuccessful if any string to be verified does not exist in the normalized actual result string.

[0052] For example, when the target verification mode is a multi-condition fuzzy verification mode, the verification process first performs a condition extraction operation on the expected result string. This condition extraction operation is based on a preset multi-condition extraction rule, which specifically includes condition segmentation, keyword positioning, and content truncation operations on the expected result string. Specifically, based on the condition segmentation operation in the preset multi-condition extraction rule, the expected result string is segmented by line, identifying multiple candidate text lines that begin with a numerical sequence number, and obtaining the numerical sequence number corresponding to each candidate text line. For each candidate text line, a keyword positioning operation is performed to locate the end position of a preset keyword in the candidate text line. The preset keyword can be "contains in the returned result," and the first character position after this end position is determined as the content start position. Based on the content start position, a content truncation operation is performed on each candidate text line, truncating from the content start position to the end of the line, resulting in multiple strings to be verified. Through the above condition extraction operation, the expected result string, originally organized in the form of multiple lines of numerical sequence numbers, is decomposed into an independent set of strings to be verified, and each string to be verified corresponds to a text segment that needs to be verified in the actual result string.

[0053] After obtaining multiple strings to be verified, the verification process performs text normalization on the actual result strings to obtain a normalized actual result string. The text normalization process specifically includes special character removal, case conversion, and leading / tailing whitespace removal. Special character removal removes preset special characters from the actual result string, including square brackets, curly braces, and dollar signs from JSON structures, which may interfere with text comparison. Case conversion converts all letters in the actual result string after special character removal to lowercase, eliminating text comparison discrepancies caused by case differences. Leading / tailing whitespace removal removes invisible characters such as spaces and newlines at the beginning and end of the converted string. Through the above text normalization process, a normalized actual result string corresponding to the actual result string is obtained. This normalized actual result string eliminates the influence of format differences on text comparison, providing a comparison benchmark for condition existence judgment.

[0054] After extracting the strings to be verified and normalizing the actual result string, the verification process iterates through multiple strings to be verified, sequentially determining whether each string exists in the normalized actual result string. The traversal is performed sequentially. For each of the multiple strings to be verified, a string inclusion check is performed. This inclusion check uses a substring matching algorithm to determine whether the string to be verified is a substring of the normalized actual result string. During the traversal, the inclusion check result of each string to be verified serves as a prerequisite for the next string to be verified. If the current string to be verified exists in the normalized actual result string, the check continues for the next string; if the current string to be verified does not exist in the normalized actual result string, the check for subsequent strings is terminated. Through this traversal and inclusion check, the verification process compares each of the multiple strings to be verified with the actual result string.

[0055] The final verification result is determined based on the results of the traversal judgment. When every string to be verified exists in the normalized actual result string during the traversal, it indicates that all verification conditions listed in the expected result string are satisfied in the actual result string, and the verification result is determined to be successful. When any string to be verified does not exist in the normalized actual result string during the traversal, it indicates that at least one verification condition in the expected result string fails to match in the actual result string, and the verification result is determined to be unsuccessful. This method of determining the verification result transforms the multi-condition verification requirement into a logical AND operation of multiple independent condition existence judgments, ensuring that a pass is determined only when all conditions are simultaneously satisfied, and a failure is determined if any condition is missing, which matches the verification requirements corresponding to the multi-condition fuzzy comparison format.

[0056] In summary, this application's embodiments decompose the expected result string (in the form of multiple lines of numerical sequences) into independent strings to be verified by performing condition extraction operations on the expected result string. This allows the verification operation to be judged condition by condition, avoiding the misprocessing of joint verification requirements of multiple conditions into a single string's overall matching. By performing text normalization processing on the actual result string, interference from special characters, capitalization differences, and leading and trailing whitespace on text comparison is eliminated. This ensures that the inclusion judgment between the string to be verified and the actual result string is not affected by format differences, solving the problem of misjudgment caused by special characters such as JSON symbols. By traversing multiple strings to be verified and sequentially judging whether each string to be verified exists in the normalized actual result string, the joint verification of multiple conditions is completed in a sequential manner. The judgment result of each condition affects the verification process. When any condition is missing, the traversal is terminated in advance, avoiding redundant operations on verifications that have already been determined to fail. By using a mechanism that determines whether a verification passes or fails based on the traversal results, the multi-condition verification requirement is transformed into a logical AND operation result of multiple condition existence judgments. This allows the verification result to reflect whether the actual result simultaneously contains all the conditions listed in the expected result, thus adapting to the verification requirement of multi-condition fuzzy comparison format.

[0057] In some instances, the target validation pattern includes an exact match validation pattern. Based on the target validation pattern, a differential validation operation corresponding to the target validation pattern is performed on the actual result string to obtain the validation result, including: When the target validation mode is exact match validation mode, perform exact content extraction on the expected result string to obtain the exact match string; Perform text normalization on the actual result string to obtain the normalized actual result string; Determine whether the exact match string is exactly equal to the normalized actual result string; The verification result is determined to be successful if the exact match string is exactly equal to the normalized actual result string; or, the verification result is determined to be unsuccessful if the exact match string is not exactly equal to the normalized actual result string.

[0058] For example, when the target verification mode is exact match verification mode, the verification process first performs an exact content extraction operation on the expected result string to obtain the exact match string. This exact content extraction operation is based on a preset exact match keyword, which can be "1. equal to in the returned result". Specifically, the end position of the preset exact match keyword is located in the expected result string, and the position of the first character after this end position is determined as the exact content start position. The text content is extracted from the exact content start position to the end of the expected result string, and the extracted text content is determined as the exact match string. Through the above exact content extraction operation, the comparison benchmark content used to perform the exact equality verification is separated from the expected result string. This comparison benchmark content is the target text that needs to be compared with the actual result during the verification process.

[0059] After obtaining the exact matching string, the verification process performs text normalization on the actual result string to obtain a normalized actual result string. Text normalization includes removing preset special characters from the actual result string, converting all letters in the result string after removing special characters to lowercase, and removing leading and trailing whitespace characters from the case-converted string. Through this text normalization process, a normalized actual result string corresponding to the actual result string is obtained. This normalized actual result string eliminates the influence of special characters, case differences, and leading and trailing whitespace on text comparison, ensuring that the exact equality judgment is performed on a consistent format basis.

[0060] After extracting the exact match string and normalizing the actual result string, the verification process determines whether the exact match string is completely equal to the normalized actual result string. This complete equality judgment is performed using a character-by-character comparison algorithm, comparing each character in the exact match string with the corresponding character in the normalized actual result string. If the two strings have the same length and all corresponding characters are the same, they are considered completely equal; if the two strings have different lengths or any corresponding characters are different, they are considered not completely equal.

[0061] The final verification result is determined based on the result of the exact equality judgment. When the exact match string is exactly equal to the normalized actual result string, it indicates that the actual result string, after normalization, matches the exact match content extracted from the expected result string, and the verification result is determined to be successful. When the exact match string is not exactly equal to the normalized actual result string, it indicates that there is a difference between the actual result string and the exact match content extracted from the expected result string, and the verification result is determined to be unsuccessful. This method of determining the verification result transforms the exact match verification requirement into a string equality judgment result, ensuring that a pass is only determined when the normalized actual result is exactly consistent with the expected benchmark.

[0062] In summary, this application's embodiments separate the exact matching requirement from ordinary conditional text by performing an exact content extraction operation based on preset exact matching keywords on the expected result string. This ensures that the verification process accurately identifies the expected content requiring complete equality verification, avoiding misprocessing exact matching content as fuzzy inclusion conditions. By performing text normalization on the actual result string, interference from special characters, capitalization differences, and leading / ending spaces on the complete equality judgment is eliminated. This ensures that the verification process compares strings on a standardized basis, avoiding misjudgments caused by format differences. By performing a complete equality judgment and determining whether the verification passes or fails based on the judgment result, the verification result accurately reflects whether the actual result is consistent with the expected content and adapts to the complete equality verification requirement corresponding to the exact matching trigger format.

[0063] In some instances, the target validation pattern includes a single-condition fuzzy validation pattern. Based on the target validation pattern, a differential validation operation corresponding to the target validation pattern is performed on the actual result string to obtain the validation result, including: When the target verification mode is a single-condition fuzzy verification mode, the expected result string is determined as the string to be verified; Perform text normalization on the actual result string to obtain the normalized actual result string; Determine if the string to be verified exists in the normalized actual result string; The verification result is determined to be successful if the string to be verified exists in the normalized actual result string; or, the verification result is determined to be unsuccessful if the string to be verified does not exist in the normalized actual result string.

[0064] For example, when the target validation mode is a single-condition fuzzy validation mode, the validation process first determines the expected result string as the string to be validated. Unlike the multi-condition fuzzy validation mode, which needs to extract multiple conditions from the expected result string, the single-condition fuzzy validation mode does not perform condition extraction. Instead, it treats the entire expected result string as the string to be validated, which is the text content whose existence needs to be verified in the actual result string.

[0065] After determining the string to be verified, the verification process performs text normalization on the actual result string to obtain a normalized actual result string. Text normalization includes removing preset special characters from the actual result string, converting all letters in the result string after removing special characters to lowercase, and removing leading and trailing whitespace characters from the converted string. Through this text normalization process, a normalized actual result string corresponding to the actual result string is obtained. This normalized actual result string eliminates the influence of special characters, case differences, and leading and trailing whitespace on text comparison.

[0066] After determining the string to be verified and normalizing the actual result string, the verification process determines whether the string to be verified exists in the normalized actual result string. This inclusion judgment operation is performed using a substring matching algorithm to determine whether the string to be verified is a substring of the normalized actual result string. Based on the result of the inclusion judgment, the final verification result is determined: if the string to be verified exists in the normalized actual result string, the verification result is determined to be successful; if the string to be verified does not exist in the normalized actual result string, the verification result is determined to be unsuccessful.

[0067] In summary, this embodiment of the application simplifies the verification process by directly determining the expected result string as the string to be verified, avoiding unnecessary parsing and extraction operations on the expected result of a single-condition format. By performing text normalization on the actual result string, interference from special characters, capitalization differences, and leading and trailing whitespace on the inclusion judgment is eliminated, ensuring that the matching between the string to be verified and the actual result string is unaffected by format differences. By performing an inclusion judgment and determining whether the verification passes or fails based on the judgment result, the verification result accurately reflects whether the actual result contains the conditional content of the expected result, thus adapting to the inclusion verification requirements corresponding to the single-condition fuzzy comparison format.

[0068] Please see Figure 2 The diagram below illustrates the structure of a verification device for expected results provided in this application embodiment, including: Result acquisition unit 21 is used to acquire the expected result string of the test case; The format recognition unit 22 is used to recognize the format of the expected result string and determine the format category of the expected result string; The pattern determination unit 23 is used to determine the target verification pattern corresponding to the expected result string from a variety of preset verification patterns based on the format category. The verification execution unit 24 is used to perform differential verification operations on the actual result string according to the target verification pattern, and obtain the verification result.

[0069] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features.

[0070] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications that fall outside the scope of this specification.

[0071] Obviously, those skilled in the art can make various modifications to this specification without departing from its spirit and scope. Therefore, this specification is intended to include any modifications that fall within the scope of the claims and their equivalents.

Claims

1. A method for verifying expected results, characterized in that, include: Get the expected result string of the test case; The expected result string is formatted to determine its format category. Based on the format category, a target verification mode corresponding to the expected result string is determined from a variety of preset verification modes; Based on the target verification pattern, perform a differential verification operation corresponding to the target verification pattern on the actual result string to obtain the verification result.

2. The method according to claim 1, characterized in that, The step of identifying the format of the expected result string and determining its format category includes: Based on a preset set of regular expressions, a format feature matching operation is performed on the expected result string to obtain the matching result; Based on the matching results, the format category of the expected result string is determined, wherein the format category includes multi-condition fuzzy comparison format, exact match triggered format, or single-condition fuzzy comparison format.

3. The method according to claim 2, characterized in that, The preset regular expression set includes a first regular expression and a second regular expression. The step of performing format feature matching on the expected result string based on the preset regular expression set to obtain the matching result includes: Based on the first regular expression, a first matching operation is performed on the expected result string to obtain a first matching sub-result, wherein the first matching sub-result is used to indicate whether the expected result string contains a numeric sequence number; Based on the second regular expression, a second matching operation is performed on the expected result string to obtain a second matching sub-result, wherein the second matching sub-result is used to indicate whether the expected result string contains a preset exact matching keyword; The matching result is generated based on the first matching sub-result and the second matching sub-result.

4. The method according to claim 3, characterized in that, The step of determining the format category of the expected result string based on the matching result includes: When the first matching sub-result indicates that the expected result string contains at least two numeric indices, the format category is determined to be the multi-condition fuzzy comparison format; When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string contains a preset exact match keyword, the format category is determined to be the exact match trigger format; When the first matching sub-result indicates that the expected result string contains a single numeric sequence number, and the second matching sub-result indicates that the expected result string does not contain a preset exact matching keyword, the format category is determined to be the single-condition fuzzy comparison format.

5. The method according to claim 1, characterized in that, The target verification mode includes a multi-condition fuzzy verification mode. The step of performing a differential verification operation corresponding to the target verification mode on the actual result string according to the target verification mode to obtain the verification result includes: When the target verification mode is the multi-condition fuzzy verification mode, a condition extraction operation is performed on the expected result string to obtain multiple strings to be verified. The actual result string is subjected to text normalization to obtain a normalized actual result string; Iterate through the multiple strings to be verified, and determine in turn whether each string to be verified exists in the normalized actual result string; The verification result is determined to be successful when each string to be verified exists in the normalized actual result string; or, the verification result is determined to be unsuccessful when any string to be verified does not exist in the normalized actual result string.

6. The method according to claim 5, characterized in that, The conditional extraction operation on the expected result string yields multiple strings to be verified, including: Based on preset multi-condition extraction rules, the expected result string is subjected to conditional segmentation to obtain multiple candidate text lines starting with a number and the corresponding number for each candidate text line. For each candidate text line, a keyword location operation is performed, and the first character position after the end position of the preset keyword in the candidate text line is determined as the content start position; Based on the starting position of the content, a content truncation operation is performed on each candidate text line to obtain multiple strings to be verified.

7. The method according to claim 1, characterized in that, The target verification pattern includes an exact match verification pattern. The step of performing a differential verification operation corresponding to the target verification pattern on the actual result string according to the target verification pattern to obtain the verification result includes: When the target verification mode is the exact match verification mode, the exact content extraction operation is performed on the expected result string to obtain the exact match string; The actual result string is subjected to text normalization to obtain a normalized actual result string; Determine whether the exact matching string is exactly equal to the normalized actual result string; The verification result is determined to be successful when the exact matching string is exactly equal to the normalized actual result string; or, the verification result is determined to be unsuccessful when the exact matching string is not exactly equal to the normalized actual result string.

8. The method according to claim 1, characterized in that, The target verification mode includes a single-condition fuzzy verification mode. The step of performing a differential verification operation corresponding to the target verification mode on the actual result string according to the target verification mode to obtain the verification result includes: When the target verification mode is the single-condition fuzzy verification mode, the expected result string is determined as the string to be verified; The actual result string is subjected to text normalization to obtain a normalized actual result string; Determine whether the string to be verified exists in the normalized actual result string; When the string to be verified exists in the normalized actual result string, the verification result is determined to be verified successfully; or, when the string to be verified does not exist in the normalized actual result string, the verification result is determined to be verified unsuccessfully.

9. The method according to any one of claims 6 to 8, characterized in that, The text normalization process performed on the actual result string to obtain the normalized actual result string includes: Perform a special character removal operation on the actual result string to obtain the first intermediate string; Perform a case-consistent conversion on the first intermediate string to obtain the second intermediate string; Perform a whitespace removal operation on the second intermediate string to obtain the normalized actual result string.

10. A device for verifying expected results, characterized in that, include: The result acquisition unit is used to obtain the expected result string of the test case; A format recognition unit is used to recognize the format of the expected result string and determine the format category of the expected result string; The pattern determination unit is used to determine the target verification pattern corresponding to the expected result string from a variety of preset verification patterns based on the format category. The verification execution unit is used to perform a differential verification operation on the actual result string according to the target verification pattern, and obtain the verification result.

Images