A software testing method, system, medium and program product

Abstract

The application relates to the technical field of CAD testing, and particularly relates to a software testing method, a system, a medium and a program product, which comprises the following steps: acquiring a test combination, the test combination comprising a test target and a standard target; selecting a use case combination according to the test combination, the use case combination comprising at least one test case, the test case being composed of at least one simulation operation instruction, and at least one test case corresponding to one test function; running the use case combination in the test combination, and correspondingly generating at least one group of first test data, the first test data comprising first data and second data obtained by respectively running the test case in the test target and the standard target; and filtering the first test data by adopting a filtering operation to generate an output scheme. For a large amount of test content, the application provides a data output scheme for filtering core differences, so that a user can quickly focus on test results.

Description

Technical Field

[0001] This invention relates to the field of CAD (Computer Aided Design) testing technology, specifically to a software testing method, system, medium, and program product. Background Technology

[0002] In the CAD development process, it is often necessary to conduct comparative analysis of CAD products from different vendors or different versions of CAD products, so as to promptly identify problems introduced by product upgrades and differences in product functions among competitors, and provide feedback for R&D.

[0003] For example, patent application 202410848198.2 discloses a white-box component testing method, device, and readable storage medium, which includes: acquiring test parameters and configuring test items according to the test parameters; in response to the running of the test items, compiling the source code files of the target component to generate a first coverage file and a dynamic library file; in response to the running of the test items, calling the test cases of the target component to generate a test program; sending the test program and the dynamic library file to the test device and receiving the test results fed back by the test device after running the test program; and determining the code coverage of the test cases according to the test results and the first coverage file.

[0004] For example, patent application 202411257581.7 discloses a white-box testing method for an SDK, comprising: creating a shell project and a target SDK, wherein the shell project contains the operating system's runtime environment, and the target SDK is the object to be tested; providing the user with a test case code writing template to obtain target test cases for white-box testing of the target SDK written by the user based on the test case code writing template; importing the target test cases into the shell project to obtain a test APP suitable for operating system operation; and running the test APP in the operating system to perform white-box testing of the target SDK.

[0005] However, the applicant noted that during the repeated iterations of CAD software, traditional white-box testing methods struggled to capture performance differences under varying environments. Summary of the Invention

[0006] The purpose of this invention is to provide a software testing method, comprising:

[0007] S300, Obtain a test set, the test set including: a test target and a standard target, the test target including: at least one version of the software under test, the standard target including: at least one version of the comparison software;

[0008] S302, Select a test case combination according to the test combination, the test case combination includes: at least one test case, the test case consists of at least one simulated operation instruction, and at least one test case corresponds to a test function;

[0009] S304, run the test case combination in the test combination to generate at least one set of first test data, the first test data including: first data and second data obtained by running the test cases on the test target and the standard target respectively;

[0010] S306, a filtering operation is performed on the first test data to generate an output scheme; wherein, S306 includes:

[0011] S3060, Calculate the difference between the first data and the second data;

[0012] S3062, when the difference is less than or equal to a preset first difference threshold, and the data content of the corresponding data belongs to a preset first type of record content, the corresponding first test data is identified as first type of data, and the other first test data is identified as second type of data.

[0013] S3064, Generate a data output scheme based on data attributes, wherein the data attributes include the following attributes: data category.

[0014] In some embodiments, step S3064 includes:

[0015] (1) When the difference between the two types of data is greater than a preset second difference threshold, the two types of data are marked as erroneous data;

[0016] (2) Generate a priority list for multiple sets of error data based on the data attributes;

[0017] (3) Sort the multiple erroneous data according to the attention priority to form a corresponding data output scheme.

[0018] In some embodiments, the data attributes further include the following attributes: the test level of the test case, and the error severity of the erroneous data; correspondingly, step (2) includes:

[0019] The attention priority is generated based on the test level and the error severity to produce the error data.

[0020] In some embodiments, the data attribute further includes the following attribute: association function; correspondingly, step (2) includes:

[0021] Obtain the test functions corresponding to error data with the highest priority.

[0022] Obtain the associated functions related to the test function, and identify at least one associated test case corresponding to the associated function;

[0023] Obtain the second priority of attention for the first test data corresponding to the associated test cases;

[0024] The second attention priority is verified or updated based on the first attention priority.

[0025] In some embodiments, before step (2), the following step is also included:

[0026] Obtain the number of the error data;

[0027] When the quantity is greater than or equal to the preset quantity threshold, the attribute within the first attribute range is selected as the data attribute in step (2);

[0028] When the quantity is less than the quantity threshold, the attribute within the second attribute range is selected as the data attribute in step (2); wherein the first attribute range is greater than the second attribute range.

[0029] In some embodiments, the data output scheme is further used to generate scoring results, and further includes:

[0030] Obtain the first rating result for the first version of the software in the first cycle;

[0031] The first data attribute is used to generate the attention priority of the error data, and the test function corresponding to the first attention priority is selected as the target function;

[0032] The target functionality was modified to obtain a second version of the software;

[0033] Obtain the second rating result of the second version of the software in the second cycle;

[0034] Calculate the score difference between the first score result and the second score result;

[0035] When the score difference is less than a preset score threshold, an attribute adjustment signal is generated, which is used to update the attribute type of the data attribute in step (2).

[0036] In some embodiments, the type of recorded content includes one or more of the following:

[0037] Grip point position, text position, annotation position, coordinate precision.

[0038] This invention provides a software testing system, comprising:

[0039] A test suite acquisition module is used to acquire test suites, wherein the test suites include: test objectives and standard objectives, and the test objectives include: at least one version of software;

[0040] The test case combination acquisition module is used to select a test case combination according to the test combination, wherein the test case combination includes: at least one test case, and the test case consists of at least one simulated operation instruction;

[0041] The testing module is used to run the test case combination in the test combination and generate at least one set of first test data, the first test data including: first data and second data obtained by running the test cases on the test target and the standard target respectively;

[0042] The output module is used to filter the first test data using a filtering operation to generate an output scheme; wherein, the output module is further used to perform the following steps:

[0043] Calculate the difference between the first data and the second data;

[0044] When the difference is less than or equal to a preset first difference threshold, and the corresponding test case belongs to a preset type of test case, the first test data is identified as type one data, and the other first test data is identified as type two data.

[0045] A data output scheme is generated based on data attributes, wherein the data attributes include: data category.

[0046] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the software testing method as described in any embodiment.

[0047] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the software testing method steps as described in any of the embodiments.

[0048] Beneficial technical effects:

[0049] It is important to note that the software testing method provided in this invention primarily targets the software development phase. In actual software development, software engineers may update the code daily. Furthermore, the updated code needs to be run and verified promptly. Typically, testing requires comprehensive testing of the software's main or all functionalities, resulting in a massive amount of test data.

[0050] In response to the massive amount of test content and results, this invention provides a data output scheme that filters core differences, enabling users to quickly focus on the test results.

[0051] This invention addresses this high-frequency testing scenario by proposing an automated testing and output scheme that prioritizes data based on its importance. This automated scheme comprehensively sets priorities based on different dimensions such as function category, data category, and functional relevance, thereby guiding users to quickly focus on errors with a relatively large impact.

[0052] Furthermore, the present invention can also select the scope of consideration (i.e., the attribute range) of priority generation factors based on the overall test situation (such as the number of erroneous data). This can reduce the difficulty of priority generation by limiting the scope of consideration (avoiding the priority sorting process from affecting the output efficiency of test data, or the overly complex sorting scheme from becoming blurred or affecting the prominence of important issues), and can also highlight key data in a timely manner so that users can quickly focus on key issues. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0054] Figure 1 This is a flowchart illustrating a software testing method in an exemplary embodiment of the present invention;

[0055] Figure 2 This is a flowchart illustrating the process of determining a data output scheme in an exemplary embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of the application process of test cases in an exemplary embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram of the application process of test cases in another exemplary embodiment of the present invention;

[0058] Figure 5 This is a partial schematic diagram of the test case selection interface in an exemplary embodiment of the present invention, showing the selected area.

[0059] Figure 6This is a schematic diagram of a computer device structure in an exemplary embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0061] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0062] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0063] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0064] In this document, "and / or" includes any and all combinations of one or more of the listed related items.

[0065] In this article, "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0066] As used in this specification, the term "about" typically means + / -5% of the value, more typically + / -4% of the value, more typically + / -3% of the value, more typically + / -2% of the value, even more typically + / -1% of the value, and even more typically + / -0.5% of the value.

[0067] In this specification, certain embodiments may be disclosed in a range-bound format. It should be understood that this "range-bound" description is merely for convenience and brevity and should not be construed as a rigid limitation on the disclosed range. Therefore, the description of a range should be considered as having specifically disclosed all possible subranges and the individual numerical values ​​within those ranges. For example, a description of the range 1-6 should be considered as having specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and the individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This rule applies regardless of the breadth of the range.

[0068] Definition of noun:

[0069] “Entity”: In this document, an entity refers to graphic data on a drawing, such as a CAD file, displayed on the CAD drawing interface. An entity has attributes, which are data values ​​that control specific visual characteristics of the entity or element (such as visibility, color, and line style). In different implementations, an entity may also be referred to as a “graphic element” or “graphic primitive.”

[0070] "Object": In this article, an object refers to information in a CAD file that is not displayed on the CAD drawing interface (the specific values ​​of this information can also be called attributes), such as layers, text styles, and annotation styles. In this article, "style" refers to a set of named attributes used to classify and define specific geometric and text elements (such as line styles or text styles).

[0071] “Element”: In this article, an element refers to all possible information in a CAD file, including “entities” and “objects,” and can also be block, group, and cell definitions based on “entities” and / or “objects.” Correspondingly, the attributes of these elements are also referred to as simulation data (or drawing data).

[0072] "Drawings": In this article, "drawings" can refer to various files that can run in a CAD system and are used for drawing, editing, modifying, storing, and viewing CAD drawings. Common CAD drawing formats include, but are not limited to, DWG, DXF, DWT, DWF, DWL, DWS, DWX, MNU, MNC, MNL, MNS, CUI, CUIX, SHX, PAT, LIN, CTB, STB, PLT, PC3, etc.

[0073] "Software": In this article, it refers to a collection of computer data and instructions organized in a specific order. Software can include both computer programs that can run on a computer and the documentation associated with those programs. In other words, software can be a combination of programs and documentation.

[0074] "Test Environment": In this article, the software test environment refers to the combination of hardware and software specifically configured to verify the functionality, performance, and stability of the software. This includes one or more necessary components such as the operating system, database, network configuration, and testing tools (e.g., test cases). Its core objective is to simulate the performance under real-world operating scenarios (e.g., drawing, displaying diagrams).

[0075] "Operation point": In this document, "operation point" refers to the position of a user's operation on a digital interface (such as a display interface) using an input device (such as a mouse, touchpad, touchscreen, etc.). It is the direct point of interaction between the user and the system or software. It can be clearly identified by visual symbols such as a cursor or text insertion point, allowing the user to intuitively perceive the object being operated on (such as an entity or object) and complete the transmission of instructions through actions such as clicking, inputting, and dragging. For example, in some embodiments, when a user needs to draw a line segment on the current interface, they can select a first operation point and a second operation point on the display interface as the two endpoints of the line segment.

[0076] Testing a feature (or simply a function): A feature is an independent unit that can complete a specific task or provide a specific service. For example, a feature can be an operation command or an interactive module. Typically, a feature is the core carrier of software interaction with the user, and usually has clearly defined input, processing, and output logic.

[0077] Example 1:

[0078] See Figures 1-2 As shown, the present invention provides a software testing method.

[0079] like Figure 1 As shown, software testing methods include:

[0080] S300, Obtain a test set, the test set including: a test target and a standard target, the test target including: at least one version of the software under test, the standard target including: at least one version of the comparison software;

[0081] For example, in some embodiments, the test target can be a software to be tested selected by the user, and the standard target can be a software selected by the user from the existing market as a benchmark for comparison.

[0082] S302, Select a test case combination according to the test combination, the test case combination includes: at least one test case, the test case consists of at least one simulated operation instruction, and at least one test case corresponds to a test function;

[0083] In some embodiments, the test suite includes at least one test function.

[0084] For example, in some embodiments, users can select one or more test functions to form a test combination according to test requirements, and select the corresponding test case combination from the test case database according to the test combination.

[0085] For example, in some embodiments, the test case database pre-stores multiple test cases, and each test case is tagged with a label that records test function information (such as test function name) so as to automatically select and call test cases according to the test combination.

[0086] S304, run the test case combination in the test combination to generate at least one set of first test data, the first test data including: first data and second data obtained by running the test cases on the test target and the standard target respectively.

[0087] Furthermore, in some embodiments, the method further includes the step of:

[0088] S306, a filtering operation is performed on the first test data to generate an output scheme; wherein, S306 includes:

[0089] S3060, Calculate the difference between the first data and the second data;

[0090] S3062, when the difference is less than or equal to a preset first difference threshold, and the data content of the corresponding data belongs to a preset first type of record content, the corresponding first test data is identified as first type of data, and the other first test data is identified as second type of data.

[0091] S3064, Generate a data output scheme based on data attributes, wherein the data attributes include the following attributes: data category.

[0092] In some embodiments, data attributes can be obtained from a preset attribute range. This attribute range may include at least one attribute category (or attribute type) pre-defined by the user, such as data category, test function, etc.

[0093] In some embodiments, the attribute range can be selected based on the testing scenario. For example, users can select the attribute range based on the testing requirements or key considerations in the current testing scenario.

[0094] For example, in some embodiments, the output priority of type II data is higher than that of type I data.

[0095] For example, in some embodiments, the data output scheme may be: outputting the first test data in a document format, and the second type of data output is ordered before the first type of data.

[0096] For example, in some embodiments, the test data includes test data for multiple test cases, and the test data for one test case may include multiple records (each record may be marked with a different output priority). Specifically, for the multiple records corresponding to one test case, the records marked as Class II data may be placed before the records marked as Class I data.

[0097] For example, in some embodiments, data tags can be generated for the output test data (or first data, second data) respectively. These data tags are used to record the function category (such as function name, etc.) and the data category (such as belonging to a type I data or a type II data).

[0098] For example, in some embodiments, the output order of test data can also be sorted according to one or more data attributes, such as data category.

[0099] For example, the output priority of Category II data is higher than that of Category I data. For example, different functional categories are pre-set with varying degrees of importance; the greater the importance, the higher the output priority.

[0100] For example, output priority (also known as focus priority) can be determined based on a combination of data category and function category, and the weighting of data category and function category in determining output priority can be set by the software test engineer according to their needs.

[0101] In some embodiments, the type of recorded content includes one or more of the following:

[0102] Grip point position, text position, annotation position, coordinate precision.

[0103] In this embodiment, by filtering a type of recorded content, the interference of subtle differences in software design on the overall performance evaluation can be reduced to a certain extent.

[0104] For example, if two software programs have slight differences in the precision of grip operation (i.e., the accuracy of grip coordinates), it may be due to differences in the design versions of the software itself. Therefore, when outputting the solution, users can be guided not to focus on this.

[0105] Alternatively, in some embodiments, when the difference is greater than a first difference threshold, or when the data content belongs to Class II record content, the corresponding test data (such as the first data and / or the second data) is identified as Class II data. Conversely, the remaining test data can also be identified as Class I data.

[0106] In other words, for a massive amount of test content and test results, this embodiment provides a data output scheme that filters core differences, so that users can quickly focus on the test results.

[0107] In some embodiments, the second type of record content can refer to multiple types of data content other than the first type of record content, such as one or more of the following:

[0108] (1) Core performance indicators: such as execution time (such as startup time, file loading time, save time, rendering time, etc.) and throughput (such as the number of operations that can be completed per unit time, for example, the number of features that can be processed per second or the number of buffers that can be generated per second).

[0109] (2) Accuracy indicators: geometric operation accuracy (e.g., buffer analysis accuracy, overlay analysis accuracy, etc.), topological accuracy (e.g., the correctness of the software in maintaining and processing topological relationships (such as adjacency, connectivity, containment. For example, check whether it can correctly identify and repair topological errors such as dangling lines, overlapping surfaces, gaps, etc.), and mapping output accuracy.

[0110] (3) Resource consumption indicators: memory usage (for example, it can be free memory usage: the amount of memory used when the software starts without performing any operations; peak memory usage: the maximum amount of memory used during the execution of large and complex tasks. Memory leaks are usually manifested as peak memory continuously increasing and not being released; working set memory: the amount of memory required for the software to run normally); CPU utilization; GPU utilization; network bandwidth usage.

[0111] (4) Stability metrics: crash / error rate, big data processing capability (which is used to evaluate whether the software can open, render, and operate datasets that are far larger than normal (such as GB-level raster data or vector data containing millions of features) without crashing or experiencing a sharp drop in performance).

[0112] For example, in some embodiments, data that does not belong to the first type of recorded content (such as first data or second data) is identified as second type of data.

[0113] In some embodiments, the testing function may be a text function (such as testing the accuracy of text display), a drawing performance function (such as testing the accuracy of drawing data, drawing efficiency, rendering efficiency, etc.), an ecosystem function (such as stability, compatibility, etc.), etc.

[0114] In some embodiments, a test function may correspond to one or more recorded contents.

[0115] In some embodiments, simulated operation instructions can be pre-written operation codes by the user to perform at least one functional task. For example, simulated operation instructions can instruct CAD software to perform specific graphic or data operations, such as drawing elements, modifying objects, managing views, or controlling file attributes.

[0116] For example, in some embodiments, a simulated operation instruction is composed of a combination of multiple operation functions.

[0117] Taking the circle drawing operation command as an example, a circle drawing operation command includes: operation function 1, which adds a circle drawing data to the display interface; operation function 2, which stores the circle drawing data (such as coordinate position, diameter, etc.) into the drawing database; and operation function 3, which refreshes the display interface based on the drawing data to display the newly added circle. Typically, the multiple operation functions have a set execution order to simulate the usual operation sequence of a drawing engineer.

[0118] In some embodiments, step S3064 includes:

[0119] (1) When the difference is greater than a preset second difference threshold, the first test data is marked as erroneous data;

[0120] (2) Prioritize the collection of multiple erroneous data based on data attributes;

[0121] For example, in some embodiments, the data attribute is the data category (such as category one or category two) corresponding to the test data. The attention priority is preferably used to define the order of data output; for example, the higher the attention priority, the earlier the output data is ranked.

[0122] (3) Sort the multiple erroneous data according to the attention priority to form a corresponding data output scheme.

[0123] In this embodiment, the priority of attention for the two types of data is preferably ranked according to the identification results of the data categories. The priority of attention for the first type of data can be directly adopted using the default output priority.

[0124] In some embodiments, the data attributes further include the following attributes: the test level of the test case, and the error severity of the erroneous data; correspondingly, step (2) includes:

[0125] The attention priority is generated based on the test level and the error severity to produce the error data.

[0126] For example, test levels can have pre-defined priorities. Software test engineers can set test levels for test cases based on different testing needs.

[0127] In some embodiments, the data attribute further includes the following attribute: association function; correspondingly, step (2) includes:

[0128] 1. Obtain the test functions corresponding to error data with the highest priority.

[0129] 2. Obtain the associated functions related to the test function, and identify at least one associated test case corresponding to the associated function;

[0130] 3. Obtain the second priority of attention for the first test data corresponding to the associated test cases;

[0131] For example, in some embodiments, the second concern priority can be a pre-set priority for the associated test cases.

[0132] For example, in some embodiments, the second concern priority can be a priority generated based on the data attributes of the first test data of the associated test case.

[0133] 4. Verify or update the second priority of attention based on the first priority of attention.

[0134] For example, in some embodiments, when the difference between the first priority and the second priority is greater than a set first priority threshold, it is recommended to update the second priority based on the first priority in order to reduce the difference between the first priority and the second priority.

[0135] For example, in some embodiments, when the first priority of concern is greater than the second priority of concern, it is recommended to increase the second priority of concern, such as updating the second priority of concern to the same priority as the first priority of concern.

[0136] For example, in some embodiments, the current priority can be maintained when the first priority and the second priority are the same.

[0137] For example, in some embodiments, when both the first attention priority and the second attention priority are greater than a preset second priority threshold, it is recommended to increase the first attention priority and the second attention priority simultaneously.

[0138] For example, in some embodiments, functional relationships can be pre-defined between different testing functions. For instance, when a user wants to test the text annotation function, the size recognition function may be used in conjunction with it.

[0139] For example, in some embodiments, the test function is pre-set with an association label, which records information (such as the name) of the test function associated with the test function.

[0140] In this embodiment, based on error priority and functional relevance, errors that are important and occur in batches are preferably output first, so as to guide users to prioritize the handling of some errors with a large impact.

[0141] It is important to note that the software testing method provided in this invention primarily targets the software development phase. In actual software development, software engineers may update the code daily. Furthermore, the updated code needs to be run and verified promptly. Typically, testing requires comprehensive testing of the software's main or all functionalities, resulting in a massive amount of test data.

[0142] To address this issue, this invention proposes an automated testing and output scheme for such high-frequency testing scenarios, capable of sorting and outputting data based on priority. This automated scheme comprehensively sets priority based on different dimensions such as function category, data category, and function relevance, thereby guiding users to quickly focus on errors with a relatively large impact.

[0143] In other words, in actual software development, especially during software function upgrades or adjustments, repeated adjustments and verifications are often required to determine the final complete solution. During a single test, the amount of test data output is often very large, and the types of errors that may occur can also be numerous. To address this, this invention categorizes and sorts the error data, allowing users to quickly focus on errors that may have a relatively large impact, thereby improving the efficiency of software optimization / improvement to a certain extent.

[0144] In some embodiments, before step (2), the following step is also included:

[0145] Obtain the number of the error data;

[0146] When the quantity is greater than or equal to the preset quantity threshold, the attribute within the first attribute range is selected as the data attribute in step (2);

[0147] When the quantity is less than the quantity threshold, the attribute within the second attribute range is selected as the data attribute in step (2); wherein the first attribute range is greater than the second attribute range.

[0148] Furthermore, this embodiment also provides a scheme for sorting erroneous data based on a range of restrictive attributes.

[0149] Specifically, in this embodiment, error data is sorted by relatively priority attribute factors in order to reduce the difficulty of priority generation while highlighting the errors in the test documents.

[0150] From another perspective, this embodiment can select the scope of consideration (i.e., the attribute range) of priority generation factors based on the overall test situation (such as the number of error data). This can reduce the difficulty of priority generation by limiting the scope of consideration (avoiding the priority sorting process from affecting the output efficiency of test data, or the overly complex sorting scheme from becoming blurred or affecting the prominence of important issues). On the other hand, it can also highlight key data in a timely manner so that users can quickly focus on key issues.

[0151] Furthermore, prioritization can also differentiate the display schemes for different test data.

[0152] For example, in some embodiments, the test results can pinpoint the code location of the test case that outputs the result. If this test result is not needed in subsequent tests, the level of this output result can be modified in the test case code. When comparing results, the result level recorded at the time of output is used compared with the comparison level configured in the current comparison tool to determine how the results are displayed. Thus, only the important results are seen on the comparison display interface.

[0153] In some embodiments, the data output scheme is further used to generate scoring results, and further includes:

[0154] Obtain the first rating result for the first version of the software in the first cycle;

[0155] The first data attribute is used to generate the attention priority of the error data, and the test function corresponding to the first attention priority is selected as the target function;

[0156] The target functionality was modified to obtain a second version of the software;

[0157] Obtain the second rating result of the second version of the software in the second cycle;

[0158] Calculate the score difference between the first score result and the second score result;

[0159] When the score difference is less than a preset score threshold, an attribute adjustment signal is generated. The attribute adjustment signal is used to update the attribute type of the data attribute in step (2), or to suggest that the user update the attribute type.

[0160] In some embodiments, the specific update scheme can be set or selected by the user.

[0161] In other words, in this embodiment, the attribute type (or attribute range) can be adjusted by the software optimization to improve the adaptability between the priority generation scheme and the current test requirements.

[0162] For example, in some embodiments, the scoring result refers to the score given to at least one test function using preset scoring rules. Furthermore, the scoring benchmark can be determined based on the test data from the testing software.

[0163] For example, in some embodiments, different scoring tables can be set for different test functions. For example, taking resource utilization efficiency as an example, different scores can be given depending on the degree of utilization efficiency.

[0164] For example, in some embodiments, taking the test target as version A professional CAD (the first version of the software refers to version A professional CAD V1.0-0715, and the second version of the software refers to version A professional CAD V1.0-0716), and the standard target as version B professional CAD, the process of adjusting the attribute range according to the software optimization cycle / efficiency is illustrated by way of example:

[0165] The professional version A, CADFAV1.0-0715 (where 0715 represents the software's generation time or batch), was tested to obtain the first set of test data. Based on this first test result, a first score was given. The first and second scores represent the overall score after the software test. That is, the score is a composite value of the scores from multiple test functions (e.g., the weighted sum of multiple scores).

[0166] Error data in the first test data is sorted and output according to the priority of attention. Software engineers first focus on one or more test data with the highest priority (the corresponding test function is called the target function), and update the code of Professional CADV1.0-0715 in version A according to the test data to obtain Professional CADV1.0-0716 in version A.

[0167] Furthermore, the A version of professional CADF1.0-0716 was tested to obtain second test data, and a second score result was generated based on the second test data.

[0168] Calculate the difference between the first and second rating results (e.g., the difference between the two). If the difference is less than the rating threshold, it is recommended to update the attribute type (or attribute range) of the data attribute.

[0169] Of course, the specific update plan can be manually adjusted by the user.

[0170] In some embodiments, software versions A and B can be the software under test and a competitor's software, respectively. Alternatively, software versions A and B can be different software released by the testing vendor itself.

[0171] It is understood that the sorting results of erroneous data can be tracked in this embodiment. That is, when the user selects the test function corresponding to the higher priority according to the current priority sorting result for local optimization, the priority generation scheme can be adjusted according to its optimization efficiency to enhance the adaptability of the priority generation scheme to the test scenario.

[0172] Furthermore, in some embodiments, different data structures can be used to output the test documents.

[0173] For example, in some embodiments, the output test document is log content organized in XML markup language, which is encrypted and stored as a binary byte file.

[0174] For example, in some embodiments, the method further includes storing test documents; for instance, in some embodiments, test results marked / identified as important results may be recorded and stored.

[0175] In some embodiments, log files (i.e. test results) output by test cases can be loaded, parsed, and displayed.

[0176] Furthermore, in some embodiments, multiple log files can be loaded at once and compared to identify differences.

[0177] In some embodiments, test cases may include different categories. Furthermore, test cases may be tagged with test case labels, and these labels may indicate the test case category, such as perspective information, execution information (e.g., manual or automated execution), phase information (e.g., different phases such as unit, integration, system, acceptance, etc.), or type information (e.g., different test types such as functional, performance, security, compatibility, etc.).

[0178] For example, from a testing perspective, test cases can include one or more of the following:

[0179] Black-box test cases, also known as functional or behavioral testing, involve treating the system under test as a completely opaque "black box" by the tester (such as a software engineer). The tester does not need to understand the system's internal code structure, implementation logic, or operating state; they only need to be concerned with the system's inputs and outputs.

[0180] White-box test cases, also known as structural testing, transparent-box testing, or glass-box testing, allow testers to fully view the system's internal structure, such as source code, architecture, and logical flow. Test cases are designed based on this internal knowledge to verify whether the internal operations are performed as expected.

[0181] Gray-box testing is a hybrid of black-box and white-box testing. Testers have limited knowledge of the system's internal structure (e.g., database schema, API definitions, algorithm selection, system architecture diagrams, etc.). However, they don't delve into every line of code as deeply as white-box testing.

[0182] For example, in some embodiments, test cases may also include: manual test cases, and / or automated test cases.

[0183] As an exemplary embodiment, the present invention provides a white-box automated test development method with self-verification, that is, users can perform more intuitive data analysis on the testing process through test results.

[0184] Example 2

[0185] The present invention also provides a software testing system, comprising:

[0186] A test suite acquisition module is used to acquire test suites, wherein the test suites include: test objectives and standard objectives, and the test objectives include: at least one version of software;

[0187] The test case combination acquisition module is used to select a test case combination according to the test combination, wherein the test case combination includes: at least one test case, and the test case consists of at least one simulated operation instruction;

[0188] The testing module is used to run the test case combination in the test combination and generate at least one set of first test data, the first test data including: first data and second data obtained by running the test cases on the test target and the standard target respectively;

[0189] The output module is used to filter the first test data using a filtering operation to generate an output scheme; wherein, the output module is further used to perform the following steps:

[0190] Calculate the difference between the first data and the second data;

[0191] When the difference is less than or equal to a preset first difference threshold, and the corresponding test case belongs to a preset type of test case, the first test data is identified as type one data, and the other first test data is identified as type two data.

[0192] A data output scheme is generated based on data attributes, wherein the data attributes include: data category.

[0193] It is understood that the system in this embodiment can be used to implement the methods and / or steps in any of the above embodiments, and will not be described again here.

[0194] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the software testing method as described in any of the embodiments.

[0195] Furthermore, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the software testing method steps as described in any of the embodiments.

[0196] Example 3:

[0197] See Figures 3-4 As shown, the present invention provides a method for running test cases, including:

[0198] S400 provides a test case library, which includes multiple pre-stored test cases.

[0199] Preferably, the test case library includes: at least one language node, one language node corresponds to multiple function nodes, and one function node records at least one test case;

[0200] For example, see Figure 5 As shown, this illustrates an exemplary test case library selection interface, which provides test cases for different languages ​​(such as C++, .NET, and Lisp). The test case library also provides a tree-like node selection interface.

[0201] For example, the node selection interface includes: language nodes (such as C++ nodes), and each language node expands into multiple function nodes (such as text nodes, mtext nodes). Furthermore, after selecting one of the function nodes, multiple test case nodes can be expanded, and users can manually select the required test cases through node selection.

[0202] S402, Obtain the test requirements input by the test user, wherein the test requirements include: at least one operation function to be tested;

[0203] S404, Obtain at least one associated functional node related to the operation function;

[0204] S406, Identify at least one test case under the associated functional node;

[0205] S408, Generate a combination of test cases based on at least one of the test cases;

[0206] S410, Run the test case combination in the test combination, the test combination including: test objective and standard objective, the test objective including: at least one version of software.

[0207] In some embodiments, S410 includes the step of:

[0208] (1) Run a first test case on the test target, wherein the first test case includes a first test element;

[0209] For example, in some embodiments, a test element may refer to the object or entity to be tested.

[0210] (2) Identify the first test parameter of the first test element, wherein the first test parameter includes one or more of the following: size, coordinates, and state;

[0211] (3) Before running the second test case on the test target, identify whether there is a second test element in the second test case that corresponds to the first test element (where the first test element and the second test element are also referred to as related elements), and the second test element has initial second test parameters; if so, perform the following steps:

[0212] (4) Update the current test environment according to the first test parameter and the second test parameter.

[0213] For example, in some embodiments, the first test element is a block A. If the block A needs to be edited during the second test case, it is assumed that there is a second test element in the second test case that corresponds to the first test element.

[0214] In some embodiments, the step of updating the current test environment based on the first test parameter and the second test parameter includes:

[0215] When the first test parameter and the second test parameter are different, the first test parameter of the first test element is modified to the second test parameter.

[0216] For example, in some embodiments, the first test element is an entity, such as a rectangle, whose corresponding first test parameters include: a first size (e.g., the length and width of the rectangle each have a first value). Correspondingly, in the initial stage of the second test case execution, the second test parameters include a second size (e.g., the length and width of the rectangle each have a second value). Correspondingly, before executing the second test case, the first test parameters are modified to the second test parameters, that is, the length and width of the rectangle are modified from the first values ​​to the second values.

[0217] For example, in some embodiments, the test parameter can also be a state, such as an entity having a frozen state or an editable state.

[0218] In this embodiment, by updating the test parameters of the test elements locally (i.e. focusing on related elements), the potential conflicts that may occur during the automated execution of batch test cases can be reduced, and the data pressure of automated verification can also be reduced.

[0219] In some embodiments, the second test element includes: the same test element as the first test element, and / or, a test element associated with the first test element. Therefore, the second test element and the first test element are referred to as associated elements.

[0220] For example, in some embodiments, the associated test element refers to a test element that is nested, referenced, or connected to the first test element.

[0221] For example, in some embodiments, if the first test element is a water pipe installed on the wall, then the second test element can be the wall itself.

[0222] In some embodiments, S410 includes the step of:

[0223] Run the first test case on the test target;

[0224] Before running the second test case on the test target, determine whether to restore the current test environment.

[0225] If so, the current test environment will be restored, and then the second test case will be executed.

[0226] In some embodiments, the step of determining whether to restore the current test environment includes:

[0227] The process of identifying whether the execution of the first test case can initiate a simulated switching operation is determined. The simulated switching operation includes one or more of the following: resolution switching, viewpoint switching (such as switching between different viewpoints such as spatial viewpoint and planar viewpoint), and zoom ratio switching.

[0228] If so, it is recommended to restore the test environment.

[0229] For example, in some embodiments, scene restoration can refer to undoing a simulated switching operation. For instance, adjusting the scaling ratio from the one modified by the first test case to the initial scaling ratio before the first test case started.

[0230] For example, in some embodiments, scene restoration can be performed by updating the coordinates of the operation point in the scene (such as a display interface).

[0231] In some embodiments, the test cases include: a first type of test case and a second type of test case; wherein, the first type of test case is stored in a first resource area, and the second type of test case is stored in a second resource area; the second type of test case is stored in the second resource area, and the index of the second type of test case is stored in a dictionary; S410 includes the step:

[0232] Multiple Class I test cases are called from the first resource area, and these multiple Class I test cases are copied into the main test program during the linking phase.

[0233] The main test program is started within the test objective; that is, one type of test case is executed one by one.

[0234] When it is necessary to identify the second type of use case, the second type of use case is found through the corresponding index path (for example, it refers to the storage address of the use case), and the second type of use case is activated to generate a dynamic call interface for at least one first type of use case;

[0235] The main test program runs the class of test cases through the dynamic call interface.

[0236] In this embodiment, the second type of use case is associated with at least one first type of use case.

[0237] For example, in some embodiments, the first resource area and the second resource area can be different resource areas.

[0238] For example, in some embodiments, the first resource area may be stored in the memory area, and the second resource area may be stored in the external memory area.

[0239] In some embodiments, the indexer is the filename of the second type of use case, which can be stored in a dictionary. In CAD, a dictionary is a core database container object used to store and manage a set of key-value pairs.

[0240] In some embodiments, the frequency of invocation of type I test cases is greater than the frequency of invocation of type II test cases. In this embodiment, type I test cases refer to test cases that are frequently used during the testing process, while type II test cases are test cases that are less frequently invoked.

[0241] For example, in some embodiments, a Class I use case refers to a test case used to run a certain software function. A Class II use case refers to a test case used to initiate at least one function. Specifically, a Class II use case can be associated with multiple Class I use cases, that is, the Class II use case generates calling interfaces for multiple Class I use cases.

[0242] Furthermore, type II test cases are often those that are less likely to be enabled (or have a relatively short activation time), or in some cases, type II test cases may be test cases that are ignored or cannot be paid attention to during test execution. For example, these test cases can be one or more of the following types: mtext.grx, kernel.grx.

[0243] Example 5:

[0244] This invention also provides a software testing method, which includes the following steps:

[0245] Obtain a test suite, the test suite including: a test target and a standard target, the test target including: at least one version of the software under test, the standard target including: at least one version of the comparison software;

[0246] Select a test case combination based on the test combination, wherein the test case combination includes: at least one test case, the test case consists of at least one simulated operation instruction, and the at least one test case corresponds to a test function;

[0247] Running the test case combination in the test combination generates at least one set of first test data, which includes: first data and second data obtained by running the test cases on the test target and the standard target, respectively.

[0248] Running the test case combination in the test combination includes the following steps:

[0249] A first test case is run on the test target, the first test case including a first test element;

[0250] Identify the first test parameter of the first test element, wherein the first test parameter includes one or more of the following: size, coordinates, and state;

[0251] Before running the second test case on the test target, identify whether there is a second test element in the second test case that corresponds to the first test element, and whether the second test element has initial second test parameters; if so, perform the following steps:

[0252] The current test environment is updated based on the first test parameter and the second test parameter.

[0253] It is understood that the method in this embodiment may employ the same or similar steps as any of the above embodiments, and will not be repeated here.

[0254] Example 6:

[0255] In another aspect, the present invention provides a method for running test cases, comprising:

[0256] It provides a test case library, which stores multiple test cases;

[0257] Obtain the test requirements input by the test user, wherein the test requirements include: at least one operation function to be tested;

[0258] Obtain at least one associated functional node related to the operation function;

[0259] Identify at least one test case under the associated functional node;

[0260] Generate a combination of test cases based on at least one of the test cases described;

[0261] The test case combination is run in a test combination that includes a test objective and a standard objective, wherein the test objective includes at least one version of the software.

[0262] Record the first test data, which includes the first data generated after the test target runs the combination of test cases, and the second data generated after the standard target runs the combination of test cases.

[0263] Furthermore, in some embodiments, the method further includes:

[0264] The first test data is filtered using a filtering operation to generate an output scheme; this includes:

[0265] Calculate the difference between the first data and the second data;

[0266] When the difference is less than or equal to a preset first difference threshold, and the data content of the corresponding data belongs to a preset first type of record content, the corresponding first test data is identified as first type of data, and the other first test data is identified as second type of data.

[0267] A data output scheme is generated based on data attributes, wherein the data attributes include the following: data category.

[0268] In some embodiments, after selecting a combination of test cases, the test can be run synchronously in both the test objective and the standard objective to record the test data (i.e., first data and second data).

[0269] In some embodiments, the second data may also be historical data recorded since the last run.

[0270] It is understood that the method in this embodiment may employ the same or similar steps as any of the above embodiments, and will not be repeated here.

[0271] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the embodiments.

[0272] The present invention also provides a computer program product containing instructions that, when the computer program product is run on an electronic device, enable the electronic device to perform the method described in any of the embodiments.

[0273] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer program can run as follows: Figure 6 It runs on the computer device shown. Figure 6 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0274] The non-volatile storage medium may store an operating system and a computer program. The computer program includes program instructions that, when executed, cause a processor to perform the methods described in any of the embodiments.

[0275] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0276] Internal memory provides an environment for the execution of computer programs in non-volatile storage media, which, when executed by a processor, enable the processor to perform the methods in any of the embodiments.

[0277] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0278] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0279] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0280] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A software testing method characterized by, Comprising: S300, obtaining a test combination, the test combination comprising: a test target and a standard target, the test target comprising: at least one version of software to be tested, the standard target comprising: at least one version of comparative software; S302, selecting a use case combination according to the test combination, the use case combination comprising: at least one test case, the test case consisting of at least one simulation operation instruction, at least one test case corresponding to one test function; S304, running the use case combination in the test combination, corresponding to generate at least one set of first test data, the first test data comprising: first data and second data obtained by running the test case in the test target and the standard target respectively; S306, filtering the first test data by using filtering operation to generate an output scheme; wherein, S306 comprises: S3060, calculating the difference between the first data and the second data; S3062, when the difference is less than or equal to a preset first difference threshold, and the data content of the corresponding data belongs to a preset type of record content, identifying the corresponding first test data as a type of data, and identifying other first test data as a type of data; wherein, the output priority of the type of data is higher than that of the type of data; S3064, generating a data output scheme according to data attributes, wherein the data attributes include the following attributes: data category; wherein, step S3064 comprises: (1) when the difference of the type of data is greater than a preset second difference threshold, the type of data is marked as error data; (2) generating a plurality of attention priorities of the error data according to the data attributes; (3) sorting a plurality of error data according to the attention priority to form a corresponding data output scheme; wherein, the data attributes further include the following attributes: test level of the test case and error degree of the error data; correspondingly, step (2) comprises: generating the attention priority of the error data according to the test level and the error degree; Wherein, before step (2), it further comprises the following steps: obtaining the number of error data; when the number is greater than or equal to a preset number threshold, the attributes in a first attribute range are selected as the data attributes in step (2); when the number is less than the number threshold, the attributes in a second attribute range are selected as the data attributes in step (2); wherein, the first attribute range is greater than the second attribute range.

2. The method of claim 1, wherein, The data attributes further include the following attributes: associated function; correspondingly, step (2) comprises: obtaining the test function corresponding to the error data with the first attention priority; obtaining the associated function associated with the test function, and identifying at least one associated test case corresponding to the associated function; obtaining the second attention priority of the first test data corresponding to the associated test case; verifying or updating the second attention priority according to the first attention priority.

3. The method of claim 1, wherein, The data output scheme is also used to generate a scoring result, and further comprises: obtaining a first score result of a first version of software in a first period; generating a first attention priority of the error data according to the first data attribute, and selecting a test function corresponding to the first attention priority as a target function; modifying the target function to obtain a second version of software; obtaining a second score result of the second version of software in a second period; calculating a score difference between the first score result and the second score result; when the score difference is less than a preset score threshold, generating an attribute adjustment signal, the attribute adjustment signal being used to update an attribute type of the data attribute in step (2).

4. The method according to any one of claims 1 to 3, characterized in that, The one type of record content includes one or more of the following: pinch point position, text position, label position, coordinate accuracy.

5. A software testing system, characterized by It includes: a test combination acquisition module, configured to acquire a test combination, the test combination including: a test target and a standard target, the test target including: at least one version of software; a use case combination acquisition module, configured to select a use case combination according to the test combination, the use case combination including: at least one test case, the test case being composed of at least one simulation operation instruction; a test module, configured to run the use case combination in the test combination, and correspondingly generate at least one set of first test data, the first test data including: first data and second data obtained by respectively running the test case in the test target and the standard target; an output module, configured to filter the first test data by using a filtering operation to generate an output scheme; wherein the output module is further configured to perform the following steps: calculating a difference between the first data and the second data; when the difference is less than or equal to a preset first difference threshold, and the test case belongs to a preset type of test case, identifying the first test data as one type of data, and correspondingly identifying other first test data as two types of data; wherein the output priority of the two types of data is higher than that of the one type of data; generating a data output scheme according to a data attribute, wherein the data attribute includes: data category; wherein generating a data output scheme according to a data attribute includes: (1) when the difference of the two types of data is greater than a preset second difference threshold, marking the two types of data as error data; (2) generating attention priorities of a plurality of error data according to the data attribute; (3) sorting a plurality of error data according to the attention priorities to form a corresponding data output scheme; wherein the data attribute further includes the following attributes: test level of the test case and error level of the error data; correspondingly, step (2) includes: generating the attention priorities of the error data according to the test level and the error level; wherein, before step (2), it further includes the following steps: obtaining the number of error data; when the number is greater than or equal to a preset number threshold, selecting an attribute of a first attribute range as the data attribute in step (2). When the number is less than the number threshold, then an attribute in a second attribute range is selected as the data attribute in step (2); wherein the first attribute range is greater than the second attribute range.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the software testing method according to any one of claims 1 to 4.

7. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the steps of the software testing method according to any one of claims 1 to 4.

Images