Test method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a test method, the method is applied to network equipment, and the method comprises the following steps: based on first system basic information of a to-be-tested version of a target application system, determining a system knowledge graph corresponding to the to-be-tested version; obtaining second system basic information of a target historical version of the target application system; based on the first system basic information, the second system basic information and the system knowledge graph, determining a to-be-tested path set of the to-be-tested version; wherein the to-be-tested path set comprises m updated service paths, and m is an integer greater than or equal to 1; and executing a test operation based on the to-be-tested path set to obtain a test result. The invention further discloses a testing device, equipment, a storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a testing method, apparatus, device, storage medium and computer program product. Background Art

[0002] With the rapid development of wireless communication technology, the application of wireless communication technology is becoming more and more extensive. In the application process of wireless communication technology, in order to ensure the user experience, the deployment and update iteration of wireless communication technology solutions are also relatively fast. In order to ensure the feasibility of wireless communication technology solutions, it is usually necessary to test and put them into use only after the test is passed. However, for example, when deploying the Network Slice Management Function (NSMF) system, different types of services constitute a variety of business scenarios, and different business processes are opened for different business scenarios. At this time, a large number of test branches will be formed. When performing scenario testing and iterative regression testing, the testing workload is large, and when the parameters change, it is difficult to evaluate the iterative test range. At present, in order to reduce the problem of difficulty in evaluating the iterative test range, the commonly used accurate test solution is to evaluate the scope of the version change by developing the code change range, and then select the corresponding automated test case to complete the version iteration test.

[0003] However, the current precise testing scheme mainly determines the impact scope of version changes through analysis of R&D codes. When there are problems with the R&D codes, the impact scope of version changes is inaccurately selected, which leads to inaccurate test results and seriously affects the test efficiency.

[0004] Application Contents

[0005] In order to solve the above-mentioned technical problems, the present application hopes to provide a testing method, apparatus, equipment, storage medium and computer program product, which solves the problem of low testing efficiency of current precise testing schemes, and proposes a testing method. When determining the test scope during the testing process, it fully considers business development needs, ensures the accuracy of the determined test scope, and improves testing efficiency.

[0006] The technical solution of this application is implemented as follows:

[0007] The present application provides a testing method, the method comprising:

[0008] Based on the first system basic information of the version to be tested of the target application system, determining the system knowledge graph corresponding to the version to be tested;

[0009] Acquire the second system basic information of the target historical version of the target application system;

[0010] Based on the first system basic information, the second system basic information and the system knowledge graph, determine a set of paths to be tested of the version to be tested; wherein the set of paths to be tested includes m updated service paths, where m is an integer greater than or equal to 1;

[0011] A test operation is performed based on the set of paths to be tested to obtain a test result.

[0012] In the above solution, the step of determining the system knowledge graph corresponding to the version to be tested based on the first system basic information of the version to be tested of the target application system includes:

[0013] Determine first priority information of the priority order of each first business key parameter corresponding to each application scenario included in the first system basic information;

[0014] Based on the product design flow chart included in the first system basic information, determine the first interface information, the second interface information and the second business key parameters corresponding to each business, and obtain the first product flow information;

[0015] Determine first triplet information of the version to be tested based on the first priority information, the first product process information and the first system interface file information; wherein the first system interface file information belongs to the first system basic information;

[0016] Based on the first triple information, the system knowledge graph is generated.

[0017] In the above solution, the first priority information of determining the priority order of each first business key parameter corresponding to each application scenario included in the first system basic information includes:

[0018] Determine a priority weight coefficient of each of the first business key parameters corresponding to each application scenario;

[0019] Based on the priority weight coefficients of all the first business key parameters included in the first system basic information, all the first business key parameters included in the first system basic information are sorted in order of the priority weight coefficients from large to small, and the priority order of each first business key parameter is obtained, and then the first priority information is obtained.

[0020] In the above solution, the step of determining the priority weight coefficient of each of the first business key parameters corresponding to each application scenario includes:

[0021] Determine a first weight coefficient and a second weight coefficient for each of the first business key parameters; wherein the sum of the first weight coefficient and the second weight coefficient is 1;

[0022] Determine during the actual operation, each of the first business key parameters is the cumulative sum of the actual values ​​of each enumerated parameter to obtain a first sum value;

[0023] Determine the cumulative sum of actual values ​​of all enumerated parameters of each of the first business key parameters during the actual operation to obtain a second sum value;

[0024] Calculating a ratio of the first sum value of each of the first business key parameters to the corresponding second sum value;

[0025] Calculating the product of the ratio of each of the first business key parameters and the corresponding first weight coefficient to obtain a first product;

[0026] Calculating the product of the preset priority order of each of the first business key parameters and the corresponding second weight coefficient to obtain a second product;

[0027] The sum of the first product of each first business key parameter and the corresponding second product is calculated to obtain a priority weight coefficient of each first business key parameter.

[0028] In the above solution, determining the first triplet information of the version to be tested based on the priority information, the first product process information and the first system interface file information includes:

[0029] Preprocessing the first priority information, the first product flow information and the first system interface file information respectively to obtain second priority information, second product flow information and second system interface file information in sequence;

[0030] The second priority information, the second product process information and the second system interface file information are analyzed to obtain a target entity, a target relationship and a target attribute; wherein the first triple information includes the target entity, the target relationship and the target attribute.

[0031] In the above solution, the second priority information, the second product process information and the second system interface file information are analyzed to obtain the target entity, including:

[0032] Parse the second product process information, obtain the business name, business link, system business interface and third business key parameters, and obtain a reference entity;

[0033] The objects included in the reference entity are unified using the second system interface file information to obtain the target entity.

[0034] In the above scheme, the target attributes include at least one or more of the following information: business identification information, business link identification information, interface attribute information and business parameter attributes; wherein, the interface attribute information includes at least one or more of the following information: interface identification information, interface parameter information, interface response information, upstream and downstream interface identification information, and the business parameter attributes include at least: the enumerated parameter value of the third business key parameter and the priority order of the third business key parameter.

[0035] In the above scheme, the target relationship includes at least one or more of the following relationships: upstream and downstream dependency relationships of scene links between multiple links, ownership relationships between services and links, ownership relationships between links and interfaces, and influence relationships between interfaces and third-party business key parameters.

[0036] In the above solution, determining the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph includes:

[0037] Determine first triplet information of the version to be tested based on the first system basic information;

[0038] Determine second triplet information of the target historical version based on the second system basic information;

[0039] The set of paths to be tested is determined based on the first triplet information, the second triplet information and the system knowledge graph.

[0040] In the above solution, determining the set of paths to be tested based on the first triplet information, the second triplet information and the system knowledge graph includes:

[0041] Performing update analysis on the first triplet information and the second triplet information to obtain an analysis result;

[0042] One or more complete links associated with the update objects included in the analysis result are determined from the system knowledge graph to obtain the set of paths to be tested.

[0043] In the above solution, the analysis result includes at least one or more of the following information: one or more updated entities, one or more updated relations, and one or more updated attributes.

[0044] In the above solution, the performing of the test operation based on the set of paths to be tested to obtain the test result includes:

[0045] Determining the path priority of each test path in the set of paths to be tested;

[0046] Arrange the paths in the set of paths to be tested based on the path priority of each of the test paths to obtain an arranged test case;

[0047] The arrangement test case is tested to obtain the test result.

[0048] In the above solution, determining the path priority of each test path in the set of paths to be tested includes:

[0049] Determining a priority weight coefficient of a fourth business critical parameter of each of the test paths in the set of paths to be tested;

[0050] The priority weight coefficients of the fourth business critical parameters of each of the test paths are accumulated to obtain the path priority of the corresponding test path.

[0051] In the above solution, the performing of the test operation based on the set of paths to be tested to obtain the test result includes:

[0052] Sending the set of paths to be tested to a test platform; wherein the test operation includes sending the set of paths to be tested,

[0053] Or, sending an orchestration test case obtained based on the set of paths to be tested to a test platform; wherein the test operation includes a sending operation of sending the orchestration test case to the test platform;

[0054] Receive the test result sent by the test platform; wherein the test result is obtained by the test platform based on the set of paths to be tested or the arranged test case.

[0055] The present application provides a testing device, the device comprising: a first determining unit, an acquiring unit, a second determining unit and an executing unit; wherein:

[0056] The first determining unit is used to determine the system knowledge graph corresponding to the version to be tested based on the first system basic information of the version to be tested of the target application system;

[0057] The acquisition unit is used to acquire the second system basic information of the target historical version of the target application system;

[0058] The second determination unit is used to determine the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph; wherein the set of paths to be tested includes m updated service paths, where m is an integer greater than or equal to 1;

[0059] The execution unit is used to execute a test operation based on the set of paths to be tested to obtain a test result.

[0060] The present application provides a network device, the device comprising: a communication interface, a memory, a processor and a communication bus; wherein:

[0061] The memory is used to store executable instructions;

[0062] The communication bus is used to realize the communication connection between the communication interface, the processor and the memory;

[0063] The processor is used to execute the test program stored in the memory to implement the steps of any of the test methods described above.

[0064] The present application provides a storage medium, on which a test program is stored. When the test program is executed, it is used to implement the steps of any of the test methods described above.

[0065] The present application provides a computer program product, including a computer program, wherein the computer program implements the steps of any of the above-mentioned testing methods when executed by a processor.

[0066] The embodiment of the present application provides a test method, device, equipment, system, storage medium and computer program product, which determines the system knowledge graph corresponding to the version to be tested based on the first system set information of the version to be tested of the target application system, obtains the second system basic information of the target historical version of the target application system, and then determines the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph, and finally, performs the test operation based on the set of paths to be tested to obtain the test result. In this way, after determining the system knowledge graph of the version to be tested of the target application system and the second system basic information of the target historical version to be compared and analyzed, the first system basic information, the second system basic information and the system knowledge graph of the version to be tested are analyzed to determine the set of paths to be tested, and the accurate determination of the test scope is achieved, which effectively narrows the test scope, so as to test the set of paths to be tested, and then obtain the test result of the version to be tested, thus solving the problem of low test efficiency of the current precise test scheme, and proposing a test method, which fully considers the business development needs when determining the test scope during the test process, ensures the accuracy of the determined test scope, and improves the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Schematic diagram of the test method provided in the embodiment of the present application Figure 1 ;

[0068] Figure 2 Schematic diagram of the test method provided in the embodiment of the present application Figure 2 ;

[0069] Figure 3 A schematic diagram of an implementation flow of an application embodiment of the test method provided in the embodiment of the present application;

[0070] Figure 4 A schematic diagram of a process for generating a knowledge graph of an NSFM system provided in an embodiment of the present application;

[0071] Figure 5 A schematic diagram of the structure of a NSMF system diagram database model provided in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of an implementation flow of determining a set of paths to be tested provided in an embodiment of the present application;

[0073] Figure 7 A schematic diagram of the structure of a testing device provided in an embodiment of the present application;

[0074] Figure 8 A schematic diagram of the structure of a network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0076] The embodiment of the present application provides a testing method, referring to Figure 1 As shown, the method is applied to a network device, and the method comprises the following steps:

[0077] Step 101: Based on the first system basic information of the version to be tested of the target application system, determine the system knowledge graph corresponding to the version to be tested.

[0078] In an embodiment of the present application, the target application system may be a service system that provides communication network services, such as an NSMF system. During the development and application process of the target application system, as the application requirements increase, and the functions are improved and enriched, the developers will continuously update and iterate the target application system, and only when the update and iteration complete the test and the test pass rate meets certain requirements, will the corresponding updated version be released. Therefore, during the test process, the network device determines the version to be tested that the target application system needs to test, and then obtains relevant information of the version to be tested, such as a product design flowchart of the version to be tested, relevant graphic and text information in the design process, development files, etc., to obtain the basic information of the first system, analyze and process the basic information of the first system, and obtain the system knowledge graph corresponding to the version to be tested.

[0079] Step 102: Obtain the second system basic information of the target historical version of the target application system.

[0080] In the embodiment of the present application, the target historical version is the historical version that is expected to be compared and analyzed with the version to be tested to determine the update range, usually a version that has passed the test in history. After determining the target historical version of the target application system, the second system basic information of the target historical version is obtained.

[0081] Step 103: Determine a set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph.

[0082] The path set to be tested includes m updated service paths, where m is an integer greater than or equal to 1.

[0083] In an embodiment of the present application, the first system basic information and the second system basic information are analyzed to obtain analysis results, and the analysis results are used to perform a determination analysis from the system knowledge graph to obtain m updated business paths in the version to be tested, which are stored as a set of paths to be tested.

[0084] Step 104: Perform a test operation based on the set of paths to be tested to obtain a test result.

[0085] In the embodiment of the present application, according to the set of paths to be tested, the corresponding test case is used to perform the test operation to obtain the test result. The corresponding test operation can be performed by the network device itself, or the network device can send the test task for the set of paths to be tested to the test platform, which is implemented by the test platform. The specific operation can be determined according to the actual application scenario and is not specifically limited here.

[0086] The test method provided in the embodiment of the present application determines the system knowledge graph corresponding to the version to be tested based on the first system set information of the version to be tested of the target application system, obtains the second system basic information of the target historical version of the target application system, and then determines the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph, and finally performs the test operation based on the set of paths to be tested to obtain the test result. In this way, after determining the system knowledge graph of the version to be tested of the target application system and the second system basic information of the target historical version to be compared and analyzed, the first system basic information, the second system basic information and the system knowledge graph of the version to be tested are analyzed to determine the set of paths to be tested, thereby achieving accurate determination of the test scope, effectively narrowing the test scope, and testing the set of paths to be tested, and then obtaining the test result of the version to be tested. In this way, the problem of low test efficiency of the current precise test scheme is solved, and a test method is proposed. When determining the test scope during the test process, the business development needs are fully considered, the accuracy of the determined test scope is guaranteed, and the test efficiency is improved.

[0087] Based on the above embodiments, the embodiments of the present application provide a testing method, which is applied to a network device, referring to Figure 2 As shown, the method comprises the following steps:

[0088] Step 201: Determine first priority information including a priority order of each first business key parameter corresponding to each application scenario from first system basic information.

[0089] In an embodiment of the present application, the first system basic information is analyzed to determine all application scenarios set in the first system basic information, and then the first business key parameters used in each application scenario are determined, and the priority order of each first business key parameter is calculated to obtain the first priority information corresponding to each application scenario included in the first system basic information.

[0090] Step 202: Based on the product design flow chart included in the first system basic information, determine the first interface information, the second interface information and the second business key parameters corresponding to each business, and obtain the first product flow information.

[0091] In an embodiment of the present application, the product design flowchart is a development flowchart drawn during the development and design process of the product, and includes at least information such as business links, interfaces corresponding to the links, and business key parameters of the interfaces. All product design flowcharts included in the first system basic information are obtained, and the first interface information, second interface information, and first business key parameters included in the product design flowchart are determined from the product design flowchart, so as to obtain the corresponding product process information, that is, the first product process information. Among them, the first interface information can be the interface information of the internal interface included in the product design flowchart, and the second interface information can be the interface information of the external interface included in the product design flowchart. The external interface is an interface used for the target application system to communicate with an external communication network, such as the Internet, a communication terminal, etc.

[0092] Step 203: Determine first triplet information of the version to be tested based on the first priority information, the first product process information and the first system interface file information.

[0093] Among them, the first system interface file information belongs to the first system basic information.

[0094] In the embodiment of the present application, the first system interface file information is obtained, which is file information used to define the system interface in the version to be tested of the target application system. The network device analyzes the first priority information, the first product process information, and the first system interface file information, and extracts all triplet information included in the version to be tested, which is recorded as the first triplet information.

[0095] Step 204: Generate a system knowledge graph based on the first triple information.

[0096] In an embodiment of the present application, logical analysis, logical reasoning and other analysis and processing operations are performed on the determined first triple information to determine the association relationships between the entities included in the version to be tested, wherein the association relationships include hidden relationships and explicit relationships between the entities, thereby expanding the system knowledge graph.

[0097] Step 205: Obtain the second system basic information of the target historical version of the target application system.

[0098] In the embodiment of the present application, the target historical version of the target application system can be any tested and tested version of the target application system. After determining the target historical version of the target application system that needs to be compared and analyzed with the version to be tested, the corresponding second system basic information is obtained.

[0099] Step 206: Determine a set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph.

[0100] The path set to be tested includes m updated service paths, where m is an integer greater than or equal to 1.

[0101] In an embodiment of the present application, the first system basic information and the second system basic information are analyzed to determine the updated content in the first system basic information, and the complete path including the updated content is determined from the system knowledge graph to obtain m updated business paths, thereby obtaining a set of paths to be tested of the version to be tested. In some application scenarios, the system knowledge graph corresponding to the second system basic information can also be generated for the second system basic information, and then the system knowledge graph corresponding to the second system basic information is compared and analyzed with the system knowledge graph of the first system basic information to determine the updated path in the first system basic information, and obtain the set of paths to be tested of the version to be tested. However, when generating the system knowledge graph information corresponding to the second system basic information, some information may be removed. Therefore, the second system basic information can be compared and analyzed with the first system basic information to reduce the leakage of updated information, so that a more comprehensive and accurate set of paths to be tested can be obtained.

[0102] Step 207: Perform a test operation based on the set of paths to be tested to obtain a test result.

[0103] Based on the above embodiments, in other embodiments of the present application, step 201 can be implemented by steps 201a to 201b:

[0104] Step 201a: Determine the priority weight coefficient of each first business key parameter corresponding to each application scenario.

[0105] In the embodiment of the present application, the priority weight coefficient of each first business-critical parameter included in each application scenario may be a preset empirical value, or may be calculated using an empirical calculation formula based on actual application.

[0106] Step 201b, based on the priority weight coefficients of all the first business key parameters included in the first system basic information, sort all the first business key parameters included in the first system basic information in order of the priority weight coefficients from large to small, obtain the priority order of each first business key parameter, and then obtain the first priority information.

[0107] In an embodiment of the present application, after determining the priority weight coefficients of all the first business key parameters included in the first system basic information, the priority weight coefficients of all the first business key parameters are sorted according to size to obtain the sorting result of the priority weight coefficients, and the corresponding first business key parameters are sorted according to the sorting result of the priority weight coefficients, and the sorting order of each first business key parameter is determined as its priority order. In this way, the first priority information corresponding to the first system basic information can be determined.

[0108] Based on the above embodiment, in other embodiments of the present application, step 201a can be implemented by steps a11 to a17:

[0109] Step a11: Determine a first weight coefficient and a second weight coefficient of each first business key parameter.

[0110] The sum of the first weight coefficient and the second weight coefficient is 1.

[0111] In an embodiment of the present application, the second weight coefficient is an empirical value pre-set for each first business key parameter. In actual situations, it can be adjusted according to actual requirements. Since the sum of the first weight coefficient and the second weight coefficient is 1, therefore, on the premise of determining the second weight coefficient, the first weight coefficient can be calculated by the formula "first weight coefficient = 1-second weight coefficient".

[0112] Step a12: determine that during the actual operation, each first business key parameter is the cumulative sum of the actual values ​​taken when each parameter is enumerated to obtain a first sum value.

[0113] In an embodiment of the present application, the version to be tested is actually run, and the actual value of each enumerated parameter of each first business-critical parameter during the actual running of the version to be tested is obtained, and then the actual value of the first business-critical parameter is accumulated and calculated to obtain a first sum value. It should be noted that for each first business-critical parameter, at least one enumerated parameter can be set during the actual setting.

[0114] Step a13: Determine the cumulative sum of actual values ​​of all enumerated parameters of each first business key parameter during the actual operation to obtain a second sum value.

[0115] In the embodiment of the present application, actual values ​​of all enumerated parameters of each first business-critical parameter are accumulated and calculated to obtain a second sum value.

[0116] Step a14: Calculate the ratio of the first sum value of each first business key parameter to the corresponding second sum value.

[0117] Step a15: Calculate the product of the ratio of each first business key parameter and the corresponding first weight coefficient to obtain a first product.

[0118] Step a16: Calculate the product of the preset priority order of each first business key parameter and the corresponding second weight coefficient to obtain a second product.

[0119] Step a17: Calculate the sum of the first product of each first business key parameter and the corresponding second product to obtain the priority weight coefficient of each first business key parameter.

[0120] Based on the above embodiments, in other embodiments of the present application, step 203 can be implemented by steps 203a to 203b:

[0121] Step 203a: pre-process the first priority information, the first product flow information and the first system interface file information respectively to obtain the second priority information, the second product flow information and the second system interface file information in sequence.

[0122] In the embodiment of the present application, the preprocessing may include one or more combined processing methods such as information cleaning operation, normalization and unification processing, and text information segmentation. The information cleaning operation may include deleting irrelevant content, such as deleting irrelevant symbols, etc. The first priority information is preprocessed to obtain the second priority information, the first product process information is preprocessed to obtain the second product process information, and the first system interface file information is preprocessed to obtain the second system interface file information.

[0123] Step 203b: Analyze the second priority information, the second product process information and the second system interface file information to obtain the target entity, the target relationship and the target attribute.

[0124] The first triplet information includes a target entity, a target relationship and a target attribute.

[0125] In an embodiment of the present application, entity objects are extracted from the second priority information, the second product process information, and the second system interface file information to obtain a target entity. After the target entity is determined, various relationship analyses of the entity objects are performed from the second priority information, the second product process information, and the second system interface file information to obtain the corresponding target relationship. Furthermore, the second priority information, the second product process information, and the second system interface file information are further processed to obtain target attributes related to the target entity.

[0126] Based on the above embodiment, in other embodiments of the present application, step 203b can be implemented by steps b11 to b12:

[0127] Step b11: parse the second product process information, obtain the business name, business link, system business interface and third business key parameters, and obtain a reference entity.

[0128] In an embodiment of the present application, the business name, business link, system business interface and third business key parameters are obtained from the second product process information, and these parameters are used as entity objects to obtain a reference entity.

[0129] Step b12: Use the second system interface file information to unify the objects included in the reference entity to obtain the target entity.

[0130] In an embodiment of the present application, the reference entity obtained from the second product process information is supplemented and unified according to the second system interface file information, so that the objects in the second system interface file information and the second product process information that are essentially the same entity object but have different naming methods are treated as equivalent. In this way, the reference entity is improved to obtain the target entity.

[0131] Based on the foregoing embodiments, in other embodiments of the present application, the target attributes include at least one or more of the following information: business identification information, business link identification information, interface attribute information and business parameter attributes; wherein, the interface attribute information includes at least one or more of the following information: interface identification information, interface parameter information, interface response information, upstream and downstream interface identification information, and the business parameter attributes include at least: the enumerated parameter value of the third business key parameter and the priority order of the third business key parameter.

[0132] In the embodiment of the present application, the service identification information is identification information used to uniquely identify the service, such as a service name, a service number, etc., and the service link identification information is identification information used to uniquely identify the service link, such as a service link name, a service link number, etc. The priority order of the third corruption key parameter can be determined by the second priority information.

[0133] Based on the foregoing embodiments, in other embodiments of the present application, the target relationship includes at least one or more of the following relationships: upstream and downstream dependency relationships of scenario links between multiple links, the relationship between services and links, the relationship between links and interfaces, and the influence relationship between interfaces and third-party business key parameters.

[0134] Based on the above embodiment, in other embodiments of the present application, step 206 may be composed of steps 206a to 206c:

[0135] Step 206a: Determine first triplet information of the version to be tested based on the first system basic information.

[0136] Step 206b: Determine the second triplet information of the target historical version based on the second system basic information.

[0137] Step 206c: Determine a set of paths to be tested based on the first triplet information, the second triplet information and the system knowledge graph.

[0138] In an embodiment of the present application, a comparison and analysis is performed on the first triplet information and the second triplet information to obtain an analysis result, and a set of paths to be tested is determined from the system knowledge graph based on the analysis result.

[0139] Based on the above embodiment, in other embodiments of the present application, step 206c can be implemented by steps c11 to c12:

[0140] Step c11: update and analyze the first triplet information and the second triplet information to obtain analysis results.

[0141] In the embodiment of the present application, the first triplet information and the second triplet information are compared and analyzed for updated content, the updated content in the version to be tested is determined, and an analysis result is obtained.

[0142] Step c12: determine one or more complete links associated with the update objects included in the analysis results from the system knowledge graph to obtain a set of paths to be tested.

[0143] The analysis result includes at least one or more of the following information: one or more updated entities, one or more updated relations, and one or more updated attributes.

[0144] In the embodiment of the present application, the system knowledge graph is analyzed using the analysis results to determine the system implementation graph as long as it includes the link of the update object in the analysis results, or the link affected by the update object, to obtain a set of paths to be tested. A complete link refers to the path from the beginning of the link to the end of the link.

[0145] Based on the above embodiments, in other embodiments of the present application, step 207 can be implemented by steps 207a to 207c:

[0146] Step 207a: Determine the path priority of each test path in the set of paths to be tested.

[0147] In an embodiment of the present application, the path priority of each test path can be determined by analyzing the priorities of the path nodes in each test path. For example, the path priority of each test path can be the accumulation of the priority order weight coefficients of all path nodes of each test path, or the accumulation of the priority order weight coefficients of all path nodes of each test path and then averaging, or the weighted averaging of the priority order weight coefficients of all path nodes of each test path. The path nodes can be all the determined business parameters, including business critical parameters. Business critical parameters are parameters that affect the direction of the business path.

[0148] Step 207b: Arrange the paths in the to-be-tested path set based on the path priority of each test path to obtain an arranged test case.

[0149] In the embodiment of the present application, according to application requirements and based on the path priority of each test path, the paths in the path set to be tested are arranged and combined to obtain different arranged test cases that need to be tested.

[0150] Step 207c: Test the arranged test case to obtain the test result.

[0151] In the embodiment of the present application, a test run is performed on the orchestration test case to obtain a test result.

[0152] Based on the above embodiment, in other embodiments of the present application, step 207a can be implemented by steps d11 to d12:

[0153] Step d11: Determine the priority weight coefficient of the fourth key service parameter of each test path in the set of paths to be tested.

[0154] In the embodiment of the present application, the priority weight coefficient of the fourth business critical parameter of each test path is determined by the second priority information.

[0155] Step d12: Accumulate the priority weight coefficient of the fourth key service parameter of each test path to obtain the path priority of the corresponding test path.

[0156] Based on the above embodiments, in other embodiments of the present application, step 207 may also be implemented by step 207d and step 207f, or steps 207e to 207f:

[0157] Step 207d: Send the path set to be tested to the test platform.

[0158] The test operation includes a sending operation of sending a set of paths to be tested to a test platform.

[0159] In the embodiment of the present application, the network device can directly send the path set to be tested to the test platform, so that the test platform generates an orchestration test case based on the path set to be tested, and then tests the orchestration test case to obtain a test result. The process of the test platform generating an orchestration test case based on the path set to be tested is the same as the process of steps 207a to 207b performed by the aforementioned network device, and will not be described in detail here.

[0160] Step 207e: Send the orchestration test case obtained based on the set of paths to be tested to the test platform.

[0161] The test operation includes a sending operation of sending the orchestrated test case to the test platform.

[0162] In the embodiment of the present application, the process of arranging test cases obtained by the network device based on the set of paths to be tested can refer to the implementation process of steps 207a to 207b, which will not be described in detail here.

[0163] Step 207f, receiving the test results sent by the test platform.

[0164] The test results are obtained by the test platform based on a set of paths to be tested or an arrangement of test cases.

[0165] Based on the above-mentioned embodiment, there are three types of services in the NSMF system: premium service, exclusive service and VIP service, corresponding to which are three types of slices: default slice type, dedicated slice type and general slice type. Furthermore, each type of slice can be divided into shared type and non-shared type. Among them, customers of different business types can share slices. In this way, different types of services can be extended to a variety of business scenarios, and a total of 6 business scenarios can be provided, namely: reuse of default slice + new dedicated data network name (Data Network Name, DNN), reuse of general slice + new dedicated DNN, original default slice + new dedicated DNN, original general slice + new dedicated DNN, original dedicated slice + new dedicated DNN, new dedicated slice + new dedicated DNN. In the 6 business scenarios, because different services have different processes for opening and changing services, there are also order interface business parameters for the first-level business orchestration center and the second-level business orchestration center. Based on this, the embodiment of the present application provides a testing method for the target application system as the NSMF system, with reference to Figure 3 As shown, the specific implementation steps include the following:

[0166] Step e11: Obtain the business knowledge of the version to be tested of the NSMF system.

[0167] The business knowledge includes: business priority, product process template, and interface document information. The business priority corresponds to the aforementioned first priority information, the product process module corresponds to the aforementioned first product process information, and the interface document information corresponds to the aforementioned first system interface file information.

[0168] The specific process of determining the business priority can be as follows: The NSMF system marks the scene priority based on the work order of the existing network production environment. Because the work order of the existing network production environment can reflect the customer's business demands, the work order information of the existing network production environment can be read through the server interface, and then the business key parameters associated with the scene can be obtained through the work order query interface through the work order information, and the priority of the obtained business key parameters can be calculated. The priority of the business key parameters can be calculated using the following formula:

[0169] Where α is the weight value of the priority statistics of the business key parameters in the work order information of the current network production environment, β is the preset priority weight value of the business key parameters, and the sum of α and β is 1. Pre(p nk )≤1 is the preset priority of the business key parameters; p nk Indicates the kth enumeration value of the nth parameter in a certain business scenario of the NSMF system, where n = 1, 2, ..., N, k = 1, 2, ..., K n , N represents the total number of key business parameters, the nth key business parameter, a total of K n value; f(p nk ) represents the parameter p nk The final priority coefficient of f mn Indicates the relationship between the nth parameter of the mth work order and a certain application scenario. If f mn =1 indicates that the nth parameter of the mth work order is related to a certain application scenario. mn =0 indicates that the nth parameter of the mth work order is irrelevant to the application scenario; It means that there are M work orders that need to be prioritized. Indicates the kth enumeration value of the key business parameter n in the mth work order.

[0170] The specific definition process of the product flow module can be as follows: For example, when the NSMF system includes services such as opening survey, opening, changing survey, changing, and stopping, each service will design a product flow chart at the initial stage of product design. The product flow chart includes information such as the links of the service, the interfaces corresponding to the links, and the key business parameters of the interfaces. Therefore, the product flow chart is parsed to obtain a product flow template. The product flow template can be stored in the form of JS key-value pair data (JavaScriptObject Notation, JSON), another markup language (YAML Ain't Markup Language, YAML) file, etc., so as to facilitate subsequent information extraction. For example, the product flow chart is parsed to obtain information such as the internal interface, external interface, and value information of key business parameters of each link corresponding to each service. In the implementation process, when parsing the product flow chart, it can be through identifying the key information involved in the product flow chart, and then realizing the generation of the product flow template through code parsing. Correspondingly, a product flow template can be recorded in the following way:

[0171]

[0172]

[0173] It should be noted that the key parameters in the above description are the same as the aforementioned business key parameters, which are interface parameters in the business process that affect the direction of the business process scenario branch.

[0174] The interface document information includes at least: internal interface document and external interface document. Among them, the internal interface recorded in the internal interface document usually refers to the interface defined by the designers of the NSMF system software to complete the internal calls of various businesses, and the external interface defined in the external interface document is the enterprise standard interface designed by the designers to enable the NSMF system to interact with the peripheral system. The interface document information includes at least: interface name, interface uniform resource locator (Uniform ResourceLocator, URL), interface parameters, interface response, upstream and downstream interface identification and other key information.

[0175] Step e12: Based on business knowledge, extract triples including entities, attributes and relationships of the NSMF system.

[0176] Among them, the specific process of triple extraction is mainly achieved through text analysis and information extraction, which specifically includes the following steps:

[0177] Step e121: document preprocessing.

[0178] Among them, the preprocessed documents include at least business knowledge: priority tables that record business priorities, product process templates, and interface document information. Document preprocessing mainly involves text cleaning of business knowledge to remove irrelevant content in text documents, such as irrelevant symbols, and corresponding processing of document content, such as word segmentation of document content and marking of word parts in the document to facilitate subsequent extraction of entities, relationships, and attributes. When analyzing and processing the cleaned document content, word segmentation tools can be used.

[0179] Step e122: entity extraction.

[0180] An entity is a specific object or concept, such as a person, a place, an organization, etc. In this application, an entity specifically refers to a specific object such as a business, a link, an interface, or a key business parameter.

[0181] Step e123: Relationship extraction.

[0182] Relationship is used to represent a certain connection or interaction between entities. In the embodiment of the present application, the relationship includes at least: the upstream and downstream dependency relationship of the scene link between multiple links, the belonging relationship between the business and the link, the belonging relationship between the link and the interface, the influence relationship between the interface and the key business parameters, etc.

[0183] Step e124: attribute extraction.

[0184] Attributes are descriptive information for entities. In the embodiment of the present application, attributes include at least: attribute business identifier of business entities; attribute link identifier of link entities; attributes of interface entities: interface identifier, interface parameters, interface response, upstream and downstream interface identifiers, etc., among which the enumeration value of key business parameters and the priority weight coefficient of key business parameters also belong to the attributes of the corresponding entity.

[0185] It should be noted that in the process of extracting entities, relationships, and attributes, the extraction methods used can be rule-based methods, machine learning-based methods, and / or unsupervised, semi-supervised, supervised, and other learning methods. Exemplarily, the documents of the business knowledge base of the NSMF system are usually structured data, so a rule-based method can be used to complete the triple extraction of entities, attributes, and relationships, that is, the product process template and interface document information can be parsed by a computer extraction application, and then the key information can be matched by a defined regular expression. In some implementation schemes, the priority table, product process template, and interface document information can also be uploaded to the multimodal large model, and the triple extraction can be completed through the multimodal large model debugging prompt.

[0186] Step e125, complete entity alignment in the product process template and interface document information.

[0187] Among them, since the product process module and interface document information usually come from different data sources, and in different data sources, different identifiers or descriptions may be used to represent the same entity. Therefore, it is necessary to use entity alignment to identify and merge entities with different descriptions to avoid the analysis impact caused by data redundancy and ambiguity. In the embodiment of the present application, entities include businesses, links, interfaces and key business parameters. The links, interfaces and key business parameters are defined in the product process template and the interface document information. The interface document information also includes internal interface documents and external interface documents. Since the product process template, internal interface documents and external interface documents are respectively derived from product personnel, designers and solutions, there is a phenomenon that the same entity is defined differently due to different system roles and different starting points. At this time, the internal interface document and the external interface document can be used to merge and align the entities extracted in step e122.

[0188] For example, taking the opening of the exploration business in the NSMF system as an example, for the key business parameters of the first link order delivery interface: the external interface document defines it as: usage = 2 new dedicated slices / reuse of old general slices + new dedicated DNN, and the product process template defines it as: usage = 2 reuse of old general slices + new dedicated DNN, usage = 6 new dedicated slices + new dedicated DNN. In this way, it can be analyzed through equivalent relationship reasoning, similarity calculation, feature calculation, etc. to determine that the usage of the key business parameters for the first link order delivery interface in the external interface document and the product process template has the same meaning and belongs to the same entity. The entity attribute values ​​of the usage parameters are merged, for example, they can be recorded as: usage = 2; usage = 6.

[0189] In this way, by parsing business knowledge, the priority weight coefficients of key business parameters are obtained to obtain a priority table. Then, based on business knowledge, entities such as business, links, interfaces and business key parameters are extracted, and the corresponding relationships and attributes are obtained to obtain triple information. Finally, interface information such as interface identification, interface parameters, interface response, upstream and downstream interface identification, etc. of internal interface documents and external enterprise standard documents are extracted as attribute values ​​of interface entities, mapped to business process templates, and the extracted interface and key parameter entity triples are updated accordingly to obtain the final triple information.

[0190] Step e13: Generate the NSMF system knowledge graph based on the triple information.

[0191] The specific process of generating the NSMF system knowledge graph in step e13 can be referred to Figure 4As shown, the following steps are included:

[0192] Step e131, define the NSFM system graph data model and obtain the selected graph database model.

[0193] Among them, select a suitable graph database, such as Neo4j, JanusGraph, ArangoDB, etc. These databases support graph data models and can effectively store and query triple data. The structure of the NSMF system graph database model can be, for example, Figure 5 As shown, it includes: business node-inclusion->link node->interface->interface node-association->key parameter-prefix->link infrastructure, where: Figure 5 In the example, the structure of interface->interface node-association->key parameter-prefix->link collectively includes the basic structure of internal interface node and external interface node.

[0194] Step e132: input the triple information into the selected graph database model, perform entity modeling and operation interface encapsulation on the triple information through the selected graph database model, and obtain the encapsulation result.

[0195] Among them, based on the selected graph database and in accordance with the business data model, entity modeling and operation interface encapsulation are completed based on triple information.

[0196] Step e133: Generate the NSFM system knowledge graph based on the encapsulation results.

[0197] Among them, because the knowledge graph can structure and integrate information, it is convenient for information organization and intelligent processing. Therefore, due to the complex relationship between the business scenarios, business key parameters, and link interfaces of the NSMF system, the knowledge graph can be used for storage and representation, which is convenient for intelligently obtaining scenario test paths, determining the business scenarios affected by software iteration updates, and making the test more precise. The three basic elements of the knowledge graph are entities, relationships, and attributes. By constructing triples of the three basic elements, logical reasoning can be performed to discover the implicit relationship between entities, thereby expanding the knowledge graph.

[0198] Taking Neo4j as an example, based on the object-oriented neomodel library, we can define the data model business node Business, step node Step, interface Interface, and key parameter Parameter class, create connections between nodes, query and update to complete automatic management, and use the application programming interface (Application Programming Interface, API) to convert triple information into the NSMF system knowledge graph.

[0199] Step e14: Differentiate the business knowledge between the version to be tested and the previous version, obtain the entity change scope of the version to be tested, and determine the path set to be tested from the NSMF system knowledge graph based on the entity change scope.

[0200] The specific implementation process of step e14 can refer to Figure 6 As shown, the specific steps include:

[0201] Step e141: Determine triplet information of the version to be tested based on the business knowledge of the version to be tested.

[0202] Step e142: Determine the triplet information of the previous version based on the business knowledge of the previous version.

[0203] Among them, the product process templates, interface documents, and calculation priorities of the version to be tested and the previous version are parsed respectively, and the attributes of the four types of entities, Business, Step, Interface, and Parameter, and the triple information of the relationship between entities are extracted.

[0204] Step e143: differentially process the triplet information of the version to be tested and the triplet information of the previous version to determine and obtain updated triplet information of the version to be tested.

[0205] Among them, by comparing and analyzing the entity triplet information of the previous version, the changed entities, changed attributes and changed relationships such as changed business, changed links, changed interfaces and changed parameters are obtained.

[0206] It should be noted that the updated triplet information includes: information that there is no new entity, only the attributes and relationships of the original entity have changed, and the other is the new entity and the corresponding attributes and relationships.

[0207] Step e144: Based on the updated triple information, determine the set of paths to be tested from the NSMF system knowledge graph.

[0208] In an embodiment of the present application, a graph database query method is used to query and update all nodes in the triplet information that have any connection with a specific node, and then for each business link, the links between the link entities from the beginning to the end are matched, including all links of associated entities. Each link corresponds to a scenario test branch, and each link information includes Business, Step, Interface entity identifiers, parameter node id, value, priority attribute value, and entity sequence identifier. In this way, the scenario analysis affected in the version to be tested can be obtained, the set of paths to be tested can be obtained, and the determination of accurate test paths can be achieved.

[0209] Step e15: Send the path set to be tested to the automated testing platform.

[0210] The set of paths to be tested is sent to the automated testing platform through an interface for communicating with the automated testing platform. The automated testing platform corresponds to the aforementioned testing platform.

[0211] Step e16: The automated testing platform receives the set of paths to be tested, executes automated testing operations, and outputs a test report.

[0212] Among them, the automated test platform performs corresponding link interface arrangement on the set of paths to be tested, and after obtaining the corresponding test cases, it executes the automated test operation to obtain the test report. The specific process can be: after the automated test platform receives the set of paths to be tested, it first calculates the link priority of each test path, wherein the link priority can be the cumulative sum of the priority attribute values ​​of all parameter nodes included in the test path, that is, the priority weight coefficient; then, according to the service identifier, link identifier, interface identifier, key service identifier, enumerated actual values ​​of key service parameters, entity sequence identifier and other information, the atomic interface automation case completed on the automation platform is sequentially arranged. During the arrangement process, due to the huge number of NSMF system scenario branches, the scenario automation case can be executed in the order of the calculated link priority value, and the test report can be output after the test is completed.

[0213] In this way, the automated testing platform can display the test report in the display area of ​​the automated testing platform, or send the test report to the network device so that it can be displayed in the corresponding display area of ​​the network device, or send it to a third-party display device so that relevant personnel can quickly know the test results.

[0214] In this way, the proposed accurate testing scheme based on knowledge graph of NSMF system extracts the triples of entities and attributes such as business, links, key business parameters, interfaces, etc. of the NSMF system to be tested, and stores them in the graph database to generate the scene link knowledge graph of NSMF system, and then analyzes it with the version to be compared. It can quickly traverse and retrieve the scene links that need to be affected by the version to be tested through the path query of the scene link knowledge graph. In addition, by defining the business priority information to form the link priority, it can prioritize the important scene links and realize automated testing. Through priority design and knowledge graph storage, the scene test of NSMF system can be completed efficiently and accurately. By screening out the scenes updated by the version to be tested for testing, the testing time is saved; compared with the method of manually determining the test scope, the definition of knowledge graph makes the search for the test scope more accurate and complete, avoiding missed tests; and the analyzed business knowledge is derived from the product side information. Starting from the business source, it can avoid the problem of inaccurate accurate test scope assessment caused by R&D code defects, making the complex scene testing process of NSMF more accurate, scientific and automated, which can effectively improve the testing efficiency and save the testing manpower cost.

[0215] It should be noted that, for the description of the same steps and the same contents in this embodiment as those in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0216] The test method provided in the embodiment of the present application determines the system knowledge graph corresponding to the version to be tested based on the first system set information of the version to be tested of the target application system, obtains the second system basic information of the target historical version of the target application system, and then determines the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph, and finally performs the test operation based on the set of paths to be tested to obtain the test result. In this way, after determining the system knowledge graph of the version to be tested of the target application system and the second system basic information of the target historical version to be compared and analyzed, the first system basic information, the second system basic information and the system knowledge graph of the version to be tested are analyzed to determine the set of paths to be tested, thereby achieving accurate determination of the test scope, effectively narrowing the test scope, and testing the set of paths to be tested, and then obtaining the test result of the version to be tested. In this way, the problem of low test efficiency of the current precise test scheme is solved, and a test method is proposed. When determining the test scope during the test process, the business development needs are fully considered, the accuracy of the determined test scope is guaranteed, and the test efficiency is improved.

[0217] Based on the above embodiments, the embodiments of the present application provide a testing device, which can be applied to Figures 1-2 In the test method provided in the corresponding embodiment, refer to Figure 7 As shown, the test device 3 may include: a first determination unit 31, an acquisition unit 32, a second determination unit 33 and an execution unit 34; wherein:

[0218] A first determining unit 31 is used to determine a system knowledge graph corresponding to the version to be tested based on the first system basic information of the version to be tested of the target application system;

[0219] An acquisition unit 32 is used to acquire the second system basic information of the target historical version of the target application system;

[0220] The second determination unit 33 is used to determine a set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph; wherein the set of paths to be tested includes m updated service paths, where m is an integer greater than or equal to 1;

[0221] The execution unit 34 is used to execute a test operation based on the set of paths to be tested to obtain a test result.

[0222] In other embodiments of the present application, the first determination unit includes: a first determination module, a second determination module, a third determination module and a fourth determination module; wherein:

[0223] A first determination module, used to determine first priority information of a priority order of each first business key parameter corresponding to each application scenario included in the first system basic information;

[0224] A second determination module is used to determine the first interface information, the second interface information and the second business key parameter corresponding to each business based on the product design flow chart included in the first system basic information, and obtain the first product flow information;

[0225] A third determination module is used to determine the first triplet information of the version to be tested based on the first priority information, the first product process information and the first system interface file information; wherein the first system interface file information belongs to the first system basic information;

[0226] The fourth determination module is used to generate a system knowledge graph based on the first triple information.

[0227] In other embodiments of the present application, the first determination module is specifically used to implement the following steps:

[0228] Determine the priority weight coefficient of each first business key parameter corresponding to each application scenario;

[0229] Based on the priority weight coefficients of all first business key parameters included in the first system basic information, all first business key parameters included in the first system basic information are sorted in order of the priority weight coefficients from large to small, to obtain the priority order of each first business key parameter, and then obtain the first priority information.

[0230] In other embodiments of the present application, when the first determination module is used to implement the step of determining the priority weight coefficient of each first business key parameter corresponding to each application scenario, it can be implemented by the following steps:

[0231] Determine a first weight coefficient and a second weight coefficient of each first business key parameter; wherein the sum of the first weight coefficient and the second weight coefficient is 1;

[0232] Determine that during the actual operation, each first business key parameter is the cumulative sum of the actual values ​​of each enumerated parameter to obtain a first sum value;

[0233] Determine the cumulative sum of actual values ​​of all enumerated parameters of each first business key parameter during the actual operation to obtain a second sum value;

[0234] Calculate the ratio of the first sum value of each first business key parameter to the corresponding second sum value;

[0235] Calculating the product of the ratio of each first business key parameter and the corresponding first weight coefficient to obtain a first product;

[0236] Calculating the product of the preset priority order of each first business key parameter and the corresponding second weight coefficient to obtain a second product;

[0237] The sum of the first product of each first business key parameter and the corresponding second product is calculated to obtain the priority weight coefficient of each first business key parameter.

[0238] In other embodiments of the present application, the third determination module is specifically used to implement the following steps:

[0239] Preprocessing the first priority information, the first product flow information and the first system interface file information respectively, and obtaining the second priority information, the second product flow information and the second system interface file information in sequence;

[0240] The second priority information, the second product process information and the second system interface file information are analyzed to obtain a target entity, a target relationship and a target attribute; wherein the first triplet information includes the target entity, the target relationship and the target attribute.

[0241] In other embodiments of the present application, when the third determination module executes the step of analyzing the second priority information, the second product process information, and the second system interface file information to obtain the target entity, the following steps can be performed:

[0242] Parse the second product process information, obtain the business name, business link, system business interface and third business key parameters, and obtain the reference entity;

[0243] The second system interface file information is used to perform unified processing on the objects included in the reference entity to obtain the target entity.

[0244] In other embodiments of the present application, the target attributes include at least one or more of the following information: business identification information, business link identification information, interface attribute information and business parameter attributes; wherein, the interface attribute information includes at least one or more of the following information: interface identification information, interface parameter information, interface response information, upstream and downstream interface identification information, and the business parameter attributes include at least: the enumerated parameter value of the third business key parameter and the priority order of the third business key parameter.

[0245] In other embodiments of the present application, the target relationship includes at least one or more of the following relationships: upstream and downstream dependency relationships of scenario links between multiple links, ownership relationships between services and links, ownership relationships between links and interfaces, and influence relationships between interfaces and third-party business key parameters.

[0246] In other embodiments of the present application, the second determination unit includes: a fifth determination module and a sixth determination module; wherein:

[0247] A fifth determination module, configured to determine first triplet information of a version to be tested based on the first system basic information;

[0248] A fifth determination module, configured to determine second triplet information of a target historical version based on the second system basic information;

[0249] The sixth determination module is used to determine a set of paths to be tested based on the first triplet information, the second triplet information and the system knowledge graph.

[0250] In other embodiments of the present application, the sixth determination module is specifically used to implement the following steps:

[0251] Performing update analysis on the first triplet information and the second triplet information to obtain an analysis result;

[0252] One or more complete links associated with the update objects included in the analysis results are determined from the system knowledge graph to obtain a set of paths to be tested.

[0253] In other embodiments of the present application, the analysis result includes at least one or more of the following information: one or more updated entities, one or more updated relations, and one or more updated attributes.

[0254] In other embodiments of the present application, the execution unit includes: a seventh determination module, an arrangement module and a test module; wherein:

[0255] A seventh determination module, used to determine the path priority of each test path in the set of paths to be tested;

[0256] An orchestration module, used for orchestrating the paths in the to-be-tested path set based on the path priority of each test path, to obtain an orchestrated test case;

[0257] The test module is used to test the arranged test cases and obtain the test results.

[0258] In other embodiments of the present application, the seventh determination module is specifically used to implement the following steps:

[0259] Determine the priority weight coefficient of the fourth business critical parameter of each test path in the set of paths to be tested;

[0260] The priority weight coefficient of the fourth key service parameter of each test path is accumulated to obtain the path priority of the corresponding test path.

[0261] In other embodiments of the present application, the execution unit includes: a sending module and a receiving module; wherein:

[0262] A sending module, used for sending the set of paths to be tested to the test platform; wherein the test operation includes a sending operation of sending the set of paths to be tested to the test platform;

[0263] Or, a sending module, used to send the orchestration test case obtained based on the set of paths to be tested to the test platform; wherein the test operation includes a sending operation of sending the orchestration test case to the test platform;

[0264] The receiving module is used to receive the test results sent by the test platform; wherein the test results are obtained by the test platform based on a set of paths to be tested or an arrangement of test cases.

[0265] It should be noted that the process of information interaction between the units and modules in this embodiment can refer to the description in other embodiments and will not be repeated here.

[0266] The test device provided in the embodiment of the present application determines the system knowledge graph corresponding to the version to be tested based on the first system set information of the version to be tested of the target application system, obtains the second system basic information of the target historical version of the target application system, and then determines the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph, and finally performs the test operation based on the set of paths to be tested to obtain the test result. In this way, after determining the system knowledge graph of the version to be tested of the target application system and the second system basic information of the target historical version to be compared and analyzed, the first system basic information, the second system basic information and the system knowledge graph of the version to be tested are analyzed to determine the set of paths to be tested, thereby achieving accurate determination of the test scope, effectively narrowing the test scope, and testing the set of paths to be tested, thereby obtaining the test result of the version to be tested. In this way, the problem of low test efficiency of the current precise test scheme is solved, and a test method is proposed. When determining the test scope during the test process, the business development needs are fully considered, the accuracy of the determined test scope is guaranteed, and the test efficiency is improved.

[0267] Based on the above embodiments, the embodiments of the present application provide a network device, which can be applied to Figures 1-2 In the test method provided in the corresponding embodiment, refer to Figure 8 As shown, the network device 4 may include: a communication interface 41, a memory 42, a processor 43 and a communication bus 44; wherein:

[0268] Memory 42, used to store executable information;

[0269] A communication bus 44, used to realize communication connection between the communication interface 41, the processor 43 and the memory 42;

[0270] The processor 43 is used to execute the test program stored in the memory 42 to implement the following Figures 1-2 The implementation process of the test method provided in the corresponding embodiment will not be repeated here.

[0271] Based on the foregoing embodiments, the embodiments of the present application provide a computer-readable storage medium, referred to as a storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement reference Figures 1-2 The implementation process of the test method provided in the corresponding embodiment will not be repeated here.

[0272] Based on the foregoing embodiments, the embodiments of the present application further provide a computer program product, including a computer program, which can be executed by the processor 43 of the network device 4 to complete any of the foregoing method steps.

[0273] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0274] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0275] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0276] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0277] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A testing method, characterized in that: The method comprises: Based on the first system basic information of the version to be tested of the target application system, determining the system knowledge graph corresponding to the version to be tested; Acquire the second system basic information of the target historical version of the target application system; Based on the first system basic information, the second system basic information and the system knowledge graph, determine a set of paths to be tested of the version to be tested; wherein the set of paths to be tested includes m updated service paths, where m is an integer greater than or equal to 1; A test operation is performed based on the set of paths to be tested to obtain a test result.

2. The method according to claim 1, characterized in that The determining, based on the first system basic information of the version to be tested of the target application system, the system knowledge graph corresponding to the version to be tested includes: Determine first priority information of the priority order of each first business key parameter corresponding to each application scenario included in the first system basic information; Based on the product design flow chart included in the first system basic information, determine the first interface information, the second interface information and the second business key parameters corresponding to each business, and obtain the first product flow information; Determine first triplet information of the version to be tested based on the first priority information, the first product process information and the first system interface file information; wherein the first system interface file information belongs to the first system basic information; Based on the first triple information, the system knowledge graph is generated.

3. The method according to claim 2, characterized in that The first priority information of determining the priority order of each first business key parameter corresponding to each application scenario included in the first system basic information includes: Determine a priority weight coefficient of each of the first business key parameters corresponding to each application scenario; Based on the priority weight coefficients of all the first business key parameters included in the first system basic information, all the first business key parameters included in the first system basic information are sorted in order of the priority weight coefficients from large to small, and the priority order of each first business key parameter is obtained, and then the first priority information is obtained.

4. The method according to claim 3, characterized in that The determining of the priority weight coefficient of each of the first business key parameters corresponding to each application scenario includes: Determine a first weight coefficient and a second weight coefficient for each of the first business key parameters; wherein the sum of the first weight coefficient and the second weight coefficient is 1; Determine during the actual operation, each of the first business key parameters is the cumulative sum of the actual values ​​of each enumerated parameter to obtain a first sum value; Determine the cumulative sum of actual values ​​of all enumerated parameters of each of the first business key parameters during the actual operation to obtain a second sum value; Calculating a ratio of the first sum value of each of the first business key parameters to the corresponding second sum value; Calculating the product of the ratio of each of the first business key parameters and the corresponding first weight coefficient to obtain a first product; Calculating the product of the preset priority order of each of the first business key parameters and the corresponding second weight coefficient to obtain a second product; The sum of the first product of each first business key parameter and the corresponding second product is calculated to obtain a priority weight coefficient of each first business key parameter.

5. The method according to claim 2, characterized in that: The determining, based on the priority information, the first product process information and the first system interface file information, first triplet information of the version to be tested includes: Preprocessing the first priority information, the first product flow information and the first system interface file information respectively to obtain second priority information, second product flow information and second system interface file information in sequence; The second priority information, the second product process information and the second system interface file information are analyzed to obtain a target entity, a target relationship and a target attribute; wherein the first triple information includes the target entity, the target relationship and the target attribute.

6. The method according to claim 5, characterized in that The step of analyzing the second priority information, the second product process information, and the second system interface file information to obtain a target entity includes: Parse the second product process information, obtain the business name, business link, system business interface and third business key parameters, and obtain a reference entity; The objects included in the reference entity are unified using the second system interface file information to obtain the target entity.

7. The method according to claim 5, characterized in that The target attributes include at least one or more of the following information: business identification information, business link identification information, interface attribute information and business parameter attributes; wherein the interface attribute information includes at least one or more of the following information: interface identification information, interface parameter information, interface response information, upstream and downstream interface identification information, and the business parameter attributes include at least: the enumerated parameter value of the third business key parameter and the priority order of the third business key parameter.

8. The method according to claim 5, characterized in that The target relationship includes at least one or more of the following relationships: upstream and downstream dependency relationships of scenario links between multiple links, ownership relationships between services and links, ownership relationships between links and interfaces, and influence relationships between interfaces and third-party business key parameters.

9. The method according to claim 1, characterized in that: The determining, based on the first system basic information, the second system basic information and the system knowledge graph, a set of paths to be tested of the version to be tested includes: Determine first triplet information of the version to be tested based on the first system basic information; Determine second triplet information of the target historical version based on the second system basic information; The set of paths to be tested is determined based on the first triplet information, the second triplet information and the system knowledge graph.

10. The method according to claim 9, characterized in that The determining the set of paths to be tested based on the first triplet information, the second triplet information and the system knowledge graph includes: Performing update analysis on the first triplet information and the second triplet information to obtain an analysis result; One or more complete links associated with the update objects included in the analysis result are determined from the system knowledge graph to obtain the set of paths to be tested.

11. The method according to claim 10, characterized in that The analysis result includes at least one or more of the following information: one or more updated entities, one or more updated relations, and one or more updated attributes.

12. The method according to claim 1, characterized in that The performing a test operation based on the set of paths to be tested to obtain a test result includes: Determining the path priority of each test path in the set of paths to be tested; Arrange the paths in the set of paths to be tested based on the path priority of each of the test paths to obtain an arranged test case; The arrangement test case is tested to obtain the test result.

13. The method according to claim 12, characterized in that The determining the path priority of each test path in the set of paths to be tested includes: Determine the priority weight coefficient of the fourth business critical parameter of each of the test paths in the set of paths to be tested; The priority weight coefficients of the fourth business critical parameters of each of the test paths are accumulated to obtain the path priority of the corresponding test path.

14. The method according to claim 1, characterized in that The performing a test operation based on the set of paths to be tested to obtain a test result includes: Sending the set of paths to be tested to a test platform; wherein the test operation includes sending the set of paths to be tested, Or, sending an orchestration test case obtained based on the set of paths to be tested to a test platform; wherein the test operation includes a sending operation of sending the orchestration test case to the test platform; Receive the test result sent by the test platform; wherein the test result is obtained by the test platform based on the set of paths to be tested or the arranged test case.

15. A testing device, characterized in that: The device comprises: a first determining unit, an acquiring unit, a second determining unit and an executing unit; wherein: The first determining unit is used to determine the system knowledge graph corresponding to the version to be tested based on the first system basic information of the version to be tested of the target application system; The acquisition unit is used to acquire the second system basic information of the target historical version of the target application system; The second determination unit is used to determine the set of paths to be tested of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph; wherein the set of paths to be tested includes m updated service paths, where m is an integer greater than or equal to 1; The execution unit is used to execute a test operation based on the set of paths to be tested to obtain a test result.

16. A network device, characterized in that: The device comprises: a communication interface, a memory, a processor and a communication bus; wherein: The memory is used to store executable instructions; The communication bus is used to realize the communication connection between the communication interface, the processor and the memory; The processor is used to execute the test program stored in the memory to implement the steps of the test method according to any one of claims 1 to 14.

17. A storage medium, characterized in that: The storage medium stores a test program, which is used to implement the steps of the test method according to any one of claims 1 to 14 when executed.

18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the testing method according to any one of claims 1 to 14.

Images