A test method, apparatus, device, storage medium, and computer program product

Abstract

The application discloses a test method, which is applied to a network device, and the method comprises the following steps: determining a system knowledge graph corresponding to a to-be-tested version of a target application system based on first system basic information of the to-be-tested version; acquiring second system basic information of a target historical version of the target application system; determining a to-be-tested path set of the to-be-tested version based on the first system basic information, the second system basic information and the system knowledge graph; wherein the to-be-tested path set comprises m updated service paths, and m is an integer greater than or equal to 1; and performing a test operation based on the to-be-tested path set to obtain a test result. The application also discloses a test device, equipment, a storage medium and a computer program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, and in particular to a test method, device, equipment, storage medium and computer program product. BACKGROUND

[0002] With the rapid development of wireless communication technology, the application of wireless communication technology is also more and more widely. In the application process of wireless communication technology, in order to ensure the user experience effect, the deployment of wireless communication technology scheme is also updated and iterated faster. In order to ensure the feasibility of the wireless communication technology scheme, it is usually necessary to test and put into application after passing the test. However, for example, when deploying a network slice management function (NSMF) system, different types of businesses constitute a variety of business scenarios, and different business processes are opened for different business scenarios. At this time, a large amount of test branches will be formed, and the test workload is large when performing scene testing and iterative regression testing, and the iterative test range evaluation is difficult when the parameters change. At present, in order to reduce the problem of difficult iterative test range evaluation, the commonly used accurate test scheme is to evaluate the version change influence range after changing the research and development code range, and then select the corresponding automatic test case to complete the version iteration test.

[0003] However, the current accurate test scheme mainly determines the version change influence range by analyzing the research and development code when determining the version change influence range. When the research and development code has a problem, the selected version change influence range is inaccurate, which further leads to inaccurate test results, and seriously affects the test efficiency of the test.

[0004] APPLICATION CONTENT

[0005] To solve the above technical problems, the present application expects to provide a test method, device, equipment, storage medium and computer program product, which solves the problem of low test efficiency of the current accurate test scheme, and proposes a test method. In the test process, when determining the test range, the business development demand is fully considered, the accuracy of the determined test range is guaranteed, and the test efficiency is improved.

[0006] The technical scheme of the present application is realized as follows:

[0007] The present application provides a test method, which comprises:

[0008] Based on the first system basic information of the target application system to be tested version, the system knowledge graph corresponding to the to-be-tested version is determined;

[0009] Obtain the second system basic information of the target historical version of the target application system;

[0010] determine a set of to-be-tested paths of the to-be-tested version based on the first system basic information, the second system basic information, and the system knowledge graph, wherein the set of to-be-tested paths comprises m updated service paths, and m is an integer greater than or equal to 1;

[0011] perform a test operation based on the set of to-be-tested paths to obtain a test result.

[0012] In the above scheme, the first system basic information of the target application system to-be-tested version is used to determine the system knowledge graph corresponding to the to-be-tested version, comprising:

[0013] determine, from the first system basic information, first priority information of a priority order of each first business critical parameter corresponding to each application scenario included in the first system basic information;

[0014] determine, based on a product design flowchart included in the first system basic information, first interface information, second interface information, and a second business critical parameter corresponding to each service, to obtain first product flow information;

[0015] determine first triple information of the to-be-tested version based on the first priority information, the first product flow information, and first system interface file information, wherein the first system interface file information belongs to the first system basic information;

[0016] generate the system knowledge graph based on the first triple information.

[0017] In the above scheme, the first priority information of a priority order of each first business critical parameter corresponding to each application scenario included in the first system basic information is determined, comprising:

[0018] determine a priority weight coefficient of each first business critical parameter corresponding to each application scenario;

[0019] sort all first business critical parameters included in the first system basic information in descending order of priority weight coefficients based on the priority weight coefficients of all first business critical parameters included in the first system basic information, to obtain a priority order of each first business critical parameter, and further obtain the first priority information.

[0020] In the above scheme, the priority weight coefficient of each first business critical parameter corresponding to each application scenario is determined, comprising:

[0021] determine a first weight coefficient and a second weight coefficient of each first business critical parameter, wherein the sum of the first weight coefficient and the second weight coefficient is 1.

[0022] determining an accumulated sum of actual values of each of the first business key parameters when each of the enumerated parameters is in actual operation, to obtain a first sum value;

[0023] determining an accumulated sum of actual values of all enumerated parameters of each of the first business key parameters in actual operation, to obtain a second sum value;

[0024] calculating a ratio of the first sum value of each of the first business key parameters to the corresponding second sum value;

[0025] calculating a product of the ratio of each of the first business key parameters and the corresponding first weight coefficient, to obtain a first product;

[0026] calculating a product of a preset priority order of each of the first business key parameters and the corresponding second weight coefficient, to obtain a second product;

[0027] calculating a sum value of the first product of each of the first business key parameters and the corresponding second product, to obtain a priority weight coefficient of each of the first business key parameters.

[0028] In the above scheme, the first triplet information of the to-be-tested version is determined based on the priority information, the first product flow information and the first system interface file information, and includes:

[0029] The first priority information, the first product flow information and the first system interface file information are preprocessed respectively, to obtain second priority information, second product flow information and second system interface file information in sequence.

[0030] The second priority information, the second product flow information and the second system interface file information are analyzed to obtain a target entity, a target relationship and a target attribute; wherein the first triplet information includes the target entity, the target relationship and the target attribute.

[0031] In the above scheme, the second priority information, the second product flow information and the second system interface file information are analyzed to obtain a target entity, and include:

[0032] The second product flow information is parsed to obtain a business name, a business link, a system business interface and a third business key parameter, to obtain a reference entity;

[0033] The second system interface file information is used to uniformly process objects included in the reference entity, to obtain the target entity.

[0034] The target attribute includes one or more of the following information: service identification information, service link identification information, interface attribute information, and service parameter attribute.

[0035] The target relationship includes one or more of the following relationships: upstream and downstream dependency relationship between multiple links on a scene link, belonging relationship between a service and a link, belonging relationship between a link and an interface, and influence relationship between an interface and a third service key parameter.

[0036] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0037] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0038] The second system basic information includes one or more of the following information: a second service identification, a second service link identification, a second interface attribute, and a second service parameter attribute.

[0039] The second system basic information includes one or more of the following information: a second service identification, a second service link identification, a second interface attribute, and a second service parameter attribute.

[0040] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0041] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0042] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0043] The analysis result includes one or more of the following information: one or more updated entities, one or more updated relationships, and one or more updated attributes.

[0044] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0045] The first system basic information includes one or more of the following information: a first service identification, a first service link identification, a first interface attribute, and a first service parameter attribute.

[0046] arrange paths in the set of to-be-tested paths based on a path priority of each of the test paths, to obtain an arranged test case;

[0047] test the arranged test case to obtain the test result.

[0048] In the foregoing solution, the determination of the path priority of each test path in the set of to-be-tested paths comprises:

[0049] determining a priority weight coefficient of a fourth service key parameter of each of the test paths in the set of to-be-tested paths;

[0050] adding the priority weight coefficient of the fourth service key parameter of each of the test paths to obtain the path priority of the corresponding test path.

[0051] In the foregoing solution, the execution of the test operation based on the set of to-be-tested paths to obtain a test result comprises:

[0052] sending the set of to-be-tested paths to a test platform; wherein the test operation comprises sending the set of to-be-tested paths,

[0053] or, sending an arranged test case obtained based on the set of to-be-tested paths to a test platform; wherein the test operation comprises a sending operation of sending the arranged test case to the test platform;

[0054] receiving the test result sent by the test platform; wherein the test result is obtained by the test platform based on the set of to-be-tested paths or the arranged test case.

[0055] The application provides a test device, which comprises a first determination unit, an acquisition unit, a second determination unit, and an execution unit; wherein:

[0056] The first determination unit is configured to determine a system knowledge graph corresponding to a to-be-tested version of a target application system based on first system basic information of the to-be-tested version.

[0057] The acquisition unit is configured to acquire second system basic information of a target historical version of the target application system.

[0058] The second determination unit is configured to determine a set of to-be-tested paths of the to-be-tested version based on the first system basic information, the second system basic information, and the system knowledge graph; wherein the set of to-be-tested paths comprises m updated service paths, and m is an integer greater than or equal to 1.

[0059] The execution unit is configured to execute a test operation based on the set of to-be-tested paths to obtain a test result.

[0060] The application provides a network device, the device comprising: a communication interface, a memory, a processor and a communication bus; wherein:

[0061] The memory is configured to store executable instructions.

[0062] The communication bus is configured to realize the communication connection between the communication interface, the processor and the memory.

[0063] The processor is configured to execute the test program stored in the memory to realize the steps of the test method according to any one of the preceding embodiments.

[0064] The application provides a storage medium, the storage medium storing a test program, the test program being executed to realize the steps of the test method according to any one of the preceding embodiments.

[0065] The application provides a computer program product comprising a computer program, the computer program being executed by a processor to realize the steps of the test method according to any one of the preceding embodiments.

[0066] The application embodiment provides a test method, device, equipment, system, storage medium and computer program product. After the system knowledge graph of the to-be-tested version of the target application system is determined based on the first system set information of the to-be-tested version, the second system basic information of the target historical version of the target application system is obtained, and then the to-be-tested path set of the to-be-tested version is determined based on the first system basic information, the second system basic information and the system knowledge graph, finally, the test operation is performed based on the to-be-tested path set to obtain the test result. In this way, after the system knowledge graph of the to-be-tested version of the target application system and the second system basic information of the target historical version to be compared and analyzed are determined, the first system basic information, the second system basic information and the system knowledge graph of the to-be-tested version are analyzed to determine the to-be-tested path set, the accurate test range is determined, the test range is effectively reduced, and the to-be-tested path set is tested, and then the test result of the to-be-tested version is obtained. In this way, the problem of low test efficiency of the current accurate test scheme is solved, and a test method is proposed. In the test process, the business development demand is fully considered when the test range is determined, the accuracy of the determined test range is ensured, and the test efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 Flowchart of the test method provided by the application embodiment Figure 1 ;

[0068] Figure 2 Flowchart of the test method provided by the application embodimentFigure 2 ;

[0069] Figure 3 An application example of the test method provided in the embodiments of the present application realizes the flowchart as shown in the figure;

[0070] Figure 4 The flowchart of generating the NSFM system knowledge graph provided in the embodiments of the present application is as shown in the figure;

[0071] Figure 5 The structure diagram of the NSFM system graph database model provided in the embodiments of the present application is as shown in the figure;

[0072] Figure 6 The implementation flowchart of determining the set of to-be-tested paths provided in the embodiments of the present application is as shown in the figure;

[0073] Figure 7 The structure diagram of the test device provided in the embodiments of the present application is as shown in the figure;

[0074] Figure 8 The structure diagram of the network device provided in the embodiments of the present application is as shown in the figure. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0076] The embodiments of the present application provide a test method, referring to Figure 1 The method is applied to a network device, and the method comprises the following steps:

[0077] Step 101, determining a system knowledge graph corresponding to a to-be-tested version of a target application system based on first system basic information of the to-be-tested version.

[0078] In the embodiments of the present application, the target application system can be a service system providing communication network services, for example, can be an NSFM system. In the development and application process of the target application system, with the increase of application requirements, and the perfection and richness of functions, the developers will continuously update and iterate the target application system. When the test is passed at a certain rate, the corresponding updated version will be released, therefore, in the test process, the network device determines the to-be-tested version of the target application system that needs to be tested, and then obtains the related information of the to-be-tested version, for example, the product design flowchart of the to-be-tested version, the related graphic and text description information in the design process, the development file, etc., to obtain the first system basic information. The first system basic information is analyzed and processed to obtain the system knowledge graph corresponding to the to-be-tested version.

[0079] Step 102, acquiring second system basic information of a target historical version of the target application system.

[0080] In the embodiment of the present application, the target historical version is a historical version that is expected to be compared and analyzed with the version to be tested to determine the update range, and is usually a historical version that has passed the test. After determining the target historical version of the target application system, the second system basic information of the target historical version is acquired.

[0081] Step 103, determining a to-be-tested path set of the version to be tested based on the first system basic information, the second system basic information and the system knowledge graph.

[0082] The to-be-tested path set includes m updated business paths, and m is an integer greater than or equal to 1.

[0083] In the embodiment of the present application, the first system basic information and the second system basic information are analyzed to obtain an analysis result, the analysis result is used to determine and analyze from the system knowledge graph to obtain m updated business paths in the version to be tested, and the m updated business paths are stored as the to-be-tested path set.

[0084] Step 104, performing a test operation based on the to-be-tested path set to obtain a test result.

[0085] In the embodiment of the present application, according to the to-be-tested path set, a corresponding test case is used to perform a test operation to obtain a test result. The corresponding test operation can be performed by the network device itself, or the network device can send a test task for the to-be-tested path set to a test platform, and the test platform can implement the test operation. The specific implementation can be determined according to the actual application scenario, which is not limited here.

[0086] The test method provided in the embodiments of the present application determines the system knowledge graph corresponding to the to-be-tested version based on the first system set information of the to-be-tested version of the target application system, acquires the second system basic information of the target historical version of the target application system, then determines the to-be-tested path set of the to-be-tested version based on the first system basic information, the second system basic information and the system knowledge graph, and finally performs a test operation based on the to-be-tested path set to obtain a test result. In this way, after the system knowledge graph of the to-be-tested version of the target application system and the second system basic information of the target historical version to be compared and analyzed are determined, the first system basic information, the second system basic information and the system knowledge graph of the to-be-tested version are analyzed to determine the to-be-tested path set, the accurate test range is determined, the test range is effectively reduced, the to-be-tested path set is tested and processed, and then the test result of the to-be-tested version is obtained. In this way, the problem of low test efficiency of the current accurate test scheme is solved, and a test method is provided. In the determination of the test range in the test process, the business development demand is fully considered, the accuracy of the determined test range is ensured, and the test efficiency is improved.

[0087] Based on the foregoing embodiments, the embodiments of the present application provide a test method, which is applied to a network device, and with reference to the method shown in Figure 2 The method comprises the following steps:

[0088] Step 201, determine the first priority information of the priority order of each first business key parameter corresponding to each application scenario included in the first system basic information.

[0089] In the embodiments of the present application, the first system basic information is analyzed to determine all the application scenarios set in the first system basic information, then the first business key parameters used in each application scenario are determined, the priority order of each first business key parameter is calculated, and thus the first priority information corresponding to each application scenario included in the first system basic information is obtained.

[0090] Step 202, determine the first interface information, the second interface information and the second business key parameter corresponding to each business based on the product design flowchart included in the first system basic information to obtain the first product flow information.

[0091] In the embodiment of the present application, the product design flowchart is a development flowchart drawn in the development and design process of a product, and at least includes a link of a business, interface corresponding to the link, and business key parameters of the interface and the like. All product design flowcharts included in the first system basic information are acquired, the first interface information, the second interface information and the first business key parameters are determined from the product design flowchart, so that the corresponding product flow information, i.e., the first product flow information, is obtained. The first interface information can be interface information of an internal interface included in the product design flowchart, and the second interface information can be interface information of an external interface included in the product design flowchart. The external interface is an interface for communication between the target application system and an external communication network, such as the Internet, a communication terminal and the like.

[0092] In step 203, the first triad information of the to-be-tested version is determined based on the first priority information, the first product flow information and the first system interface file information.

[0093] The first system interface file information belongs to the first system basic information.

[0094] In the embodiment of the present application, the first system interface file information is file information for defining a system interface in the to-be-tested version of the target application system. The network device analyzes the first priority information, the first product flow information and the first system interface file information, and extracts all triad information included in the to-be-tested version, which is recorded as the first triad information.

[0095] In step 204, the system knowledge graph is generated based on the first triad information.

[0096] In the embodiment of the present application, the first triad information determined is subjected to analysis and processing operations such as logical analysis and logical reasoning, so that the association relationship between entities included in the to-be-tested version is determined, the association relationship including hidden relationships and explicit relationships between the entities, so that the system knowledge graph is extended.

[0097] In step 205, the second system basic information of a target historical version of the target application system is acquired.

[0098] In the embodiment of the present application, the target historical version of the target application system can be any one version of the target application system that has passed testing. After the target historical version of the target application system that needs to be compared and analyzed with the to-be-tested version is determined, the corresponding second system basic information is acquired.

[0099] In step 206, the to-be-tested path set of the to-be-tested version is determined based on the first system basic information, the second system basic information and the system knowledge graph.

[0100] The to-be-tested path set includes m updated service paths, where m is an integer greater than or equal to 1.

[0101] In the embodiments of the present application, the first system basic information and the second system basic information are analyzed to determine the updated content in the first system basic information, the complete path including the updated content is determined from the system knowledge graph, m updated service paths are obtained, and thus the to-be-tested path set of the to-be-tested version is obtained. In some application scenarios, the second system basic information can also be used to generate a system knowledge graph corresponding to the second system basic information, and then the system knowledge graph corresponding to the second system basic information is compared and analyzed with the system knowledge graph of the first system basic information to determine the path in the first system basic information that has been updated, and thus the to-be-tested path set of the to-be-tested version is obtained. However, when the system knowledge graph corresponding to the second system basic information is generated, some information can be removed, and therefore, the second system basic information can be compared and analyzed with the first system basic information to reduce the leakage of updated information, and thus a more comprehensive and accurate to-be-tested path set can be obtained.

[0102] In step 207, a test operation is performed based on the to-be-tested path set, and a test result is obtained.

[0103] Based on the foregoing embodiments, in other embodiments of the present application, step 201 can be implemented by steps 201a and 201b.

[0104] In step 201a, the priority weight coefficient of each first service key parameter corresponding to each application scenario is determined.

[0105] In the embodiments of the present application, the priority weight coefficient of each first service key parameter included in each application scenario can be a pre-set empirical value, or can be calculated according to an actual application by using an empirical calculation formula.

[0106] In step 201b, based on the priority weight coefficients of all first service key parameters included in the first system basic information, all first service key parameters included in the first system basic information are sorted in descending order of priority weight coefficients to obtain the priority order of each first service key parameter, and thus the first priority information is obtained.

[0107] In the embodiment of the present application, after the priority weight coefficients of all the first service key parameters included in the first system basic information are determined, the priority weight coefficients of all the first service key parameters are sorted according to the size relationship to obtain a sorting result of the priority weight coefficients, the corresponding first service key parameters are sorted according to the sorting result of the priority weight coefficients, and the sorting order of each first service key parameter is determined as the priority order thereof. In this way, the first priority information corresponding to the first system basic information can be determined.

[0108] Based on the foregoing embodiment, in other embodiments of the present application, step 201a can be implemented by steps a11-a17.

[0109] Step a11, determining the first weight coefficient and the second weight coefficient of each first service key parameter.

[0110] The sum of the first weight coefficient and the second weight coefficient is 1.

[0111] In the embodiment of the present application, the second weight coefficient is an empirical value pre-set for each first service key parameter. In actual situations, the second weight coefficient can be adjusted according to actual requirements. Since the sum of the first weight coefficient and the second weight coefficient is 1, the first weight coefficient can be calculated by the formula “first weight coefficient = 1-second weight coefficient” on the premise that the second weight coefficient is determined.

[0112] Step a12, determining the cumulative sum of the actual value of each enumerated parameter of each first service key parameter in the actual running process to obtain a first sum value.

[0113] In the embodiment of the present application, the actual running of the to-be-tested version is performed to obtain the actual value of each enumerated parameter of each first service key parameter in the actual running process of the to-be-tested version, and then the actual value of the first service key parameter is accumulated and calculated to obtain the first sum value. It should be noted that at least one enumerated parameter can be set for each first service key parameter in actual setting.

[0114] Step a13, determining the cumulative sum of the actual value of all enumerated parameters of each first service key parameter in the actual running process to obtain a second sum value.

[0115] In the embodiment of the present application, the actual value of all enumerated parameters of each first service key parameter is accumulated and calculated to obtain the second sum value.

[0116] Step a14, calculating the ratio of the first sum value of each first service key parameter to the corresponding second sum value.

[0117] Step a15, calculating the product of the ratio of each first service key parameter and the corresponding first weight coefficient, to obtain a first product.

[0118] Step a16, calculating the product of the preset priority order of each first service key parameter and the corresponding second weight coefficient, to obtain a second product.

[0119] Step a17, calculating the sum of the first product and the corresponding second product of each first service key parameter, to obtain the priority weight coefficient of each first service key parameter.

[0120] Based on the foregoing embodiments, in other embodiments of the present application, step 203 can be implemented by steps 203a-203b:

[0121] Step 203a, respectively pre-processing the first priority information, the first product process information and the first system interface file information, to obtain the second priority information, the second product process information and the second system interface file information in sequence.

[0122] In the embodiments of the present application, the preprocessing can include one or more combined processing methods such as information cleaning operation, normalization and unification processing, and text information segmentation. The information cleaning operation can include deleting irrelevant content, such as deleting irrelevant symbols, etc. The first priority information is pre-processed to obtain the second priority information, the first product process information is pre-processed to obtain the second product process information, and the first system interface file information is pre-processed to obtain the second system interface file information.

[0123] Step 203b, analyzing the second priority information, the second product process information and the second system interface file information to obtain the target entity, the target relationship and the target attribute.

[0124] Among them, the first triple information includes the target entity, the target relationship and the target attribute.

[0125] In the embodiments of the present application, the target entity is obtained by extracting the entity object from the second priority information, the second product process information and the second system interface file information. After the target entity is determined, various relationship analyses of the entity object are performed on the second priority information, the second product process information and the second system interface file information to obtain the corresponding target relationship. Further, the second priority information, the second product process information and the second system interface file information are further processed to obtain the target attribute related to the target entity.

[0126] Based on the foregoing embodiments, in other embodiments of the present application, step 203b can be implemented by steps b11-b12:

[0127] Step b11, parsing the second product process information to obtain the business name, business link, system business interface and third business key parameter, and obtaining a reference entity.

[0128] In the embodiment of the application, the business name, the business link, the system business interface and the third business key parameter are obtained from the second product process information, and these parameters are taken as entity objects to obtain the reference entity.

[0129] Step b12, uniformly processing the objects included in the reference entity by using the second system interface file information to obtain a target entity.

[0130] In the embodiment of the application, the reference entity obtained from the second product process information is supplemented and uniformly processed according to the second system interface file information, so as to realize the equivalent processing of the objects in the second system interface file information and the objects in the second product process information which are essentially the same entity objects but have different naming methods, and thus the reference entity is perfectly processed to obtain the target entity.

[0131] Based on the foregoing embodiment, in other embodiments of the application, the target attribute at least includes one or more of the following information: business identification information, business link identification information, interface attribute information and business parameter attribute; wherein the interface attribute information at least includes one or more of the following information: interface identification information, interface parameter information, interface response information, upstream and downstream interface identification information, and the business parameter attribute at least includes an enumeration parameter value of the third business key parameter and a priority order of the third business key parameter.

[0132] In the embodiment of the application, the business identification information is identification information for uniquely identifying a business, which can be a business name, a business number or the like, and the business link identification information is identification information for uniquely identifying a business link, which can be a business link name, a business link number or the like. The priority order of the third business key parameter can be determined by the second priority information.

[0133] Based on the foregoing embodiment, in other embodiments of the application, the target relationship at least includes one or more of the following relationships: a scene link upstream and downstream dependency relationship between a plurality of links, a belonging relationship between a business and a link, a belonging relationship between a link and an interface, and an influence relationship between an interface and a third business key parameter.

[0134] Based on the foregoing embodiment, in other embodiments of the application, step 206 can be step 206a-206c:

[0135] Step 206a, determining the first triple information of the to-be-tested version based on the first system basic information.

[0136] Step 206b, determining the second triple information of the target historical version based on the second system basic information.

[0137] Step 206c, determining the set of to-be-tested paths based on the first triple information, the second triple information and the system knowledge graph.

[0138] In the embodiment of the present application, the first triple information and the second triple information are compared and analyzed to obtain an analysis result, and the set of to-be-tested paths is determined from the system knowledge graph according to the analysis result.

[0139] Based on the foregoing embodiment, in other embodiments of the present application, step 206c can be implemented by steps c11-c12:

[0140] Step c11, performing update analysis on the first triple information and the second triple information to obtain an analysis result.

[0141] In the embodiment of the present application, the first triple information and the second triple information are compared and analyzed to obtain an analysis result, and the set of to-be-tested paths is determined from the system knowledge graph according to the analysis result.

[0142] Step c12, determining one or more complete links having a correlation relationship with the update object included in the analysis result from the system knowledge graph to obtain the set of to-be-tested paths.

[0143] The analysis result includes one or more of the following information: one or more update entities, one or more update relationships, and one or more update attributes.

[0144] In the embodiment of the present application, the analysis result is used to analyze the system knowledge graph, and the links in the system knowledge graph that only include the update object in the analysis result or are affected by the update object are determined to obtain the set of to-be-tested paths. The complete link refers to the link from the beginning to the end.

[0145] Based on the foregoing embodiment, in other embodiments of the present application, step 207 can be implemented by steps 207a-207c:

[0146] Step 207a, determining the path priority of each test path in the set of to-be-tested paths.

[0147] In the embodiments of the present application, the path priority of each test path can be determined according to the priority of the path nodes in each test path. For example, the path priority of each test path can be obtained by adding the priority order weight coefficients of all path nodes in each test path, or by averaging the priority order weight coefficients of all path nodes in each test path, or by weighting and averaging the priority order weight coefficients of all path nodes in each test path. The path nodes can be all the service parameters determined, including service key parameters. The service key parameters are parameters that affect the service path.

[0148] In step 207b, the paths in the set of test paths are arranged based on the path priority of each test path, to obtain an arranged test case.

[0149] In the embodiments of the present application, the paths in the set of test paths are arranged and combined based on the path priority of each test path according to application requirements, to obtain different arranged test cases that need to be tested.

[0150] In step 207c, the arranged test case is tested to obtain a test result.

[0151] In the embodiments of the present application, the arranged test case is tested to obtain a test result.

[0152] Based on the foregoing embodiments, in other embodiments of the present application, step 207a can be implemented by steps d11-d12:

[0153] In step d11, the priority weight coefficient of the fourth service key parameter of each test path in the set of test paths is determined.

[0154] In the embodiments of the present application, the priority weight coefficient of the fourth service key parameter of each test path is determined by the second priority information.

[0155] In step d12, the priority weight coefficients of the fourth service key parameters of each test path are added to obtain the path priority of the corresponding test path.

[0156] Based on the foregoing embodiments, in other embodiments of the present application, step 207 can also be implemented by steps 207d and 207f, or steps 207e-207f:

[0157] In step 207d, the set of test paths is sent to a test platform.

[0158] The test operation includes the sending operation of sending the set of test paths to the test platform.

[0159] In the embodiment of the present application, the network device can directly send the to-be-tested path set to the test platform, so that the test platform generates the arranged test case based on the to-be-tested path set, tests the arranged test case, and obtains the test result. The process of generating the arranged test case based on the to-be-tested path set by the test platform is the same as the process of steps 207a-207b performed by the network device, which will not be described in detail here.

[0160] Step 207e, sending the arranged test case based on the to-be-tested path set to the test platform.

[0161] The test operation includes a sending operation of sending the arranged test case to the test platform.

[0162] In the embodiment of the present application, the process of the network device based on the arranged test case obtained from the to-be-tested path set can refer to the implementation process of steps 207a-207b, which will not be described in detail here.

[0163] Step 207f, receiving the test result sent by the test platform.

[0164] The test result is obtained by the test platform based on the to-be-tested path set or the arranged test case.

[0165] Based on the foregoing embodiment, there are three types of services in the NSMF system: premium services, exclusive services, and VIP services, which correspond to three types of slices: default slice type, dedicated slice type, and general slice type. Further, each type of slice can be divided into shared type and non-shared type. Among them, customers of different service types can share slices, so different types of services can be extended to multiple service scenarios, which can be counted as six service scenarios, which are: old default slice + new dedicated data network name (DNN), old general slice + new dedicated DNN, original default slice + new dedicated DNN, original general slice + new dedicated DNN, original dedicated slice + new dedicated DNN, and new dedicated slice + new dedicated DNN. Under the six service scenarios, because different services have different processes for opening and changing services, there are order interface service parameters of a primary service arrangement center and a secondary service arrangement center. Based on this, the embodiment of the present application provides a test method for a target application system of an NSMF system, as shown in Figure 3 The implementation steps include the following:

[0166] Step e11, obtaining service knowledge of a to-be-tested version of the NSMF system.

[0167] The business knowledge includes: business priority, product process template, interface document information. The business priority corresponds to the first priority information, the product process module corresponds to the first product process information, and the interface document information corresponds to the first system interface file information.

[0168] The specific process of determining the business priority can be as follows: the NSMF system marks the scenario priority based on the work order of the live production environment. Because the work order of the live production environment can reflect the business demands of the customer, the work order information of the live production environment can be read through the service interface, and then the business key parameters associated with the scenario can be obtained through the work order query interface based on the work order information, and the priority of the obtained business key parameters is calculated. The priority of the business key parameters can be calculated using the following formula:

[0169] In the formula, a is the weight value of the priority statistics of the business key parameters in the live production environment work order information, β is the preset priority weight value of the business key parameters, the sum of a and β is 1, Pre(p nk )≤1 is the preset priority of the business key parameter; p nk represents the kth enumeration value of the nth parameter in a certain business scenario of the NSMF system, where n=1, 2, …, N, k=1, 2, …, K n , N represents the total number of key business parameters, the nth key business parameter, and a total of K n values; f(p nk ) represents the final priority coefficient of the parameter p nk . f mn represents the relationship between the nth parameter of the mth work order and a certain application scenario. If f mn =1, it indicates that the nth parameter of the mth work order is related to the certain application scenario, and if f mn =0, it indicates that the nth parameter of the mth work order is not related to the certain application scenario. M represents the total number of work orders that need to be prioritized, represents the kth enumeration value of the key business parameter n in the mth work order.

[0170] The specific definition process of the product flow module can be as follows: for example, when the NSMF system includes opening an investigation, opening, changing an investigation, changing, closing, and other businesses, each business will design a product flowchart at the initial stage of product design. The product flowchart includes information such as the links of the business, the interfaces corresponding to the links, and the key business parameters of the interfaces. Therefore, the product flowchart is parsed to obtain a product flow template, which can be stored in the form of JS key-value pair data (JavaScript Object Notation, JSON), another markup language (YAML Ain't Markup Language, YAML) file, and the like, so as to facilitate subsequent information extraction. For example, the product flowchart is parsed to obtain information such as the internal interfaces, external interfaces, and key business parameter value information of each link corresponding to each business in the product flowchart. In this implementation process, when the product flowchart is parsed, the product flow template can be generated by recognizing the key information involved in the product flowchart and then implementing code parsing. Correspondingly, a product flow template can be recorded in the following way:

[0171]

[0172]

[0173] It should be noted that the key parameters in the above description are the same as the aforementioned business key parameters, which are interface parameters that affect the branching of the business flow scenario in the business flow.

[0174] The interface document information at least includes internal interface documents and external interface documents. The internal interfaces recorded in the internal interface documents are usually interfaces defined by the designers of the NSMF system software for internal calls of various businesses, and the external interfaces defined in the external interface documents are standard interfaces designed by the designers to enable the NSMF system to interact with surrounding systems. The interface document information at least includes key information such as interface name, interface uniform resource locator (Uniform Resource Locator, URL), interface parameter, interface response, and upstream and downstream interface identifier.

[0175] Step e12, based on business knowledge, extracting triples of the NSMF system including entities, attributes, and relationships.

[0176] The specific process of triple extraction is mainly achieved through text analysis and information extraction, and specifically includes the following steps:

[0177] Step e121, document preprocessing.

[0178] The preprocessed document includes at least business knowledge: a priority table recording business priorities, a product process template, and interface document information. Document preprocessing mainly includes text cleaning of business knowledge to remove irrelevant content such as irrelevant symbols in the text document, and corresponding processing of the document content, such as word segmentation of the document content, and part-of-speech tagging of the word segmentation in the document to facilitate subsequent extraction of entities, relationships, and attributes. In the analysis and processing of the cleaned document content, a word segmentation tool can be used to complete the analysis and processing.

[0179] Step e122, entity extraction.

[0180] An entity is a specific object or concept, such as a person, a place, an organization, etc. In this application, the entity is specifically a business, a link, an interface, and a key business parameter.

[0181] Step e123, relationship extraction.

[0182] A relationship is used to represent the connection or interaction between entities. In the embodiments of the present application, the relationship includes at least the upstream and downstream dependency relationship of the scene link between multiple links, the belonging relationship between the business and the link, the belonging relationship between the link and the interface, and the influence relationship between the interface and the key business parameter.

[0183] Step e124, attribute extraction.

[0184] An attribute is descriptive information for an entity. In the embodiments of the present application, the attribute includes at least the attribute business identifier of the business entity, the attribute link identifier of the link entity, and the attribute interface identifier, interface parameter, interface response, and upstream and downstream interface identifier of the interface entity. The enumeration value of the key business parameter and the priority weight coefficient of the key business parameter also belong to the attributes of the corresponding entity.

[0185] It should be noted that in the process of extracting entities, relationships, and attributes, the extraction method can be a rule-based method, a machine learning-based method, and / or a supervised, semi-supervised, or unsupervised learning method. For example, the documents of the business knowledge base of the NSMF system are usually structured data, so a rule-based method can be used to extract the triplets of entities, attributes, and relationships, i.e., a computer extraction application can be used to parse the product process template and interface document information, and then a defined regular expression can be used to match the key information. In some implementation schemes, the priority table, product process template, and interface document information can also be uploaded to a multi-modal large model, and the triplets can be extracted by prompting the multi-modal large model.

[0186] Step e125, complete the entity alignment in the product flow template and interface document information.

[0187] In the embodiment of the present application, the entity includes a business, a link, an interface, and a key business parameter. The link, the interface, and the key business parameter are defined in the product flow template and the interface document information. The interface document information further includes internal interface documents and external interface documents. Since the product flow template, the internal interface documents, and the external interface documents are respectively from product personnel, design personnel, and a solution, there are different definitions of the same entity due to different starting points of different system roles. At this time, the internal interface documents and the external interface documents can be used to merge and align the entities extracted in step e122.

[0188] For example, taking the NSMF system opening of the exploration business as an example, the key business parameter of the first link order issuing interface is defined as usage = 2 in the external interface document, that is, a new special slice is added / old general slice + a new special DNN. In the product flow template, it is defined as usage = 2, that is, old general slice + new special DNN, and usage = 6, that is, new special slice + new special DNN. In this way, the meanings of the usage of the key business parameter of the first link order issuing interface in the external interface document and the product flow template can be determined to be the same by analyzing, determining, and calculating the equivalence relationship, similarity, and features, and the usage parameter belongs to the same entity. The entity attribute values of the usage parameter are merged, for example, usage = 2 and usage = 6.

[0189] In this way, the priority weight coefficient of the key business parameter is obtained by analyzing the business knowledge, and the priority table is obtained. Then, based on the business knowledge, the entities of the business, the link, the interface, and the key business parameter are extracted, the corresponding relationship and the corresponding attribute are obtained, the triple information is obtained, and finally the interface information of the internal interface document and the external interface document, such as the interface identifier, the interface parameter, the interface response, and the upstream and downstream interface identifier, is extracted as the attribute value of the interface entity, which is mapped to the business flow template, and the extracted interface and key parameter entity triple are updated correspondingly, and the final triple information is obtained.

[0190] Step e13, generating the NSMF system knowledge graph based on the triple information.

[0191] The specific process of step e13 of generating the NSMF system knowledge graph can refer to the description of step e13 of generating the NSMF system knowledge graph in the method for generating the NSMF system knowledge graph. Figure 4As shown, comprising the following steps:

[0192] Step e131, defining the NSFM system graph data model, obtaining the selected graph database model.

[0193] Among them, a suitable graph database is selected, such as Neo4j, JanusGraph, ArangoDB, etc. These databases support graph data models and can effectively store and query triple data. The structure of the NSFM system graph database model may be, for example, Figure 5 As shown, it includes: business node-contains->link node->interface->interface node-association->key parameter-preceding->link infrastructure, wherein, Figure 5 The structure of interface->interface node-association->key parameter-preceding->link includes the infrastructure of internal interface node and external interface node.

[0194] Step e132, input the triple information into the selected graph database model, and perform entity modeling and operation interface packaging on the triple information through the selected graph database model, to obtain a packaging result.

[0195] Among them, based on the selected graph database, entity modeling and operation interface packaging are completed based on the triple information according to the business data model.

[0196] Step e133, generating the NSFM system knowledge graph based on the packaging result.

[0197] Among them, since the knowledge graph can structure and integrate information, it is convenient for information organization and intelligent processing, therefore, due to the complexity of the business scenarios, business key parameters and link interface association relationship of the NSFM system, the knowledge graph can be used for storage and representation, which is convenient for intelligent acquisition of scenario test path and determination of the business scenarios affected by software iteration update, so as to make the test accurate. The three basic elements of the knowledge graph are entity, relationship and attribute. Through the construction of triplets of the three basic elements, logical reasoning can be performed to discover the implicit relationship between entities, thereby expanding the knowledge graph.

[0198] Among them, taking Neo4j as an example, based on the object-oriented neomodel library, the data model business node Business, link node Step, interface Interface and key parameter Parameter class can be defined, the connection between nodes can be created, and the query and update can be completed automatically managed, and the triple information can be converted into the NSFM system knowledge graph using the operation application programming interface (API).

[0199] Step e14, differentially processing the service knowledge between the to-be-tested version and the previous version, obtaining an entity change range of the to-be-tested version, and determining a to-be-tested path set from the NSMF system knowledge graph according to the entity change range.

[0200] The specific implementation process of step e14 can refer to Figure 6 and specifically includes the following steps:

[0201] Step e141, determining the triple information of the to-be-tested version based on the service knowledge of the to-be-tested version.

[0202] Step e142, determining the triple information of the previous version based on the service knowledge of the previous version.

[0203] The product process template, interface document, and calculation priority of the to-be-tested version and the previous version are respectively analyzed to extract the triple information of the four types of entities, i.e., Business, Step, Interface, and Parameter.

[0204] Step e143, differentially processing the triple information of the to-be-tested version and the triple information of the previous version to determine the updated triple information of the to-be-tested version.

[0205] By comparing and analyzing the entity triple information of the previous version, the changed entities, changed attributes, and changed relationships are obtained, such as the changed business, changed link, changed interface, and changed parameter.

[0206] It should be noted that the updated triple information includes: no new entity, only the attributes and relationships of the original entity are changed, and the other is a new entity and the corresponding attributes and relationships.

[0207] Step e144, determining the to-be-tested path set from the NSMF system knowledge graph based on the updated triple information.

[0208] In the embodiment of the present application, the graph database query method is used to query all nodes that have any connection with a specific node in the updated triple information, and then for each business link, the links between the start and end link entities are matched, all links containing associated entities are included, each link corresponds to a scene test branch, and each link information includes Business, Step, Interface entity identifier, parameter node id, value, priority attribute value, and entity order identifier. In this way, the affected scene analysis in the to-be-tested version can be obtained, the to-be-tested path set can be obtained, and the determination of the precise test path is realized.

[0209] Step e15, sending the to-be-tested path set to the automatic test platform.

[0210] The set of paths to be tested is sent to the automated testing platform through an interface in communication with the automated testing platform. The automated testing platform corresponds to the aforementioned testing platform.

[0211] Step e16, the automated testing platform receives the set of paths to be tested, performs an automated testing operation, and outputs a test report.

[0212] The automated testing platform performs corresponding link interface arrangement on the set of paths to be tested, obtains corresponding test cases, performs an automated testing operation, and obtains a test report. The specific process can be as follows: after the automated testing platform receives the set of paths to be tested, it first calculates the link priority of each test path, wherein the link priority can be the cumulative sum of the priority attribute values, i.e., the priority weight coefficients, of all parameter nodes included in the test path; then, according to the information such as the business identifier, the link identifier, the interface identifier, the enumeration actual values of the key business identifier and the business key parameter, and the entity order identifier, the automated test cases of the atomized interfaces that have been completed on the automated platform are sequentially arranged. During the arrangement process, due to the huge number of NSMF system scenario branches, the scenario automated test cases can be executed in the order of the size of the calculated link priority values, and the test report is output after the test is completed.

[0213] In this way, the automated testing platform can display the test report in the display area of the automated testing platform, or send the test report to the network device for display in the corresponding display area of the network device, or send the test report to a third-party display device, so that relevant personnel can quickly know the test result.

[0214] In this way, the proposed precise test scheme based on the knowledge graph of the NSMF system extracts the triples of entities and attributes such as business, links, key business parameters, and interfaces of the NSMF system to be tested, stores them in a graph database, generates a scenario link knowledge graph of the NSMF system, and then analyzes the version to be compared. The path query of the scenario link knowledge graph can quickly search and retrieve the scenario link affected by the version to be tested. In addition, by defining the business priority information to form the link priority, important scenario links can be marked in priority, and automatic testing can be realized. Through the priority design and knowledge graph storage, the scenario test of the NSMF system can be efficiently and accurately completed. By selecting the scenarios to be tested for testing, the test time is saved. Compared with the manual test range determination method, the definition of the knowledge graph makes it more accurate, perfect, and avoids missing tests. The business knowledge analyzed is derived from product information, and the problem of inaccurate precise test range evaluation caused by code defects can be avoided, making the NSMF complex scenario test process more accurate, scientific, and automated, which can effectively improve the test efficiency and save the test labor cost.

[0215] It should be noted that the descriptions of the same steps and contents in other embodiments in this embodiment can refer to the descriptions in other embodiments, and will not be repeated here.

[0216] The test method provided by the embodiments of the present application determines the system knowledge graph corresponding to the to-be-tested version based on the first system set information of the to-be-tested version of the target application system, acquires the second system basic information of the target historical version of the target application system, then determines the to-be-tested path set of the to-be-tested version based on the first system basic information, the second system basic information, and the system knowledge graph, and finally performs a test operation based on the to-be-tested path set to obtain a test result. In this way, after determining the system knowledge graph of the to-be-tested version of the target application system and the second system basic information of the target historical version to be compared and analyzed, the first system basic information, the second system basic information, and the system knowledge graph of the to-be-tested version are analyzed to determine the to-be-tested path set, which realizes the accurate determination of the test range, effectively reduces the test range, and tests the to-be-tested path set, and then obtains the test result of the to-be-tested version. In this way, the problem of low test efficiency of the current precise test scheme is solved, and a test method is proposed. In the test process, the business development demand is fully considered when determining the test range, the accuracy of the determined test range is guaranteed, and the test efficiency is improved.

[0217] Based on the foregoing embodiments, an embodiment of the present application provides a test device, which can be applied to Figures 1-2 In the test method provided by the corresponding embodiments, reference is made toFigure 7 As shown in the figure, the test device 3 can include a first determining unit 31, an obtaining unit 32, a second determining unit 33, and an executing unit 34; wherein:

[0218] The first determining unit 31 is configured to determine a system knowledge graph corresponding to the to-be-tested version based on first system basic information of the to-be-tested version of the target application system.

[0219] The obtaining unit 32 is configured to obtain second system basic information of a target historical version of the target application system.

[0220] The second determining unit 33 is configured to determine a to-be-tested path set of the to-be-tested version based on the first system basic information, the second system basic information, and the system knowledge graph; wherein the to-be-tested path set includes m updated business paths, and m is an integer greater than or equal to 1.

[0221] The executing unit 34 is configured to perform a test operation based on the to-be-tested path set to obtain a test result.

[0222] In other embodiments of the present application, the first determining unit includes a first determining module, a second determining module, a third determining module, and a fourth determining module; wherein:

[0223] The first determining module is configured to determine, from the first system basic information, first priority information of a priority order of each first business key parameter corresponding to each application scenario included in the first system basic information.

[0224] The second determining module is configured to determine, based on a product design flowchart included in the first system basic information, first interface information, second interface information, and a second business key parameter corresponding to each business, to obtain first product flow information.

[0225] The third determining module is configured to determine first triple information of the to-be-tested version based on the first priority information, the first product flow information, and first system interface file information; wherein the first system interface file information belongs to the first system basic information.

[0226] The fourth determining module is configured to generate the system knowledge graph based on the first triple information.

[0227] In other embodiments of the present application, the first determining module is specifically configured to implement the following steps:

[0228] Determine a priority weight coefficient of each first business key parameter corresponding to each application scenario.

[0229] The first priority information is obtained by sorting all the first service key parameters included in the first system basic information according to the priority weight coefficients from large to small, and obtaining the priority order of each first service key parameter.

[0230] In other embodiments of the present application, when the first determining module is used to determine the priority weight coefficient of each first service key parameter corresponding to each application scenario, the following steps can be used to achieve this purpose:

[0231] The first weight coefficient and the second weight coefficient of each first service key parameter are determined, wherein the sum of the first weight coefficient and the second weight coefficient is 1.

[0232] The cumulative sum of the actual value of each first service key parameter when each enumerated parameter is determined in the actual running process is determined to obtain a first sum value.

[0233] The cumulative sum of the actual value of all enumerated parameters of each first service key parameter in the actual running process is determined to obtain a second sum value.

[0234] The ratio of the first sum value of each first service key parameter to the corresponding second sum value is calculated.

[0235] The product of the ratio of each first service key parameter and the corresponding first weight coefficient is calculated to obtain a first product.

[0236] The product of the preset priority order of each first service key parameter and the corresponding second weight coefficient is calculated to obtain a second product.

[0237] The sum of the first product of each first service key parameter and the corresponding second product is calculated to obtain the priority weight coefficient of each first service key parameter.

[0238] In other embodiments of the present application, the third determining module is specifically used to implement the following steps:

[0239] The first priority information, the first product flow information and the first system interface file information are preprocessed respectively to obtain the second priority information, the second product flow information and the second system interface file information in turn.

[0240] The second priority information, the second product flow information and the second system interface file information are analyzed to obtain target entities, target relationships and target attributes; wherein the first triple information includes target entities, target relationships and target attributes.

[0241] In other embodiments of the present application, when the third determining module performs the step of analyzing the second priority information, the second product process information and the second system interface file information to obtain the target entity, the following steps can be implemented:

[0242] parsing the second product process information to obtain the business name, the business link, the system business interface and the third business key parameter to obtain the reference entity;

[0243] uniformly processing the objects included in the reference entity by using the second system interface file information to obtain the target entity.

[0244] In other embodiments of the present application, the target attribute at least includes one or more of the following information: business identification information, business link identification information, interface attribute information and business parameter attribute; wherein the interface attribute information at least includes one or more of the following information: interface identification information, interface parameter information, interface response information, upstream and downstream interface identification information, and the business parameter attribute at least includes the enumeration parameter value of the third business key parameter and the priority order of the third business key parameter.

[0245] In other embodiments of the present application, the target relationship at least includes one or more of the following relationships: the scene link upstream and downstream dependency relationship between multiple links, the belonging relationship between the business and the link, the belonging relationship between the link and the interface, and the influence relationship between the interface and the third business key parameter.

[0246] In other embodiments of the present application, the second determining unit includes a fifth determining module and a sixth determining module; wherein:

[0247] The fifth determining module is configured to determine the first triple information of the to-be-tested version based on the first system basic information;

[0248] The fifth determining module is configured to determine the second triple information of the target historical version based on the second system basic information;

[0249] The sixth determining module is configured to determine the to-be-tested path set based on the first triple information, the second triple information and the system knowledge graph.

[0250] In other embodiments of the present application, the sixth determining module is specifically configured to implement the following steps:

[0251] updating and analyzing the first triple information and the second triple information to obtain an analysis result;

[0252] determining one or more complete links having a correlation relationship with the update object included in the analysis result from the system knowledge graph to obtain the to-be-tested path set.

[0253] In other embodiments of the present application, the analysis result includes one or more of the following information: one or more updated entities, one or more updated relationships, and one or more updated attributes.

[0254] In other embodiments of the present application, the execution unit includes a seventh determination module, an arrangement module, and a test module, wherein:

[0255] The seventh determination module is configured to determine a path priority of each test path in the set of test paths to be tested.

[0256] The arrangement module is configured to arrange the paths in the set of test paths to be tested based on the path priority of each test path, to obtain an arranged test case.

[0257] The test module is configured to test the arranged test case, to obtain a test result.

[0258] In other embodiments of the present application, the seventh determination module is specifically configured to implement the following steps:

[0259] Determine a priority weight coefficient of the fourth business critical parameter of each test path in the set of test paths to be tested.

[0260] Accumulate the priority weight coefficient of the fourth business critical parameter of each test path, to obtain a path priority of the corresponding test path.

[0261] In other embodiments of the present application, the execution unit includes a sending module and a receiving module, wherein:

[0262] The sending module is configured to send the set of test paths to be tested to the test platform; wherein the test operation includes a sending operation of sending the set of test paths to be tested to the test platform.

[0263] Alternatively, the sending module is configured to send the arranged test case obtained based on the set of test paths to be tested to the test platform; wherein the test operation includes a sending operation of sending the arranged test case to the test platform.

[0264] The receiving module is configured to receive a test result sent by the test platform; wherein the test result is obtained by the test platform based on the set of test paths to be tested or the arranged test case.

[0265] It should be noted that the process of information exchange between the units and modules in the present embodiment can refer to the description in other embodiments, which will not be repeated here.

[0266] The test device provided by the embodiment of the application determines the system knowledge graph corresponding to the to-be-tested version based on the first system set information of the to-be-tested version of the target application system, acquires the second system basic information of the target historical version of the target application system, then determines the to-be-tested path set of the to-be-tested version based on the first system basic information, the second system basic information and the system knowledge graph, and finally performs a test operation based on the to-be-tested path set to obtain a test result. In this way, after the system knowledge graph of the to-be-tested version of the target application system and the second system basic information of the target historical version to be compared and analyzed are determined, the first system basic information, the second system basic information and the system knowledge graph of the to-be-tested version are analyzed to determine the to-be-tested path set, the accurate test range is determined, the test range is effectively reduced, the to-be-tested path set is tested, and then the test result of the to-be-tested version is obtained. In this way, the problem of low test efficiency of the current accurate test scheme is solved, and a test method is provided. In the determination of the test range in the test process, the business development demand is fully considered, the accuracy of the determined test range is ensured, and the test efficiency is improved.

[0267] Based on the foregoing embodiment, the embodiment of the application provides a network device, which can be applied to Figures 1-2 In the test method provided by the corresponding embodiment, as shown in Figure 8 The network device 4 can include a communication interface 41, a memory 42, a processor 43 and a communication bus 44; wherein:

[0268] The memory 42 is configured to store executable information.

[0269] The communication bus 44 is configured to realize the communication connection among the communication interface 41, the processor 43 and the memory 42.

[0270] The processor 43 is configured to execute the test program stored in the memory 42 to realize the method as Figures 1-2 The implementation process in the test method provided by the corresponding embodiment is not described herein again.

[0271] Based on the foregoing embodiment, the embodiment of the application provides a computer readable storage medium, referred to as a storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to realize the method as Figures 1-2 The implementation process in the test method provided by the corresponding embodiment is not described herein again.

[0272] Based on the foregoing embodiment, the embodiment of the application further provides a computer program product, which includes a computer program. The computer program can be executed by the processor 43 of the network device 4 to complete any method step described above.

[0273] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a "circuit" or "module." Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, optical memory etc.) embodying computer program code adapted to carry out the steps of any of the methods described herein.

[0274] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0275] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0276] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0277] The above description is embodied in the context of preferred embodiments of the application and is not intended to limit the scope of the application.

Claims

1. A testing method, characterized in that, The method includes: Based on the first system basic information of the test version of the target application system, a system knowledge graph corresponding to the test version is determined; wherein, the first system basic information includes at least the product design flowchart of the test version, explanatory information including relevant graphics and text in the design process, and development documents, and the system knowledge graph is used to represent the relationship between the entities included in the test version; Obtain the second system basic information of the target historical version of the target application system; wherein, the second system basic information includes at least the product design flowchart of the target historical version, explanatory information including relevant graphics and texts in the design process, and development documents; Based on the basic information of the first system, the basic information of the second system, and the system knowledge graph, a set of test paths for the version to be tested is determined; wherein, the set of test paths includes m updated business paths, where m is an integer greater than or equal to 1; The test operation is performed based on the set of paths to be tested, and the test results are obtained; The step of determining the set of test paths for the version to be tested based on the first system basic information, the second system basic information, and the system knowledge graph includes: Based on the basic information of the first system, the first triplet information of the version to be tested is determined; Based on the basic information of the second system, the second triplet information of the target historical version is determined; Based on the first triplet information, the second triplet information, and the system knowledge graph, the set of paths to be tested is determined; The step of determining the set of paths to be tested based on the first triplet information, the second triplet information, and the system knowledge graph includes: The basic information of the first system and the basic information of the second system are analyzed to determine the updated content in the basic information of the first system. The complete path including the updated content is determined from the system knowledge graph to obtain the set of paths to be tested.

2. The method according to claim 1, characterized in that, Based on the first system basic information of the target application system's version to be tested, the system knowledge graph corresponding to the version to be tested is determined, including: From the first system basic information, determine the first priority information of the priority order of each first business key parameter corresponding to each application scenario; Based on the product design flowchart included in the basic information of the first system, the first interface information, the second interface information, and the second business key parameters corresponding to each business are determined to obtain the first product process information. Based on the first priority information, the first product process information, and the first system interface file information, the first triplet information of the version to be tested is determined; wherein, the first system interface file information belongs to the first system basic information; Based on the information from the first triplet, the system knowledge graph is generated.

3. The method according to claim 2, characterized in that, The first priority information for determining the priority order of each first business key parameter corresponding to each application scenario from the first system basic information includes: Determine the priority weight coefficient for each of the first business key parameters corresponding to each application scenario; Based on the priority weight coefficients of all the first business key parameters included in the first system basic information, all the first business key parameters included in the first system basic information are sorted in descending order of priority weight coefficients to obtain the priority order of each first business key parameter, and thus the first priority information is obtained.

4. The method according to claim 3, characterized in that, The step of determining the priority weight coefficient of each first business key parameter corresponding to each application scenario includes: Determine a first weighting coefficient and a second weighting coefficient for each of the first key parameters of the business; wherein the sum of the first weighting coefficient and the second weighting coefficient is 1; During actual operation, each of the first key business parameters is determined to be the sum of the actual values ​​of each enumerated parameter, thus obtaining the first sum value. The second sum is obtained by summing the actual values ​​of all enumerated parameters of each of the first key business parameters during actual operation. Calculate the ratio of the first sum value to the corresponding second sum value for each of the first key business parameters; Calculate the product of the ratio of each of the first key business parameters and the corresponding first weight coefficient to obtain the first product; Calculate the product of the preset priority order of each of the first business key parameters and the corresponding second weight coefficient to obtain the second product; Calculate the sum of the first product and the corresponding second product of each of the first key business parameters to obtain the priority weight coefficient of each of the first key business parameters.

5. The method according to claim 2, characterized in that, The step of determining the first triplet information of the version to be tested based on the priority information, the first product process information, and the first system interface file information includes: The first priority information, the first product process information, and the first system interface file information are preprocessed respectively to obtain the second priority information, the second product process information, and the second system interface file information in sequence. The second priority information, the second product process information, and the second system interface file information are analyzed to obtain the target entity, the target relationship, and the target attribute; wherein, the first triplet information includes the target entity, the target relationship, and the target attribute.

6. The method according to claim 5, characterized in that, The analysis of the second priority information, the second product process information, and the second system interface file information to obtain the target entity includes: The second product process information is parsed to obtain the business name, business steps, system business interfaces, and third business key parameters, thus obtaining the reference entity. The objects included in the reference entity are processed using the information from the second system interface file to obtain the target entity.

7. The method according to claim 5, characterized in that, The target attribute includes at least one or more of the following: business identification information, business process identification information, interface attribute information, and business parameter attribute; wherein, the interface attribute information includes at least one or more of the following: interface identification information, interface parameter information, interface response information, and upstream and downstream interface identification information, and the business parameter attribute includes at least: the enumerated parameter values ​​of the third business key parameter and the priority order of the third business key parameter.

8. The method according to claim 5, characterized in that, The target relationship includes at least one or more of the following relationships: upstream and downstream dependency relationships in the scenario link between multiple links, ownership relationships between business and links, ownership relationships between links and interfaces, and influence relationships between interfaces and third-party business key parameters.

9. The method according to claim 1, characterized in that, The step of determining the set of paths to be tested based on the first triplet information, the second triplet information, and the system knowledge graph includes: The information of the first triplet and the information of the second triplet are updated and analyzed to obtain the analysis results; The set of paths to be tested is obtained by identifying one or more complete links in the system knowledge graph that are associated with the updated objects included in the analysis results.

10. The method according to claim 9, characterized in that, The analysis results include at least one or more of the following: one or more updated entities, one or more updated relationships, and one or more updated attributes.

11. The method according to claim 1, characterized in that, The step of performing test operations based on the set of paths to be tested and obtaining test results includes: Determine the path priority of each test path in the set of paths to be tested; Based on the path priority of each test path, the paths in the set of paths to be tested are arranged to obtain orchestrated test cases. The orchestrated test cases are tested to obtain the test results.

12. The method according to claim 11, characterized in that, Determining the path priority of each test path in the set of paths to be tested includes: Determine the priority weight coefficient of the fourth business key parameter for each of the test paths in the set of test paths; The priority of the corresponding test path is obtained by summing the priority weight coefficients of the fourth business key parameter of each test path.

13. The method according to claim 1, characterized in that, The step of performing test operations based on the set of paths to be tested and obtaining test results includes: Send the set of paths to be tested to the test platform; wherein, the test operation includes sending the set of paths to be tested. Alternatively, orchestrated test cases based on the set of paths to be tested are sent to the test platform; wherein, the test operation includes the sending operation of sending the orchestrated test cases to the test platform; Receive the test results sent by the test platform; wherein the test results are obtained by the test platform based on the set of paths to be tested, or the orchestration of test cases.

14. A testing apparatus, characterized in that, The device includes: a first determining unit, an acquiring unit, a second determining unit, and an execution unit; wherein: The first determining unit is used to determine the system knowledge graph corresponding to the version to be tested based on the first system basic information of the version to be tested of the target application system; wherein, the first system basic information includes at least the product design flowchart of the version to be tested, explanatory information including relevant graphics and text in the design process, and development documents, and the system knowledge graph is used to represent the relationship between the entities included in the version to be tested; The acquisition unit is used to acquire the second system basic information of the target historical version of the target application system; wherein, the second system basic information includes at least the product design flowchart of the target historical version, explanatory information including relevant graphics and texts in the design process, and development documents; The second determining unit is used to determine the set of test paths for the version to be tested based on the first system basic information, the second system basic information, and the system knowledge graph; wherein the set of test paths includes m updated business paths, where m is an integer greater than or equal to 1; The execution unit is used to perform test operations based on the set of paths to be tested and obtain test results; The second determining unit is implemented through the following steps: Based on the basic information of the first system, the first triplet information of the version to be tested is determined; Based on the basic information of the second system, the second triplet information of the target historical version is determined; Based on the first triplet information, the second triplet information, and the system knowledge graph, the set of paths to be tested is determined; When the second determining unit executes the step of determining the set of paths to be tested based on the first triplet information, the second triplet information, and the system knowledge graph, it does so through the following steps: The basic information of the first system and the basic information of the second system are analyzed to determine the updated content in the basic information of the first system. The complete path including the updated content is determined from the system knowledge graph to obtain the set of paths to be tested.

15. A network device, characterized in that, The device includes: a communication interface, a memory, a processor, and a communication bus; wherein: The memory is used to store executable instructions; The communication bus is used to realize the communication connection between the communication interface, the processor and the memory; The processor is configured to execute the test program stored in the memory, and implement the steps of the test method as described in any one of claims 1 to 13.

16. A storage medium, characterized in that, The storage medium stores a test program, which, when executed, is used to implement the steps of the test method as described in any one of claims 1 to 13.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the test method as described in any one of claims 1 to 13.

Images