Method and apparatus for testing online system, and device, medium and program product

Abstract

A method and apparatus for testing an online system, and a device, a medium and a program product, which belong to the field of data processing. The method comprises: on the basis of service message features of an acquired service message, constructing a multi-layer message node network (S101), wherein the multi-layer message node network comprises message nodes and connection edges between the message nodes; on the basis of the multi-layer message node network, using a graph embedding technique and a neural network model, so as to obtain the category of the service message (S102); on the basis of the service message, the category of the service message and a pre-trained service message family division model, obtaining at least one target service message family (S103), wherein the service message family division model is obtained by performing training on the basis of known service message families in a production environment, and each service message family is a collection of service messages of the categories required for realizing service functions; and on the basis of a set testing policy, sending to a testing application a service message in the target service message family, such that the testing application processes the service message in the target service message family (S104).

Description

Testing method, device, equipment, medium and program product for online system

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202410331354.8, filed on March 21, 2024, entitled “Testing methods, devices, equipment, media and program products for online systems,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of data processing, and in particular to a testing method, device, equipment, medium and program product for an online system. Background Art

[0004] The real-time nature of the business operations performed by online systems allows for continuous, uninterrupted operation, with periodic fluctuations in transactions per second (TPS). As business demands increase, online systems require corresponding updates and upgrades. To mitigate the risks of these updates and upgrades and ensure the availability of the updated and upgraded online systems, at least some business messages can be diverted to a test environment before the online trading system is officially put into use. The execution results of these messages in the test environment can be compared and verified with those in the production environment to complete the testing process.

[0005] However, the pairing, orchestration, and other processing processes of business messages in the test environment need to be manually summarized and hard-coded in the processing device. Once a new business category or external interactive interface is required, manual development and configuration work is required, which increases the workload of the online system testing work and reduces the efficiency of the online system testing work. Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, device, medium, and program product for testing an online system, which can reduce the development workload of online system testing.

[0007] In the first aspect, an embodiment of the present application provides a testing method for an online system, including: constructing a multi-layer message node network based on the business message characteristics of the acquired business messages, the multi-layer message node network including message nodes and connection edges between message nodes; obtaining the category of the business message according to the multi-layer message node network using graph embedding technology and a neural network model; obtaining at least one target business message family according to the business message, the category of the business message and a pre-trained business message family partitioning model, the business message family partitioning model is trained based on known business message families in a production environment, and the business message family is a collection of business messages of categories required to implement business functions; according to the set test strategy, sending the business messages in the target business message family to the test application so that the test application processes the business messages in the target business message family.

[0008] In the second aspect, an embodiment of the present application provides a testing device for an online system, including: a network construction module, which is used to construct a multi-layer message node network based on the business message characteristics of the acquired business messages, and the multi-layer message node network includes message nodes and connection edges between message nodes; a category determination module, which is used to obtain the category of the business message based on the multi-layer message node network using graph embedding technology and a neural network model; a message family partitioning module, which is used to obtain at least one target business message family based on the business message, the category of the business message and a pre-trained business message family partitioning model, and the business message family partitioning model is trained based on known business message families in the production environment, and the business message family is a collection of business messages of categories required to implement business functions; a sending module, which is used to send business messages in the target business message family to the test application in accordance with the set test strategy, so that the test application processes the business messages in the target business message family.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the online system testing method of the first aspect is implemented.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the online system testing method of the first aspect is implemented.

[0011] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the online system testing method of the first aspect when the computer program is executed by a processor.

[0012] The embodiments of the present application provide a testing method, apparatus, device, medium, and program product for an online system. These methods can construct a multi-layer message node network based on the service message characteristics of the service messages. The multi-layer message node network can reflect the association relationship between service messages based on the service message characteristics. Based on the multi-layer message node network, the categories of service messages can be obtained using graph embedding technology and a neural network model. A service message family partitioning model can be used to partition service messages into at least one target service message family. Service messages in the target service message family are then sent to a test application according to a set test strategy, so that the test application processes the service messages in the target service message family, thereby implementing testing. The business message family division model is a model that can automatically learn the relationship between the categories of business messages and business message families. Even when new message categories or new external interaction interfaces are added, resulting in new categories of business messages, the business message family division model can learn the association between the new categories of business messages and business message families, thereby automatically dividing the new categories of business messages into corresponding target business message families. This process does not require manual development and configuration work for the newly added message categories or newly added external interaction interfaces, reducing the development workload of online system testing and improving the testing efficiency of the online system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] FIG1 is a flow chart of a method for testing an online system according to an embodiment of the present application;

[0015] FIG2 is a flow chart of a method for testing an online system according to another embodiment of the present application;

[0016] FIG3 is a logic diagram of an example of generating a feature knowledge graph provided in an embodiment of the present application;

[0017] FIG4 is a logical diagram of an example of building a multi-layer message node network according to an embodiment of the present application;

[0018] FIG5 is a logical diagram of an example of obtaining a category of a service message provided in an embodiment of the present application;

[0019] FIG6 is a flowchart of a method for testing an online system according to another embodiment of the present application;

[0020] FIG7 is a schematic diagram of an example of training a service message segmentation model in an embodiment of the present application;

[0021] FIG8 is a schematic diagram of an example of an application system architecture of a method for testing an online system provided in an embodiment of the present application;

[0022] FIG9 is a schematic structural diagram of a testing device for an online system according to an embodiment of the present application;

[0023] FIG10 is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating examples of the present application. It should be noted that the acquisition, storage, use, processing, etc. of information and data in the embodiments of the present application are authorized by the user or relevant agencies and comply with the relevant provisions of national laws and regulations.

[0025] The real-time nature of the services executed by online systems allows for continuous, uninterrupted operation, with periodic fluctuations in transactions per second (TPS). As business demands increase, online systems require corresponding updates and upgrades. To reduce the risks of these updates and upgrades and ensure the availability of the updated and upgraded online systems, at least some business messages can be directed to a test environment before the online transaction system is officially put into use. This allows for comparison and verification of the execution results of the business messages in the test environment with those in the production environment to complete the testing process. However, the matching and orchestration of business messages in the test environment must be manually summarized and hard-coded into the processing device. Whenever new services or external interaction interfaces are needed, manual development and configuration are required, increasing the development workload for online system testing and reducing its efficiency.

[0026] The testing method, device, equipment, medium and program product of the online system provided by the present application can use the business messages obtained by diversion to obtain the business message characteristics of the business messages, and obtain the category of the business messages based on the business message characteristics. The business message family division model can automatically learn the association between the business messages and the message family that realizes the business function, complete the training of the business message family division model, and use the trained business message family division model to divide the business messages diverted to the test environment in real time into business message families based on the business messages diverted to the test environment in real time and the categories of the business messages, and then send the business messages in the business message family to the test application according to the test strategy to realize the test. In the case of a new category of business messages appearing due to a new message category or a new external interaction interface, the business message family division model can automatically learn the relationship between the new category of business messages and the business group that appear due to the new message category or the new external interaction interface, without the need for manual supporting development and configuration work, thereby reducing the development workload of the online system test and improving the test efficiency and test accuracy of the online system.

[0027] The following describes the testing method, device, equipment, medium and program product of the online system provided by this application.

[0028] The first aspect of the present application provides a testing method for an online system, which can be applied to a test scenario for updating and upgrading an online system, and the test scenario can specifically be a parallel testing scenario. The online system may include a system that performs business in one or more business areas. For example, the online system can perform transaction business, that is, the online system can be implemented as an online transaction business system, but is not limited to this. In an embodiment of the present application, the "online" in the online system focuses on the meaning of online (i.e., online), that is, the online system can be regarded as an online system that can work continuously and uninterruptedly to realize business functions. The testing method of the online system can be executed by a testing device, equipment, system, etc. of the online system, and is not limited here. Figure 1 is a flow chart of the testing method of the online system provided in an embodiment of the present application. As shown in Figure 1, the testing method of the online system may include steps S101 to S104.

[0029] In step S101, a multi-layer message node network is constructed based on the acquired service message characteristics.

[0030] The acquired business messages may include business messages acquired from online applications that interact with the online system using the offload frontend. The specific content of the business messages is related to the business and is not limited to the type of business. For example, if the business is a transaction, the business messages may include transaction messages.

[0031] Business messages can be parsed to obtain business message features. Business messages may include text, binary, or other encoding methods and storage structures, but this is not limited here. Corresponding business message features can be identified for all types of business messages. Business message features may include message features and / or business features. Message features reflect the characteristics of a business message. In some examples, message features may include, but are not limited to, one or more of the following: message format, business code, business message channel, business message domain, business message usage region, business medium, and business credential information. The business code may include a code representing the service type included in the business message. The business message channel identifies the source channel of the business message. The business message domain identifies the usage region of the business message. The business medium identifies the medium that carries the service. Business credential information includes information related to the credentials required to execute the service. For example, if the business message is a transaction message, message features may include, but are not limited to, transaction message format, transaction code, transaction channel, merchant terminal domain, transaction region, transaction medium, and transaction card information. Message formats may include communication protocols, International Organization for Standardization (ISO) protocols, message interface specifications, and business party agreements. Transaction codes may include debit, goods and services, cash, check, letter of credit, and direct transfer. Transaction channels may be categorized by automated teller machines (ATMs), point-of-sale (POS), internet, and mobile payment. Merchant terminal sectors may be categorized by government services, public utilities, railways and aviation, gas stations, travel rentals, and supermarkets. Transaction regions may be categorized by whether a transaction card in one region is used in another, or by specific transaction regions. Transaction media may be categorized by magnetic cards, chip cards, cardless cards, and barcodes. Transaction card information may be categorized by card product and card attributes. Business characteristics reflect the business-focused nature of business messages. In some examples, these characteristics may include, but are not limited to, the number of transactions, business classification, business success rate, queries per second (QPS) / transactions per second (TPS), business distribution, and business scenario coverage. Business distribution can represent the distribution of business across certain categories. Business scenario coverage categories can represent the categories of scenarios that a business can cover. For example, if the business message is a transaction message, business characteristics may include but are not limited to transaction quantity, transaction category, transaction success rate, QPS / TPS, transaction distribution, and transaction scenario coverage categories. Transaction classification can be categorized by communication protocol, external system, transaction type, etc. Transaction distribution can be categorized by forward transactions, reverse transactions, and internal management transactions.The transaction scenario coverage categories can be classified according to general transactions, transaction discounts, risk assessments, transaction limits, resource party barcodes, regional alliances, etc.

[0032] Based on the business message characteristics, the relationship between business messages and business message characteristics can be established, and then based on the association relationship between business messages and business message characteristics, the association relationship between business messages can be constructed. A multi-layer message node network can represent the association relationship between business messages. A multi-layer message node network includes message nodes and connection edges between message nodes. Message nodes represent business messages, and the connection edges between message nodes represent that there is an association relationship between message nodes. A multi-layer message node network is a multi-layer structure. Each layer of structure can correspond to a feature. The feature corresponding to each layer can be a single feature or a composite feature, which is not limited here. Each layer of structure includes connection edges between message nodes and message nodes. Message nodes representing the same business message in different layer structures have connection edges, and message nodes representing different business messages in different layer structures do not have connection edges.

[0033] In step S102, based on the multi-layer message node network, the category of the service message is obtained using graph embedding technology and a neural network model.

[0034] A multi-layer message node network can reflect the relationships between message nodes. Graph embedding technology can be used to navigate within this multi-layer message node network. Node vectors for each message node are derived based on the path of the network. These node vectors can reflect the characteristics of the service message. A pre-trained neural network model is used to process these node vectors and classify them, thereby deriving the service message category.

[0035] In some examples, if the business messages are diverted to the test environment by mirror diversion, that is, the generation environment has the same business messages as those diverted to the test environment, then the steps of building a multi-layer message node network and obtaining the category of the business messages can be performed in the production environment. When the test environment needs to further process the business messages, the category of the business messages already obtained in the production environment can be directly obtained, and the business messages can be further processed using the category of the business messages.

[0036] In other examples, if proportional diversion is used to divert business messages to a test environment, that is, the generated environment does not have the same business messages as those diverted to the test environment, then a multi-layer message node network can be constructed in the test environment and the categories of the business messages can be obtained, and the business messages can be further processed using the categories of the business messages.

[0037] In step S103, at least one target service message family is obtained according to the service message, the category of the service message and the pre-trained service message family classification model.

[0038] The service message family classification model is trained based on known service message families in the production environment. A service message family includes at least one service message. A service message family is a collection of service messages of the required categories to implement a service function. That is, the service messages in a service message family can collectively support the implementation of the service function. For example, if the service function is to cancel a transaction after it has already been executed, the service message family may include messages requesting a transaction and messages requesting a transaction cancellation. The service message family classification model classifies service messages into corresponding service message families, with the target service message family being the service message family output by the service message family classification model. The training process of the service message family classification model automatically learns the relationship between service message categories and service message families. The training of the service message group classification model is continuous. When new message categories or external interaction interfaces are added, resulting in new service message categories, the service message family classification model can also learn the relationship between new service message categories and service message families, thereby automatically classifying the new service message categories into corresponding service message families.

[0039] In step S104, according to the set test strategy, the service messages in the target service message family are sent to the test application, so that the test application processes the service messages in the target service message family.

[0040] The test strategy can be set based on specific test requirements. The target service message family includes at least one service message. Based on the set test strategy, the order and timing of service messages sent to the test application can be determined, thereby simulating test scenarios and enabling the test application to process received service messages. This allows for simulation of service message processing in test scenarios, enabling more test scenarios and expanding test scenarios.

[0041] In an embodiment of the present application, a multi-layer message node network can be constructed based on the service message characteristics of the service messages. The multi-layer message node network can reflect the association relationship between service messages based on the service message characteristics. Based on the multi-layer message node network, the categories of service messages can be obtained using graph embedding technology and neural network models. The service messages can be divided into at least one target service message family using the service message family partitioning model. The service messages in the target service message family are sent to the test application according to the set test strategy, so that the test application processes the service messages in the target service message family, thereby realizing the test. The business message family division model is a model that can automatically learn the relationship between the categories of business messages and business message families. Even when new message categories or new external interaction interfaces are added, resulting in new categories of business messages, the business message family division model can learn the association between the new categories of business messages and business message families, thereby automatically dividing the new categories of business messages into corresponding target business message families. This process does not require manual development and configuration work for the new message categories or new external interaction interfaces, reducing the development workload of online system testing and improving the testing efficiency and accuracy of the online system.

[0042] In some embodiments, a network that can characterize the relationship between business messages and business message features can be generated first, and then a multi-layer message node network that characterizes the association relationship between business messages can be obtained based on this. The multi-layer message node network is used to obtain node vectors of message nodes using the wandering path technology and continuous hopping model in the graph embedding technology, and the business messages represented by the message nodes are classified using the node vectors. Figure 2 is a flowchart of a testing method for an online system provided by another embodiment of the present application. The difference between Figure 2 and Figure 1 is that step S101 in Figure 1 can be specifically refined into steps S1011 to S1013 in Figure 2, and step S102 in Figure 1 can be specifically refined into steps S1021 to S1023 in Figure 2.

[0043] In step S1011, dimensionality reduction processing is performed on the service message features of multiple dimensions of the service message to obtain the service core features.

[0044] Due to the complex nature of business message features, the effective features required to identify different business functions vary. Feature engineering can be used to reduce the dimensionality of business message features across multiple dimensions and filter out core business features. Core business features include the effective features required to identify different business functions. They can include individual features or composite features. A single feature can represent a single business message characteristic, while a composite feature can simultaneously represent multiple business message characteristics. By filtering core business features, the efficiency of building a multi-layer message node network can be further improved, preventing invalid features from affecting the relationships between business messages.

[0045] In step S1012, a multi-layer service message feature network is generated according to the service core feature and the value of the service core feature of the service message.

[0046] The values ​​of the same business core feature of different business messages may be different. Each business core feature may correspond to a layer of structure in a multi-layer business message feature network. Each layer in the multi-layer business message feature network includes a message node and a feature value node corresponding to the business core feature. The message node represents the business message. The feature value node represents the business core feature and the value of the business core feature. In the multi-layer business message feature network, the message node whose business core feature has a value represented by the feature value node is connected to the corresponding feature value node, that is, for each business core feature, the message node can be connected to the feature value node corresponding to the value of the business core feature of the business message represented by the message node, thereby forming the structure of each layer of the multi-layer business message feature network, and then forming a multi-layer business message feature network. The business message feature network can reflect the correlation between the business message and the business core feature.

[0047] In step S1013, a multi-layer message node network is obtained according to the multi-layer service message feature network.

[0048] Each layer in the multi-layer message node network includes message nodes corresponding to the core service characteristics of that layer. The message nodes connected within each layer are message nodes connected to the same feature value node in the multi-layer service message feature network. In some examples, message nodes connected to the same feature value node in the same layer of the multi-layer service message feature network can be connected and the feature value node can be deleted. Similarly, the same message nodes in different layers of the multi-layer service message feature network can be connected to form a multi-layer message node network. Identical message nodes in different layers represent the same service message.

[0049] In some examples, a feature knowledge graph can be generated based on the business core features and the values ​​of the business core features of the business message. The feature knowledge graph has a single-layer structure. Then, a multi-layer business message feature network is generated based on the feature knowledge graph. Knowledge graph technology is applied to the business identification scenario. With the business core features as attributes, a feature knowledge graph is constructed. Based on the rule that the same business type has similar features, business messages of the same type are associated through different features. Transaction messages with similar business core features can be associated through the meta-path of <message node-feature value node-message node>. Based on the feature knowledge graph and the multi-layer business message feature network, the direct association relationship of business messages in different feature dimensions is extracted based on the association relationship generated by the same feature value node, that is, the same value of the same business core feature in the same feature dimension, to obtain a multi-layer message node network.

[0050] For example, FIG3 is a logical diagram of an example of generating a feature knowledge graph provided by an embodiment of the present application. As shown in FIG3, the business message feature 1, business message feature 2, business message feature 3, business message feature 4, business message feature 5, ..., business message feature x can be first subjected to dimensionality reduction processing to obtain business core feature 1, business core feature 2 and business core feature 3. According to the values ​​of the business core features of the business message, the feature knowledge graph shown in FIG3 is generated; the feature knowledge graph in FIG3 includes message nodes and feature value nodes, and the message nodes include transaction message 1 As for transaction message 10, the feature value nodes include business core feature 1: value 1, business core feature 1: value 2, business core feature 2: value 1, business core feature 2: value 2, business core feature 2: value 3, business core feature 3: value 1 and business core feature 3: value 2; the connection relationship between the message nodes and the feature value nodes in the feature knowledge graph is shown in Figure 3, for example, the value of the business core feature 2 of transaction message 2 is value 1, and the value of the business core feature 3 of transaction message 2 is value 1. The relationship between the message nodes and the feature value nodes in Figure 3 is not described one by one here. FIG4 is a logic diagram of an example of building a multi-layer message node network provided by an embodiment of the present application. In order to simplify the text, the transaction message in the message node is simplified to a message, the business core features in the feature value node in each layer are simplified to feature 1, feature 2, feature 3, etc., and the values ​​are simplified to value 1, value 2, value 3, etc. As shown in FIG4 , the business core features can be layered according to the feature knowledge graph. The first layer corresponds to the business core feature 1 (i.e., feature 1 in FIG4 ), and the second layer corresponds to the business core feature 2 (i.e., feature 3 in FIG4 ). 2) Correspondingly, the third layer corresponds to the business core feature 3 (i.e., feature 3 in Figure 4), and a multi-layer business message feature network is obtained. The connection relationship between the message nodes and the feature value nodes in the multi-layer business message feature network is consistent with the feature knowledge graph; the message nodes connected to the same feature value node in the same layer of the multi-layer business message feature network are connected, and the feature value nodes are deleted, and the same message nodes in different layers are connected to form a multi-layer message node network, in which the connection lines of the message nodes in the same layer are solid lines, and the connection lines of the message nodes in different layers are dotted lines.

[0051] In some examples, a connection edge between two message nodes in a multi-layer message node network has a weight, which can be derived based on the message nodes to which each of the two message nodes is connected. A connection edge between two message nodes connected on the same layer of a multi-layer message node network can represent a direct association between the two message nodes under a core business characteristic, and the weight of the connection edge between message nodes on the same layer can represent the similarity between the two message nodes under that core business characteristic. Connection edges between message nodes on different layers can represent the association between the same message node under different core business characteristics, and the weight of the connection edges between message nodes on different layers can represent the similarity between the same message node under different core business characteristics.

[0052] For example, the weight of the connection edge between message nodes in the same layer in a multi-layer message node network can be calculated by the Jaccard method, as shown in the following formula (1):

[0053] Among them, W i,j Message node V i and message node V j The weight of the connection edge; N(i) is the weight of the connection edge with the message node V i The set of directly connected message nodes; N(j) is the number of nodes connected to the message node V j The set of directly connected message nodes; ∩ is the intersection symbol; ∪ is the union symbol.

[0054] The weight calculation of the connection edge between message nodes in different layers in a multi-layer message node network can be referred to the following formulas (2) and (3):

[0055] in, is the message node V at layer m i m With the message node V at the nth layer j n The weight of the connecting edge between them; is the message node V in the mth layer i m The set of directly connected message nodes; is the message node V in the nth layer j n The set of directly connected message nodes; ∩ is the intersection symbol; ∪ is the union symbol. In the case of i≠j, 0 indicates the message node V i m With message node V j n There are no connecting edges between them.

[0056] In step S1021 , starting from each message node in the multi-layer message node network, a wandering path with the highest probability for each message node is obtained.

[0057] In a multi-layer message node network, any message node can be used as the starting point to obtain a walking path starting from the message node. The length of the walking path can be preset. For example, a walking path starting from message node v0 with a length of i can be expressed as walk v =(v0, v1, v2, ..., v i ), the probability of the walking path Pr(v0, v1, v2, ..., v i-1 ∣v i ) is the maximum value, which can be achieved by optimizing the objective function To obtain the most probable wandering path, thus achieving graph embedding of nodes. In a multi-layer message node network, different layers only connect to the same message nodes, ensuring that nodes between layers wander only between the same message nodes. This avoids mixing up the associations of service messages with different core business characteristics, further improving the accuracy of the associations between nodes and messages reflected in the multi-layer message node network.

[0058] Graph embedding technology can map the high-dimensional, dense structure of a multi-layered message node network into a low-dimensional, dense vector. This allows the correlation between the service message characteristics represented by the multi-layered message node network to be efficiently integrated with machine learning algorithms. In some examples, the Node2Vec algorithm can be used to generate travel paths for message nodes. These travel paths can also be called travel queue path lists.

[0059] In step S1022 , a node vector corresponding to each message node is obtained based on the most probable wandering path and the continuous word-hopping model of each message node.

[0060] The continuous hopping model can be trained using the most probable wandering path of the message node, thereby obtaining a node vector of the message node. The node vector is a vector that reflects the service message characteristics of the service message.

[0061] In step S1023, the node vector corresponding to the message node is input into the neural network model to obtain the category of the service message represented by the message node.

[0062] The node vector corresponding to the message node can be used as the input of the neural network model to obtain the category of the business message represented by the message node output by the neural network. The neural network model can be obtained through supervised learning training.

[0063] For example, FIG5 is a logical diagram of an example of obtaining the category of a business message provided by an embodiment of the present application. As shown in FIG5, the Node2Vec algorithm can be used to generate a wandering path for the message node. The wandering path can be used as a representation of the message node, and the wandering path can be regarded as a target word w(t). The context vocabulary w(t-2), w(t-1), w(t+1) and w(t+2) of the target word can be predicted by the continuous skip-gram model (i.e., the Skip-Gram model). The weight matrix of the mapping relationship between the input and output of the Skip-Gram model can be obtained by predicting the context vocabulary of the Skip-Gram model. The weight matrix can represent the vector of each message node, that is, the node vector can be obtained by the weight matrix; the node vector is input into the neural network model so that the neural network model classifies the business message based on the node vector and outputs the category of the business message. The category of the business message can reflect the relationship between the business messages to a certain extent. For example, the category of the business message can include the original business type and the subsequent business type. The subsequent business type can realize a certain business function together with the original business type.

[0064] The multi-layer message node network obtained by using the feature knowledge graph can understand the complex relationship between different business messages, support association methods with no primary key, single primary key or multiple primary keys, and combine graph embedding technology and deep learning algorithms such as neural network models to identify the categories of transaction messages, effectively improving the accuracy of transaction message category identification.

[0065] In some embodiments, based on the service message and its category, a category vector corresponding to the service message category is obtained, thereby classifying the service message into at least one target service message family according to the category vector. FIG6 is a flowchart of a test method for an online system provided in another embodiment of the present application. FIG6 differs from FIG1 in that step S103 in FIG1 can be specifically refined into steps S1031 and S1032 in FIG5.

[0066] In step S1031 , based on the service message and the category of the service message, a category vector corresponding to the category of the service message is determined.

[0067] In some examples, the business message and the category of the business message can be processed by the category vector learning model to obtain the category vector corresponding to the category of the business message. The category vector learning model can be trained in advance based on a known business message family, and the category vector of the category of the business message can be obtained by learning the business message family through the category vector learning model. In other examples, the category vector corresponding to the category of the currently given business message can be found in the category vector corresponding to the category of the business message obtained in advance by the category vector learning model based on the known business message family. In some other examples, the business message family division model itself also has the function of generating vectors, and the category vector corresponding to the category of the business message can be obtained by processing the business message and the category of the business message within the business message family division model.

[0068] In step S1032, the service packets are divided into target service packet families including sorted service packets according to the category vectors corresponding to the service packets using the service packet family classification model.

[0069] The category vector corresponding to the category of the business message is input into the business message family division model, and the business message family division model processes the category vector corresponding to the category of the business message, thereby dividing the business message into the corresponding target business message family. Different business messages may be divided into the same target business message family, or may be divided into different target business message families. The business message family division model can not only divide the business messages into the corresponding target business message family, but also ensure that the business messages in the target business message family are arranged in the business order, so that the business messages can be sent to the test application according to the test strategy to simulate various test scenarios. In some examples, the business message family division model may include but is not limited to the transformer model.

[0070] Before using the business message family partitioning model to partition the business messages into target business message families, the business message family partitioning model may be trained first. Specifically, the business message families known in the production environment may be input into the category vector learning model, and the category vector corresponding to the category of the business message is obtained based on the business messages in the known business message family, the category of the business message, and the weight matrix in the category vector learning model; the business message family partitioning model is trained using the category vector corresponding to the category of the business message in the known business message family until the training result meets the preset training requirements. In some examples, the category vector learning model may include but is not limited to the word2vec model. The category vector can characterize the relationship between the category of the business message and the business message family.

[0071] The weight matrix between the input layer and the output layer can be calculated through the category vector learning model. This weight matrix can represent the weights between the input nodes and the hidden layer nodes in the category vector learning model. The vectors of the hidden layer nodes can be calculated based on the weight matrix, and the category vector of the service message can be obtained based on the vectors of the hidden layer nodes. For example, if the known service message family input to the category vector learning model is W, the capacity of the service message family is V, and the hidden layer size of the input category vector learning model is N, then the category vector of the service message can be obtained according to the following equations (4) to (6):

[0072] Among them, M is the weight matrix of the category vector learning model. Each row of the matrix represents an N-dimensional vector related to the input layer business, denoted as V M ;M vn The weight between the vth input node and the nth hidden layer node of the category vector learning model; h i is the i-th hidden layer node of the category vector learning model; h is the vector of the hidden layer of the category vector learning model, that is, the category vector of the business message; X is the vector matrix of the input layer node; x i is the vector representation of the input layer.

[0073] The service message family classification model is trained using the category vector. Whether to stop training can be determined by maximizing the likelihood function to obtain the service message family classification model of the training number. The preset training requirements may include but are not limited to maximizing the likelihood function to reach a maximum value. The maximization likelihood function can be referred to the following formula (7): L(D) = ∑logP(c|m1,m2,...,m n ) (7)

[0074] in, is the maximum likelihood function; c is a constant; m i is the category vector.

[0075] The service message family classification model can classify service messages into corresponding service message families based on the relationships between the learned service message categories and service message families. Even if new message categories or external interaction interfaces are added, resulting in new service message categories, the service message family classification model can automatically learn and accurately classify new service message categories into corresponding service message families, achieving paired service message orchestration.

[0076] For example, Figure 7 is a schematic diagram of an example of training a business message segmentation model in an embodiment of the present application. As shown in Figure 7, a known business message family is input into the word2vec model to obtain a category vector of the business message category, and the category vector is input into the transformer model for training, so that the trained transformer model can output the target business message family. For specific content, please refer to the relevant description in the above embodiment, which will not be repeated here.

[0077] In an embodiment of the present application, deep learning algorithms such as word vector technology and transformer models are used to predict the combination and sequence of business messages, and the received business messages are processed into a target business message family with time-series associations, so as to achieve transmission control and business replay in the target business message family under the test scenario. In the case of new message categories or new external interactive interfaces, business messages can be automatically divided into target business message families to improve test efficiency. By automatically learning the relationship between new message categories and business message families, manual omissions can be avoided, thereby improving test accuracy.

[0078] In some embodiments, after obtaining the target business message family, the test scenario can be simulated by the set test strategy for testing. The business messages in the target business message family can be sent to the test application in a predetermined order according to the business message sending conditions indicated by the set test strategy. Through the setting of the test strategy, a variety of business scenarios can be simulated and identified, such as, but not limited to, business message delayed sending scenarios, subsequent business messages and original business message timing control scenarios, business message timeout scenarios, concurrent initiation scenarios of unrelated business messages, business message category matching response scenarios, conditional subsequent business message initiation scenarios, business message TPS control scenarios, offline business message replay scenarios, etc. The business message sending conditions may include business message sending time, business message sending order, etc., and the conditions related to business message sending are all within the protection scope of the embodiments of the present application. For example, the sending time of at least one business message in the target business message family can be delayed, thereby simulating a business message delayed sending scenario; the subsequent business messages in the target business message family can be sent first, and then the original business messages in the target business message family can be sent, thereby simulating a scenario in which the timing of subsequent business messages and original business messages is reversed; the sending time of each business message in the target business message family can be controlled, and the TPS of the business message can be adjusted, thereby simulating a business message TPS control scenario. Examples are not given here one by one.

[0079] In some examples, a first processing result obtained by processing the target business message family through the test application and a second processing result obtained by processing the target business message family through the production environment can be obtained; the first processing result and the second processing result are compared to obtain the test result. The test application can realize the functions of the updated and upgraded online system, and the production environment can be an online system that has not been updated and upgraded but is in normal use. By comparing the first processing result and the second processing result, it can be determined whether the updated and upgraded online system can accurately complete the business completed by the production environment. If the first processing result is the same as the second processing result, it means that the updated and upgraded online system can accurately complete the business completed by the production environment, and the reliability of the updated and upgraded online system is high; if the first processing result is different from the second processing result, it means that the updated and upgraded online system cannot accurately complete the business completed by the production environment, and the reliability of the updated and upgraded online system is poor and needs further adjustment.

[0080] The testing method for an online system provided in the embodiment of the present application can be implemented through several parts, such as a model training function module, a feature engineering function module, a message calculation function module, and a message processing function module. To facilitate understanding, the application system architecture of the testing method for an online system in the embodiment of the present application is illustrated below with an example. FIG8 is a schematic diagram of an example of the application system architecture of the testing method for an online system provided in the embodiment of the present application. As shown in FIG8 , the application system architecture may include a model training function module 21, a feature engineering function module 22, a message calculation function module 23, and a message processing function module 24.

[0081] The training function module 21 and the feature engineering function module 22 can be deployed in the production area, which includes the production environment. The message calculation function module 23 and the message processing function module 24 can be deployed in the test area, which includes the test environment. The online application 31 can generate business messages, which can flow to the feature engineering function module 22 through the diversion front 32 in the form of data traffic. The business messages are processed by the model training function module 21 and the feature engineering function module 22. The model training function module can construct the association relationship between business messages for the business messages, and the association relationship between business messages can be represented by a multi-layer message node network. The feature engineering function module 22 can extract the features of the business messages, such as obtaining the node vector of the message node corresponding to the business message and obtaining the category of the business message. The features of the business messages extracted by the feature engineering function module 22 can form a feature library 33. The service messages generated by the online application 31 can flow as data traffic through the diversion front-end 34 to the message processing module 24 in the test area. The service messages are processed by the message calculation module 23 and the message processing module 24. The message calculation module 23 can classify the service messages into target service message families based on the data in the feature library 33. The message processing module 24 can perform functions such as message loading, message arrangement, message transmission and reception, and result comparison. Specifically, message arrangement involves determining the transmission time and order of service messages within the target service message family according to the set test strategy to simulate test scenarios. The message processing module 24 uses the message transmission and reception function to send service messages to the test application 35 and also receives response messages from the test application 35. The message processing module 24 uses the result comparison function to compare the service message processing results in the production area with those of the test application 35, thereby helping to determine whether the updated online system corresponding to the test application is suitable for use. The production database 36, located in the production area, can store the results of service message processing in the production area. The test database 37, located in the test area, can store the results of service message processing by the test application 35. The comparison database 38, located in the test area, can store the comparison results of service message processing in the production area and the results of service message processing by the test application.

[0082] A second aspect of the present application provides a testing device for an online system. FIG9 is a schematic diagram of the structure of a testing device for an online system according to an embodiment of the present application. As shown in FIG9 , the testing device 400 for an online system may include a network construction module 401 , a category determination module 402 , a message family classification module 403 , and a sending module 404 .

[0083] The network construction module 401 may be used to construct a multi-layer message node network based on the acquired service message features. The multi-layer message node network includes message nodes and connection edges between message nodes.

[0084] The category determination module 402 may be configured to obtain the category of the service message based on a multi-layer message node network using graph embedding technology and a neural network model.

[0085] The message family division module 403 can be used to obtain at least one target business message family based on business messages, business message categories and a pre-trained business message family division model. The business message family division model is trained based on known business message families in the production environment. The business message family is a collection of business messages of categories required to implement business functions.

[0086] The sending module 404 may be configured to send service messages in the target service message family to the test application according to a set test strategy, so that the test application processes the service messages in the target service message family.

[0087] In an embodiment of the present application, a multi-layer message node network can be constructed based on the service message characteristics of the service messages. The multi-layer message node network can reflect the association relationship between service messages based on the service message characteristics. Based on the multi-layer message node network, the categories of service messages can be obtained using graph embedding technology and neural network models. The service messages can be divided into at least one target service message family using the service message family partitioning model. The service messages in the target service message family are sent to the test application according to the set test strategy, so that the test application processes the service messages in the target service message family, thereby realizing the test. The business message family division model is a model that can automatically learn the relationship between the categories of business messages and business message families. Even when new message categories or new external interaction interfaces are added, resulting in new categories of business messages, the business message family division model can learn the association between the new categories of business messages and business message families, thereby automatically dividing the new categories of business messages into corresponding target business message families. This process does not require manual development and configuration work for the new message categories or new external interaction interfaces, reducing the development workload of online system testing and improving the testing efficiency and accuracy of the online system.

[0088] In some embodiments, the network construction module 401 can be specifically used to: perform dimensionality reduction processing on the business message features of multiple dimensions of the business message to obtain the business core features; generate a multi-layer business message feature network based on the business core features and the values ​​of the business core features of the business message, each layer in the multi-layer business message feature network includes message nodes and feature value nodes corresponding to the business core features, and the message nodes whose business core features have values ​​represented by the feature value nodes are connected to the corresponding feature value nodes; based on the multi-layer business message feature network, a multi-layer message node network is obtained.

[0089] In some examples, the network construction module 401 can be specifically used to: connect the message nodes connected to the same feature value nodes in the same layer of the multi-layer business message feature network, and delete the feature value nodes; connect the same message nodes in different layers of the multi-layer business message feature network to obtain a multi-layer message node network.

[0090] In some examples, the weight of a connection edge between two message nodes in a multi-layer message node network is obtained based on the message nodes connected to each of the two message nodes.

[0091] In some embodiments, the category determination module 402 can be specifically used to: start from the message nodes in the multi-layer message node network, and obtain the most probable wandering path of each message node; obtain the node vector corresponding to each message node based on the most probable wandering path of each message node and the continuous hopping model; input the node vector corresponding to the message node into the neural network model to obtain the category of the business message represented by the message node.

[0092] In some embodiments, the message family division module 403 can be specifically used to: determine the category vector corresponding to the category of the business message based on the business message and the category of the business message; and divide the business message into a target business message family including sorted business messages according to the category vector corresponding to the category of the business message through the business message family division model.

[0093] In some examples, the online system testing device 400 may further include a training module. The training module may be configured to: input a known service packet family in a production environment into a category vector learning model; obtain a category vector corresponding to the service packet category based on the service packets in the known service packet family, the service packet category, and a weight matrix in the category vector learning model; and train the service packet family classification model using the category vectors corresponding to the service packet categories in the known service packet family until the training results meet preset training requirements.

[0094] In some embodiments, the sending module 404 may be specifically configured to send the service messages in the target service message family to the test application in a predetermined order according to the service message sending conditions indicated by the set test policy.

[0095] In some embodiments, the online system testing device 400 may further include a comparison module. The comparison module may be configured to: obtain a first processing result obtained by the test application processing the target service message family and a second processing result obtained by processing the target service message family in the production environment; and compare the first processing result with the second processing result to obtain a test result.

[0096] FIG10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in FIG10 , the electronic device 500 includes a memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0097] In some examples, the processor 502 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0098] The memory 501 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the testing method of the online system according to the embodiment of the present application.

[0099] The processor 502 reads the executable program code stored in the memory 501 to run a computer program corresponding to the executable program code, so as to implement the online system testing method in the above embodiment.

[0100] In some examples, the electronic device 500 may further include a communication interface 503 and a bus 504. As shown in FIG10 , the memory 501, the processor 502, and the communication interface 503 are connected via the bus 504 and communicate with each other.

[0101] The communication interface 503 is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. Input devices and / or output devices can also be connected through the communication interface 503.

[0102] The bus 504 includes hardware, software, or both that couples the components of the electronic device 500 to each other. By way of example and not limitation, the bus 504 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of the above. Where appropriate, the bus 504 may include one or more buses. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0103] In a fourth aspect, the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the online system testing method described in the above embodiment can be implemented, and the same technical effects can be achieved. To avoid repetition, the details are not described here. The computer-readable storage medium may include a non-transitory computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., but is not limited thereto.

[0104] The fifth aspect of the present application provides a computer program product, which, when executed by a processor, implements the online system testing method in the above-mentioned embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0105] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For device embodiments, equipment embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description section of the method embodiment. This application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications and additions, or change the order of the steps after understanding the spirit of this application. In addition, for the sake of brevity, a detailed description of known method technologies is omitted here.

[0106] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.

[0107] Those skilled in the art should understand that the above embodiments are illustrative rather than restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, the specification and the claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other devices or steps; the quantifier "one" does not exclude a plurality; the terms "first" and "second" are used to identify names rather than to indicate any specific order. Any figure marks in the claims should not be understood as limiting the scope of protection. The functions of multiple parts appearing in the claims can be implemented by a separate hardware or software module. The fact that certain technical features appear in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A method for testing an online system, comprising: Building a multi-layer message node network based on the acquired service message features, wherein the multi-layer message node network includes message nodes and connection edges between message nodes; According to the multi-layer message node network, using graph embedding technology and neural network model, obtaining the category of the service message; Obtain at least one target service message family based on service messages, service message categories, and a pre-trained service message family classification model, wherein the service message family classification model is trained based on known service message families in a production environment, and a service message family is a collection of service messages of categories required to implement service functions; According to the set test strategy, the service messages in the target service message family are sent to the test application, so that the test application processes the service messages in the target service message family.

2. The method according to claim 1, wherein The step of constructing a multi-layer message node network based on the acquired service message features includes: Perform dimensionality reduction processing on the business message features of multiple dimensions to obtain the core business features; Generate a multi-layer service message feature network based on the service core feature and the value of the service core feature of the service message, wherein each layer of the multi-layer service message feature network includes a message node and a feature value node corresponding to the service core feature, and the message node whose service core feature has a value represented by the feature value node is connected to the corresponding feature value node; The multi-layer message node network is obtained according to the multi-layer service message feature network.

3. The method according to claim 2, wherein: The step of obtaining the multi-layer message node network according to the multi-layer service message feature network includes: Connecting message nodes connected to the same feature value node in the same layer in the multi-layer service message feature network, and deleting the feature value node; The same message nodes in different layers of the multi-layer service message feature network are connected to obtain the multi-layer message node network.

4. The method according to claim 2, wherein: The weight of the connection edge between two message nodes connected in the multi-layer message node network is obtained based on the message nodes connected to each of the two message nodes.

5. The method according to claim 1, wherein The method of obtaining the category of the service message based on the multi-layer message node network using graph embedding technology and a neural network model includes: Starting from the message nodes in the multi-layer message node network, respectively, obtaining a wandering path with the highest probability for each message node; According to the most probable wandering path and continuous hopping model of each message node, the node vector corresponding to each message node is obtained; The node vector corresponding to the message node is input into the neural network model to obtain the category of the service message represented by the message node.

6. The method according to claim 1, wherein The step of obtaining at least one target service message family based on the service message, the service message category, and the pre-trained service message family classification model includes: Determine, based on the service message and the category of the service message, a category vector corresponding to the category of the service message; The service message family classification model is used to classify the service messages into the target service message family including the sorted service messages according to the category vectors corresponding to the categories of the service messages.

7. The method according to claim 1, further comprising: Inputting a known service message family in the production environment into a category vector learning model, and obtaining a category vector corresponding to the category of the service message based on the service messages in the known service message family, the category of the service message, and the weight matrix in the category vector learning model; The service message family classification model is trained using category vectors corresponding to categories of service messages in a known service message family until a training result meets a preset training requirement.

8. The method according to claim 1, wherein The sending of the service message in the target service message family to the test application according to the set test strategy includes: According to the service message sending conditions indicated by the set test policy, the service messages in the target service message family are sent to the test application in a predetermined order.

9. The method according to claim 1, further comprising: Obtaining a first processing result obtained by processing a target service message family by a test application and a second processing result obtained by processing the target service message family in a production environment; Compare the first processing result and the second processing result to obtain a test result.

10. A testing device for an online system, comprising: A network construction module, configured to construct a multi-layer message node network based on the acquired service message features, wherein the multi-layer message node network includes message nodes and connection edges between message nodes; A category determination module, configured to obtain the category of the service message based on the multi-layer message node network using graph embedding technology and a neural network model; A message family classification module is configured to obtain at least one target service message family based on service messages, service message categories, and a pre-trained service message family classification model. The pre-trained service message family classification model is trained based on known service message families in a production environment. A service message family is a collection of service messages of the categories required to implement service functions. The sending module is used to send the service messages in the target service message family to the test application according to the set test strategy, so that the test application processes the service messages in the target service message family.

11. An electronic device comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the online system testing method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the online system testing method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the online system testing method according to any one of claims 1 to 9.

Images