Method and apparatus for verifying system performance

Abstract

The embodiment of the application provides a system performance verification method and device, wherein the method comprises the following steps: sending a synchronous acquisition instruction to a plurality of verification nodes, so that each verification node acquires state information corresponding to a real-time interactive event based on the synchronous acquisition instruction, and acquires a target vector clock stored by the each verification node; acquiring state information and a target vector clock carried in acquisition response information sent by the each verification node, and determining a partial order relation between two verification nodes in acquiring the state information corresponding to the real-time interactive event according to components in the target vector clock; and verifying a first performance of a target system according to the state information of the each verification node and the partial order relation between the two verification nodes in acquiring the state information corresponding to the real-time interactive event, wherein the target system is an interactive message system corresponding to the real-time interactive event.

Description

Technical Field

[0001] This application relates to the field of system testing, and more specifically, to a method and apparatus for verifying system performance. Background Technology

[0002] In existing testing technologies for live streaming interactive messaging systems, when faced with user behaviors (such as barrage of comments, gift bombardment, and special effects triggering) under the concurrent access of millions of users, state consistency verification has long relied on simple timestamp comparison or eventual consistency judgment, which cannot accurately determine the state consistency of the interactive messaging system.

[0003] Regarding the issue of how to verify the state consistency of an interactive messaging system under high concurrency of user behavior data, no effective solution has yet been proposed.

[0004] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0005] This application provides a system performance verification method and apparatus to at least solve the problem in the related art of how to verify the state consistency of an interactive message system under high concurrency of user behavior data.

[0006] According to one embodiment of this application, a method for verifying system performance is provided, comprising: sending a synchronous acquisition command to multiple verification nodes, so that each verification node acquires state information corresponding to a real-time interactive event based on the synchronous acquisition command, and acquires a target vector clock stored by each verification node; acquiring the state information and target vector clock carried in the acquisition response information sent by each verification node, and determining a partial order relationship between pairs of verification nodes in acquiring the state information corresponding to the real-time interactive event based on the components in the target vector clock; verifying a first performance of a target system based on the state information of each verification node and the partial order relationship between pairs of verification nodes in acquiring the state information corresponding to the real-time interactive event, wherein the target system is an interactive message system corresponding to the real-time interactive event.

[0007] According to another embodiment of this application, a system performance verification device is provided, comprising: a sending module, configured to send a synchronous acquisition instruction to multiple verification nodes, so that each verification node acquires state information corresponding to a real-time interactive event based on the synchronous acquisition instruction, and acquires a target vector clock stored by each verification node, wherein each component of the target vector clock is used to indicate the number of state update information sent by the corresponding server instance to each verification node; an acquisition module, configured to acquire the state information and target vector clock carried in the acquisition response information sent by each verification node, and determine the partial order relationship of the state information acquired by each pair of verification nodes corresponding to the real-time interactive event based on the components in the target vector clock; and a verification module, configured to verify a first performance of a target system based on the state information of each verification node and the partial order relationship of the state information acquired by each pair of verification nodes corresponding to the real-time interactive event, wherein the target system is an interactive message system corresponding to the real-time interactive event.

[0008] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0009] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0010] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0011] This application achieves the following: During testing, a synchronous acquisition command is sent to multiple verification nodes. Each verification node acquires the status information corresponding to the real-time interactive event based on the synchronous acquisition command. Simultaneously, each verification node acquires the vector clock most recently updated by the server instance after the status update message. The components in the target vector clock determine the partial order relationship between the status information acquired by each pair of verification nodes corresponding to the real-time interactive event. By comparing the partial order relationship between the status information acquired by each pair of verification nodes corresponding to the real-time interactive event and the status information, the consistency of the interactive message system's status is verified (i.e., the first performance), thus solving the problem of accurately identifying the status consistency of the interactive message system in high-concurrency scenarios. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0014] Figure 1 This is a hardware structure block diagram of a computer device for a system performance verification method according to an embodiment of this application.

[0015] Figure 2 This is a flowchart of a system performance verification method according to an embodiment of this application;

[0016] Figure 3 This is a flowchart of a method for determining a behavior pattern according to an embodiment of this application;

[0017] Figure 4 This is a flowchart of a state change verification method according to an embodiment of this application;

[0018] Figure 5 This is a flowchart of a method for simulating a multi-level network according to an embodiment of this application;

[0019] Figure 6 This is a flowchart of a fault recovery assessment method according to an embodiment of this application;

[0020] Figure 7 This is a structural block diagram of a system performance verification device according to an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0023] The methods and embodiments provided in this application can be executed in a computer device or similar computing device. Taking running on a computer device as an example, Figure 1 This is a hardware structure block diagram of a computer device for a system performance verification method according to an embodiment of this application. For example... Figure 1 As shown, a computer device may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor (MPU) or a programmable logic device (PLD)) and a memory 104 for storing data are also shown. The computer device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer device described above. For example, the computer device may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0024] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the system performance verification method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to computer devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0025] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer equipment. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0026] This embodiment provides a system performance verification method, applied to the aforementioned computer terminal. Figure 2 This is a flowchart of a system performance verification method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0027] Step S202: Send a synchronization acquisition command to multiple verification nodes so that each verification node can acquire the status information corresponding to the real-time interactive event based on the synchronization acquisition command, and acquire the target vector clock stored by each verification node.

[0028] It should be noted that, in this embodiment, the aforementioned real-time interactive events can be understood as live streaming events. During the continuous operation of the simulated high-concurrency interactive events, a central control server periodically sends synchronization collection instructions to the deployed verification nodes (e.g., interfaceless verification clients). These synchronization collection instructions carry timestamps, requiring all verification nodes to simultaneously (e.g., once per second) obtain a locally updated copy of the live stream state and record the corresponding state snapshot and current vector clock value. Each verification node subscribes to an MQTT topic, receives all interactive message streams, and updates the live stream state information locally in real time. This state information includes, but is not limited to: total accumulated gift value, live stream popularity value, activated special effects ID, fan club level, and number of online viewers.

[0029] Each time a verification node processes a state update from a server instance, it updates the target vector clock. The target vector clock is an integer array of length N, where N is the number of background service instances participating in the state update (e.g., if the system deploys a total of 6 Broker instances, then N=6). The i-th component in the array represents the number of state update messages sent by the i-th server instance (that is, the number of state update messages sent by the i-th service instance that the verification node has received and processed).

[0030] It should be noted that when the server instance sends out state update information, it embeds a vector clock in the message payload. This vector clock is a logical clock used to capture the happened-before relationship between events.

[0031] When a verification node receives a data collection command, it obtains a copy of its current state and sends the state information and the corresponding vector clock to the control server via a secure channel. For example, at 7 minutes and 32 seconds into the test, the control server sends a synchronous data collection command, and all 12 verification nodes simultaneously return snapshot data: Verification node 1 records a cumulative gift of 987,652 yuan, a popularity value of 2,105,300, and an effect ID of "Starry River Blooming," with its vector clock being [1542,1540,1543,1539,1541,1542]; while verification node 2, due to higher network latency, records a cumulative gift of 987,501 yuan, a popularity value of 2,105,000, and the same effect ID, with its vector clock being [1541,1539,1542,1538,1540,1541].

[0032] Step S204: Obtain the status information and target vector clock carried in the acquisition response information sent by each verification node, and determine the partial order relationship of the status information corresponding to the real-time interactive event between each pair of verification nodes based on the components in the target vector clock.

[0033] In step S204, the snapshot of the live room's business status and its corresponding vector clock recorded by each distributed verification node at the time of synchronous acquisition are extracted from the acquisition response information returned by that node. The status information includes key business indicators such as popularity value, special effects activation status, fan group level, and total accumulated gifts. By comparing the magnitude relationship of each component in the vector clocks carried by any two verification nodes, the causal order formed when these two nodes receive real-time interactive events and update their states is determined: if all components in node A's vector clock are not less than the corresponding components in node B, and at least one component is strictly greater, then node A's state update occurs after node B, and there is a clear causal partial order between the two; if there are components in the two vector clocks that are mutually greater, then the two are considered concurrent events, and their state difference belongs to the temporary inconsistency allowed by the system.

[0034] Step S206: The partial order relationship between the two verification nodes to obtain the state information corresponding to the real-time interactive event verifies the first performance of the target system, wherein the target system is the interactive message system corresponding to the real-time interactive event.

[0035] After receiving the acquisition response information, the state information of each verification node is aligned and compared with the vector clock to determine the first performance of the target system. The first performance may include, but is not limited to, state consistency.

[0036] Through the above steps, during the testing process, synchronous acquisition instructions are sent to multiple verification nodes. Each verification node obtains the status information corresponding to the real-time interactive event based on the synchronous acquisition instructions. At the same time, each verification node obtains the vector clock most recently updated by the server instance after the status update message. The components in the target vector clock determine the partial order relationship between the status information obtained by each pair of verification nodes corresponding to the real-time interactive event. By comparing the partial order relationship between the status information obtained by each pair of verification nodes corresponding to the real-time interactive event and the status information, the consistency of the interactive message system's status is verified (i.e., the first performance), thus solving the problem of accurately identifying the status consistency of the interactive message system in high-concurrency scenarios.

[0037] Optionally, verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between pairs of verification nodes includes: determining whether the partial order relationship between the target vector clocks of pairs of verification nodes conforms to a preset partial order relationship; if the partial order relationship between the target vector clocks of pairs of verification nodes conforms to the preset partial order relationship, determining whether there is a difference in the state information of the pairs of verification nodes; if there is no difference in the state information of the pairs of verification nodes, determining that the first performance of the target system is normal; if there is a difference in the state information of the pairs of verification nodes, determining that the first performance of the target system is abnormal; if the partial order relationship between the target vector clocks of pairs of verification nodes does not conform to the preset partial order relationship, determining whether the state difference information of the state information of the pairs of verification nodes conforms to a preset state difference standard; if the state difference information of the state information of the pairs of verification nodes conforms to the preset state difference standard, determining that the first performance of the target system is normal; if the state difference information of the state information of the pairs of verification nodes does not conform to the preset state difference standard, determining that the first performance of the target system is abnormal.

[0038] In this embodiment, the vector clocks of any two verification nodes are compared pairwise. If all components in the vector clock of node A are greater than or equal to the corresponding components of node B, then it is determined that the state of node A precedes that of node B. At this time, there is a causal partial order relationship between node A and node B, and the causal partial order relationship conforms to the preset partial order relationship, that is, the state updates have a clear order. Conversely, if the components of the two vector clocks are interleaved (e.g., the first component of A is higher, and the third component of B is higher), then it is determined that the two are concurrent events, and the partial order relationship does not conform to the preset partial order relationship, that is, the state updates do not have a clear order.

[0039] If the partial order relation conforms to the preset partial order relation, the state information of the two nodes is further compared. If the two are consistent in key state variables (such as total cumulative gift amount, live stream popularity value, and special effect activation status), the state is determined to be consistent, and the first performance is "normal". For example, at 6 minutes and 45 seconds of the test, the vector clocks of the Shanghai node and the Guangzhou node show that node 1 is [1582,1580,1583,1581,1582,1581] and node 2 is [1581,1580,1582,1580,1581,1581]. All components of node 1 are not less than those of node 2, and the first and third components are larger, which meets the standard. At the same time, the states of both locations show a cumulative gift amount of 987,600 yuan and a popularity value of 2,105,000, which determines that this synchronization is normal and there is no state drift.

[0040] If the partial order relationship does not conform to the preset partial order relationship, but there are differences in the status information (such as a difference of 2,000 yuan in gift value), the system will determine it as abnormal.

[0041] When the partial order relationship between two nodes does not conform to the preset partial order relationship, the absolute difference of the key state variables between the two nodes is calculated and compared with the preset threshold. For example, the preset thresholds are "cumulative gift value difference tolerance threshold δ = 5000 yuan", "popularity value difference tolerance threshold δ = 2000", and "special effect IDs must be completely identical". If the gift value of node 1 is 987800 yuan and that of node 2 is 987200 yuan, the difference is 600 yuan, which is less than 5000 yuan; the difference in popularity value is 1500, which is lower than the 2000 threshold; and the special effect IDs are identical, then the difference is determined to be within an acceptable range, and the first performance is still "normal".

[0042] Conversely, if the state difference exceeds the threshold, such as if node 2 fails to receive the "gifts greater than one million" event in time, its cumulative gift value only shows 975,000 yuan, while node 1 has reached 988,000 yuan, with a difference of 13,000 yuan, then it is determined to be "concurrent state drift". If the system's state synchronization mechanism is found to have a problem, then the first performance is determined to be "abnormal".

[0043] Optionally, before verifying the first performance of the target system based on the status information of each verification node and the partial order relationship between the status information corresponding to the real-time interactive events obtained by each pair of verification nodes, the method further includes: acquiring historical log data in the production environment corresponding to the real-time interactive events, and clustering the target objects in the production environment based on the historical log data to determine different types of target objects; determining the parameter values ​​of the interaction parameters of the different types of target objects, wherein the interaction parameters include at least one of the following: interaction timing parameters, interaction type parameters, interaction value parameters, and interaction concentration parameters; generating behavior parameter configuration files for the different types of target objects based on the parameter values ​​of the interaction parameters of the different types of target objects; generating interaction event sequences for the different types of target objects based on the behavior parameter configuration files, and running the interaction event sequences for the different types of target objects to verify the second performance of the target system.

[0044] Specifically, when the aforementioned real-time interactive events are live events, the system obtains anonymized user interaction logs (i.e., historical log data) from the past period. These logs include: anonymous user IDs (anonymized and desensitized), interaction types (e.g., bullet comments, likes, gifts), interaction timestamps, interaction value (e.g., gift amount), user attributes (e.g., fan level, historical activity level), and session identifiers. Based on a weighted K-means clustering algorithm, users are categorized using average interaction frequency, the proportion of high-value interactions, and the concentration of interactions within a session as feature vectors, identifying role types such as "die-hard fans," "regular viewers," "lurkers," and "gift enthusiasts."

[0045] For each target object category, historical behavior is statistically modeled, and methods such as Maximum Likelihood Estimation (MLE) are used to fit the behavioral characteristics of each target object category, generating corresponding parameterized templates. Among these, the interaction time series parameter represents the probability distribution type (e.g., Poisson distribution, power-law distribution) and core parameters (e.g., λ) of the time intervals between interaction events; the interaction type parameter is a discrete probability distribution vector indicating the probability of actions such as sending comments, liking, and gifting; the interaction value parameter indicates the numerical distribution (e.g., log-normal distribution) and parameters of interactions involving economic value (e.g., gifts); and the interaction concentration parameter indicates the degree of clustering and burst tendency of interaction events in a single session over time.

[0046] The parameterized results (i.e., the parameter values ​​of the interaction parameters) of the above-mentioned different types of target objects are encapsulated into standardized, machine-readable JSON format parameter configuration files. Each parameter configuration file contains a complete behavioral model definition, including: distribution type, parameter values, goodness-of-fit index, and applicable scenario description. The parameter configuration file serves as the basis for behavioral decisions of virtual user instances.

[0047] Based on the behavioral parameter configuration files of different types of target objects, interactive event time series conforming to statistical distribution are generated through Monte Carlo sampling. Based on the Hawkes self-excitation process, the impulse effect caused by hot events (such as a 300% surge in interaction during the climax of a program) and the group propagation effect based on social network topology are simulated to generate interactive event sequences with timestamps, user identifiers, interaction types and values. The generated interactive event sequences are then injected into the interactive messaging system under test as stress test data input.

[0048] Through the above steps, an interactive event sequence that closely resembles real user types is generated. Compared to user interaction events generated based on uniform distribution or static rules in related technologies, the interactive event sequences generated in this embodiment are highly consistent with real user behavior, effectively solving the problem of how to construct a real user behavior model and simulate interaction impulse characteristics and group propagation effects in related technologies.

[0049] In one exemplary embodiment, a method for generating an interaction event sequence of different types of target objects based on the behavior parameter configuration file is provided, comprising: sampling parameter information in the behavior parameter configuration file using a target sampling algorithm to generate a first interaction event sequence of the different types of target objects; determining a second interaction event sequence based on a conditional intensity function corresponding to the different types of target objects when a preset event exists in the real-time interaction events, wherein the conditional intensity function is the response intensity of the different types of target objects to the preset event; and generating the interaction event sequence according to the first interaction event sequence and the second interaction event sequence.

[0050] In this embodiment, a target sampling algorithm is used to randomly sample preset parameter information in the behavior parameter configuration file to generate a basic interaction event sequence for each target object, namely the first interaction event sequence. The target sampling algorithm generates statistically consistent interaction events one by one on the timeline based on the interaction time distribution of different types of target objects (e.g., hardcore fans follow a Poisson distribution, λ=2.5 times / minute), interaction type probability vectors (e.g., gift-giving probability is 30%, bullet comments 60%, likes 10%), and interaction value distribution (e.g., gift amount follows a log-normal distribution, μ=2.1, σ=0.8). For example, a virtual user who is a "hardcore fan" will automatically trigger approximately 25 interactions according to the corresponding Poisson distribution parameters within a 10-minute test period, of which approximately 7 will be gift-giving behaviors, with the amount of each gift concentrated in the range of 8 to 120 yuan, forming a basic interaction event sequence that conforms to historical behavioral characteristics.

[0051] When a pre-set high-impact event (i.e., a pre-set event) is triggered in a real-time interactive event (such as "the final guest appears," "the host announces a giveaway," or "the live stream's popularity exceeds ten million"), a second sequence of interactive events will be dynamically generated based on a conditional strength function for different types of target objects. This conditional strength function is defined as follows: ,in, This represents the base interaction strength of this type of target object under normal conditions. For the excitation kernel function (e.g., (tt i )=α exp(-β(tti ))) indicates the reinforcing effect of past events on the present moment.

[0052] Finally, the first and second interactive event sequences are merged and superimposed to generate the final complete interactive event sequence. The rules for merging and superimposing the first and second interactive event sequences are as follows: during periods when no preset event is triggered, only the first interactive event sequence is retained; within the time window of the preset event, the second interactive event sequence is superimposed on the first interactive event sequence. Simultaneously, the system automatically removes duplicates and calibrates to ensure the continuity and uniqueness of the timestamp sequence. For example, in the test, a basic interactive stream of 100,000 users was first generated. When the "grand finale guest appearance" was triggered at the 8th minute, a high-density instantaneous gift-giving and bullet screen event was injected into 1,500 "die-hard fans" and 800 "gift enthusiasts" within 0.5 seconds, causing the overall message rate to surge from 4,500 messages / second to 15,600 messages / second within 3 seconds.

[0053] In an exemplary embodiment, determining a second interactive event sequence based on conditional intensity functions corresponding to the different types of target objects includes: determining the basic response intensity of the different types of target objects to the preset event based on a basic response intensity function, and determining the stimulus response intensity of the different types of target objects to the preset event based on an excitation kernel function, wherein the conditional intensity function includes the basic response intensity function and the excitation kernel function; determining the response intensity of the target object within a target time period based on the basic response intensity and the stimulus response intensity, wherein the target time period is the time period after the occurrence of the preset event; and generating the second interactive event based on the response intensity.

[0054] In this embodiment, a basic response intensity function is constructed for each type of target object based on its historical behavioral characteristics. This function represents the user's natural reaction ability when not affected by external events. For example, the basic response intensity μ(t) for "die-hard fans" is set to 2.8 interactions per minute, while for "ordinary viewers" it is only 0.6 interactions per minute.

[0055] Excitation kernel function Φ(tt) i ) adopts an exponential decay form: Φ(tt) i )=α×exp(-β×(tt i In the excitation kernel function, α is the excitation intensity coefficient, and β is the decay rate. Different types of target objects have different α and β parameters. For example, a "die-hard fan" has α=0.75 and β=0.9, meaning that the response peak is high and decays quickly after being stimulated by the event; while a "gift connoisseur" has α=1.2 and β=0.5, meaning that the excitation response is stronger and lasts longer. The input to the excitation kernel function is the timestamp t of the preset event.i After an event is triggered, the intensity of the stimulus response at the current time t is calculated in real time for each virtual user, which is the cumulative impact of all relevant past interactive events on the current behavior.

[0056] The baseline response intensity is then superimposed with the stimulus response intensity to obtain the comprehensive response intensity of the target object within the target time period after the event occurs. Finally, based on the comprehensive response intensity within the target time period (0-10 seconds after the event), a third interactive event sequence is dynamically generated using a Poisson process sampling method, and the second interactive events in the third interactive event sequence whose comprehensive response intensity is greater than the baseline threshold are retained.

[0057] Optionally, this application also provides a method for determining user behavior patterns. Figure 3 This is a flowchart of a method for determining a behavior pattern according to an embodiment of this application, such as... Figure 3 As shown, it includes:

[0058] Step S31: Obtain the anonymized user interaction event data;

[0059] User interaction event data includes: anonymized user ID (SHA-256 hash), interaction type (bullet comments, likes, gifts, etc.), timestamps accurate to milliseconds, interaction value (e.g., gift amount), user attributes (follower level, historical active days, session duration), and a unique session identifier. After data cleaning, invalid records (such as null values, abnormal timestamps, and non-live streaming events) are removed, and the data is aggregated by session dimension to form a structured behavior trajectory dataset.

[0060] Step S32: Classify the preprocessed dataset using the weighted K-means algorithm;

[0061] The user feature vector includes: average interaction frequency (weight 0.6), high-value interaction ratio (number of gift interactions / total number of interactions, weight 0.3), and in-session interaction concentration (standard deviation of interaction events per unit time, weight 0.1). The optimal number of clusters is determined to be 4 using the elbow rule. The clustering results output four typical user roles (i.e., different types of target objects): Hardcore Fans: high frequency, high value, high concentration; General Audience: medium frequency, low value, low concentration; Lurking Users: extremely low frequency, no value, random behavior; Gift Experts: low frequency, extremely high value, occasional bursts of activity.

[0062] Step S33: For each type of role, statistical modeling of the four interaction parameter dimensions is performed using the parameter fitting engine based on the maximum likelihood estimation method, and a standardized behavior profile is output.

[0063] The four interaction parameter dimensions include: Interaction Time Sequence Parameter: Fitting a time distribution model of the corresponding interaction interval for each type of role (e.g., Poisson distribution for hardcore fans; power-law distribution for lurking users); Interaction Type Parameter: Generating a discrete probability distribution vector, defining the probability of various interactive behaviors such as sending bullet comments, giving gifts, and liking (e.g., hardcore fans: 30% for bullet comments, 50% for gifts, 20% for likes); Interaction Value Parameter: Fitting a numerical distribution for value-based interactions, defining the probability distribution model (e.g., log-normal distribution) and parameters (e.g., log-normal distribution μ=2.1, σ=0.8); Interaction Concentration Parameter: Calculating the statistical measure (e.g., standard deviation threshold) of the degree of clustering and burst tendency of interactive events in a single session. All parameters are encapsulated into a JSON format behavior parameter configuration file after goodness-of-fit testing (e.g., KS test, AIC criterion).

[0064] Step S34: Generate and enhance the sequence of interactive events.

[0065] For each virtual user, a basic interactive event sequence (i.e., the first interactive event sequence) is generated based on the parameter library of their respective roles, using Monte Carlo sampling and the temporal and type distribution of each role.

[0066] When a pre-set "key event" (such as "guest appearance" or "lottery start") is triggered in the test scenario, the Hawkes self-stimulation process is activated to dynamically enhance the response strength of the affected role group. Its conditional strength function is calculated as follows:

[0067] Where μ(t) is the character's base strength, and α and β are the corresponding activation parameters for the character. The activation kernel function generates an exponentially decaying impulse response within 3 seconds after the event occurs, causing the interaction frequency to surge instantaneously (e.g., 320%).

[0068] If λ(t) ≤ the base threshold: use the base interaction event sequence; if λ(t) > the base threshold: superimpose new impulse events onto the base event sequence. Simultaneously, load a virtual social network topology (based on sampling from real follower relationships). When a key node generates a high-value interaction, trigger a chain reaction of its follower nodes according to the probability of link strength, simulating the "bandwagon effect." Finally, merge the base sequence and the enhanced sequence on the timeline to generate a complete MQTT message stream, which serves as input to the test system. The final output is a structured event sequence file with timestamps, user identifiers, interaction types, and interaction values, and is injected into the target system using the Kafka or MQTT protocol.

[0069] In the above embodiments, by constructing a behavior pattern simulator based on real user behavior profiles and Hawkes processes, a highly realistic interactive message stream with pulse characteristics and group propagation effects can be generated. This overcomes the limitations of related technologies that only simulate simple connections and throughput, enabling stress testing to accurately reproduce real business scenarios such as "a 300% surge in interaction within 3 seconds during a program's climax," thereby more effectively exposing the system's performance bottlenecks and logical defects under complex, sudden business loads.

[0070] In an exemplary embodiment, after verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive event obtained between each pair of verification nodes, the method further includes: when a preset business rule triggers the real-time interactive event, determining whether multiple verification nodes in a first time window have experienced an event corresponding to the preset business rule; when all multiple verification nodes in the first time window have experienced an event corresponding to the preset business rule, determining that the third performance of the target system is normal; and when no verification node in the first time window has experienced an event corresponding to the preset business rule, determining that the third performance of the target system is abnormal.

[0071] When the generated interactive event sequence reaches a preset business threshold (e.g., "cumulative gifts exceed 1 million yuan") during its execution, the preset business rule is automatically triggered, i.e., "activate advanced special effects". Within the first time window after the triggering event (e.g., ΔT=5 seconds), the verification nodes deployed in different regions are monitored to check whether each verification node observes the state change event (e.g., triggering special effects) corresponding to the business rule within this window.

[0072] For example, the generated interactive sequence triggers a special effect after virtual users have accumulated gifts totaling 1,000,500 yuan. The control server then starts a timer, entering a 5-second verification window. Within this window, verification nodes 1 and 2 successfully receive the state change event within 4.8 seconds, and their local state copies are synchronously updated to "Triggered Special Effect XXX," recording the corresponding timestamp and vector clock. However, due to the cumulative latency of cross-regional networks, verification node 3 does not receive the update message until the 5.3-second mark, exceeding the 5-second window. Therefore, within the first time window, verification node 3 fails to complete the response to the business rule. Consequently, the system determines the target system's third-party performance as "abnormal" and marks it as "Business rule propagation delay exceeds the standard."

[0073] Optionally, this application also provides a method for verifying state changes. Figure 4 This is a flowchart of a state change verification method according to an embodiment of this application, such as... Figure 4 As shown, it includes:

[0074] Step S41: Initialize the subscription relationship of multiple verification nodes in different regions so that each verification node subscribes to the MQTT topic corresponding to the target live room, so as to receive all interactive messages and status update events published by the server in real time.

[0075] After receiving the message stream, each verification node dynamically updates its local copy of the live stream status based on the message content, including key status variables such as the total value of accumulated gifts, the live stream popularity value, the currently activated special effects ID, the fan group level, and the number of online viewers.

[0076] When processing each state update message from the server, the verification node extracts the vector clock embedded in the message payload. The vector clock is an integer array of length N, where N is the number of service instances participating in the state sharding update. Each component records the event count processed by the corresponding service node.

[0077] Step S42: Based on the preset business rule base, monitor whether the status copy meets specific threshold conditions, such as "total value of gifts ≥ 1,000,000 yuan" or "popularity value ≥ 2,000,000", and mark it as a "trigger event" when the condition is met for the first time, and record the trigger timestamp T_trigger.

[0078] Step S43: The control server broadcasts a synchronization acquisition command to all verification nodes at a fixed period (e.g., once per second). The command carries a synchronization timestamp. After receiving the command, each node acquires a copy of its current state and the corresponding vector clock, and encapsulates them into an acquisition response packet.

[0079] Step S44: After receiving the synchronization acquisition instruction, each verification node sends the current state snapshot (including state variable values ​​and trigger event status) and the local vector clock to the control server.

[0080] Step S45: After receiving all acquisition responses, the control server aligns the state snapshots of all nodes according to the time sequence, and performs a partial order comparison of the vector clocks of each pair of nodes to determine whether they satisfy the three relationships of "comparable and equal", "comparable but unequal" or "incomparable (concurrent)".

[0081] Step S46: For the time T_trigger that triggers the state change corresponding to the business rule, within the time window of [T_trigger, T_trigger+ΔT] (ΔT is the preset maximum propagation delay, such as 5 seconds), check whether all verification nodes have observed the state change (such as special effect activation, level upgrade) corresponding to the business rule, and record the observation time of each node.

[0082] Step S47: If all verification nodes complete the state change within the time window and the vector clock relationship conforms to causal consistency, then the state transition verification is considered successful and the system's state transition consistency performance (i.e., the third performance) is "normal"; if any node fails to complete the state change within the window, or the vector clock shows that the state update is not correctly transmitted, then the state transition is considered to have failed and the system's state transition consistency performance is "abnormal".

[0083] Step S48: Generate a state transition verification report, including: the global consistency rate of the triggering event, the state propagation delay distribution, and the threshold trigger verification results (success / failure and timeline).

[0084] In the above embodiments, by deploying distributed verification nodes and combining them with a vector clock algorithm, the consistency drift of the client states in a distributed environment is continuously and accurately detected, and the correctness and timeliness of state transitions (such as threshold triggering) are objectively verified. This solves the problem of "lack of consistency verification for state transitions" in related technologies, and can detect and locate state asynchrony problems caused by message out-of-order, delay, or loss in advance, fundamentally reducing the risk of user experience complaints caused by this.

[0085] In an exemplary embodiment, after verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between pairs of verification nodes, the method further includes: generating a sending strategy for different regions based on network parameter configuration files corresponding to different regions, wherein the network parameter configuration files include at least one of the following: network latency, packet loss rate, and jitter; sending target state update information to the target verification nodes in the corresponding regions based on the sending strategy; and determining whether the target verification nodes in different regions receive the target state update information in a second time window; if the target verification nodes in different regions all receive the target state update information in the second time window, determining that the fourth performance of the target system is normal; if the target verification nodes in different regions do not receive the target state update information in the second time window, determining that the fourth performance of the target system is abnormal.

[0086] In this embodiment, pre-configured network parameter configuration files for different regions are loaded. These configuration files define differentiated network parameters for different geographical regions, including network latency, packet loss rate, and jitter. During the simulation test, status update information, such as message loads like "live stream popularity value updated to 2150000," is generated in real time based on the currently generated sequence of interactive events. Depending on the deployment region of the verification node, the corresponding network parameter configuration file is dynamically applied. Network traffic shaping devices (such as Linux tc and NetEm) are used to inject MQTT message streams destined for verification nodes in different regions: for message streams destined for node 1, an average one-way latency of 110ms, 1.8% random packet loss, and 35ms jitter are injected; for messages destined for node 2, a 20ms latency and slight jitter are injected.

[0087] Within the second time window (e.g., ΔT = 3 seconds), it is determined whether the verification node in each region has successfully received the corresponding state update information within this window. If all target verification nodes in all regions successfully receive the state update information within the second time window, the cross-domain state synchronization capability (i.e., the fourth performance) of the target system is "normal"; conversely, if any region fails to receive the update within the window, the cross-domain state synchronization capability is determined to be "abnormal," indicating that the system has a risk of unreliable state propagation under the corresponding network conditions, which may reduce the user experience.

[0088] Optionally, embodiments of this application provide a method for simulating multi-level networks. Figure 5 This is a flowchart of a multi-level network simulation method according to an embodiment of this application, such as... Figure 5 As shown, it includes:

[0089] Step S51: Load the preset national multi-level network parameter configuration file.

[0090] Step S52: Use traffic shaping devices (such as Linux tc, NetEm, or a dedicated network emulator) to perform precise delay injection, random packet loss simulation, and jitter perturbation on the MQTT message streams sent to the verification nodes in each region.

[0091] Step S53: After receiving the message processed by network impairment, each verification node updates its local copy of the live room status, including key business statuses such as total gift amount, popularity value, special effects activation status, and fan group level.

[0092] Step S54: For each important state update event (such as "gifts exceed one million trigger special effect"), start a consistency judgment window (e.g., W=2 seconds) from the time it occurs, and count whether all verification nodes have observed the same state change result within the window.

[0093] Step S55: For each state update event, calculate the number of verification nodes that reach consensus within the time window W, divide it by the total number of nodes, and obtain the "regional consistency rate" of the event; at the same time, for nodes that do not reach consensus, record their region, network parameter configuration and state difference (such as gift value difference, popularity deviation).

[0094] Step S56: Compare the regional consistency rate with a preset threshold (e.g., ≥95% is normal) to determine the system's cross-domain state synchronization capability.

[0095] If the regional consistency rate of all key events meets the standard and the state deviation is within the tolerance threshold (e.g., gift difference ≤ 5,000 yuan), then the cross-domain state synchronization capability (i.e., the fourth performance) of the system in this network environment is judged to be "normal"; if the consistency rate of any event is lower than the threshold, or there is state drift exceeding the tolerance threshold, then the cross-domain state synchronization capability of the system in this network environment is judged to be "abnormal".

[0096] In the above embodiments, by constructing multi-level network simulation environments in different regions, complex situations such as latency, packet loss, and jitter existing in the public network can be reproduced in a controlled test field. This not only allows for the evaluation of the system's eventual consistency achievement rate under different network conditions, but also enables rapid correlation of network monitoring data when synchronization problems occur, achieving precise localization from the "phenomenon" (inconsistent state) to the "root cause" (deterioration of a specific network link), providing direct evidence for system optimization and the formulation of operational and maintenance plans.

[0097] In an exemplary embodiment, after verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between each pair of verification nodes, the method further includes: in the event of a failure event, obtaining failure recovery information of the verification nodes and the server instance in the target system, wherein the failure event is a network interruption event between the server instance and the verification node; in the event that the network between the verification node and the server instance is determined to be restored, verifying the availability of the server instance; and determining the fifth performance of the target system based on the failure recovery information and the availability of the server instance.

[0098] In this embodiment, a predefined fault event is triggered, such as remotely executing a service termination command (e.g., `systemctl stop mosquitto`) via an operations and maintenance interface to simulate a sudden server instance crash. Furthermore, a network interruption policy (e.g., blocking TCP connections between the server instance and the Broker via iptables) can be injected into the verification nodes in the region covered by the server instance to create a "service unreachable + network interruption" fault scenario.

[0099] After fault injection, the connection status and message sending / receiving behavior of all verification nodes are continuously monitored. Simultaneously, on the server side, the health detection mechanism of the Broker cluster, the automatic takeover process of the standby node, and client reconnection attempt records are monitored. Once the standby Broker instance successfully takes over the load of the original master node and rebuilds the message queue and session state, it is determined that the network connection has been restored. Alternatively, the Broker fault can be manually canceled, and a probe request is sent to the recovered server instance to verify whether it can normally respond to core APIs such as message subscription, publishing, and status query. At the same time, by comparing snapshots of the business status of all verification nodes before and after the fault (such as accumulated gift value and special effect activation status), it is confirmed whether the global state before the fault has been restored, with no data loss or state rollback. For example, if the total accumulated gift value in the master node before the fault was 987,600 yuan, after the fault recovery, the value synchronously confirmed by all verification nodes is still 987,600 yuan, and there is no double counting or state rollback.

[0100] Fault recovery information includes, but is not limited to: average client reconnection time (RTO), message retransmission success rate, service interface response recovery time, state integrity matching degree, and the final delivery rate of unacknowledged messages during the fault period—these are weighted and scored to determine the fifth performance of the target system. If all the above indicators meet the preset fault tolerance thresholds (e.g., RTO ≤ 20 seconds, state recovery integrity ≥ 99.9%, message recovery rate ≥ 99.95%), the fault recovery performance (i.e., the fifth performance) is judged as "normal"; if any indicator exceeds the limit (e.g., reconnection time reaches 45 seconds, or the cumulative gift value loss is 2000 yuan), it is judged as "abnormal".

[0101] Optionally, embodiments of this application provide a method for evaluating fault recovery. Figure 6 This is a flowchart of a fault recovery assessment method according to an embodiment of this application, such as... Figure 6 As shown, it includes:

[0102] Step S61: Load the predefined standardized fault mode library, which includes distributed fault scenarios such as Broker node crash, network partition, and message queue overflow.

[0103] Step S62: At the predetermined time point of the test process (such as the peak period of the simulated live broadcast event), the fault injection controller selects the specified fault type from the fault mode library according to the current test target, and sends control commands to the target MQTT Broker instance or network device through SSH, API or container orchestration interface to trigger the fault.

[0104] Step S63: Before fault injection, trigger the global state checkpoint mechanism, instructing all Broker nodes and core service components to create a persistent checkpoint for the current running state (including accumulated gift value, online user list, activated effect ID, session context, and vector clock snapshot), and record the global vector clock at this time.

[0105] Step S64: During the fault period, monitor key operational metrics: number of client reconnection attempts, reconnection success rate, message retransmission queue backlog, backup broker takeover response time, whether service degradation strategies are enabled (e.g., read-only mode), and verification node status update interruption duration, to form a complete fault runtime behavior log.

[0106] Step S65: After the preset fault duration ends, execute recovery commands, such as restarting the failed Broker process, disabling firewall rules, clearing the backlog queue, or triggering cluster re-election, so that the system enters the recovery phase and starts listening for the recovery process.

[0107] Step S66: Continuously collect key feature information (i.e. fault recovery information) during the recovery process: detect whether the client has re-established a stable MQTT connection, whether the verification node has received the message stream again, whether the service API has resumed normal response, and whether the Broker cluster has completed state synchronization and stable switching of primary and backup roles.

[0108] Step S67: After the system recovers and stabilizes, compare the current final state copy of each verification node with the checkpoint created before the failure, calculate the matching degree of the core business state (such as cumulative gift value, fan group level, special effect status, etc.), and identify whether there are any anomalies such as data loss, duplicate calculation, state rollback or timing disorder.

[0109] Step S68: The system performs end-to-end tracing of messages that were not acknowledged (QoS 1 / 2) during the fault period, and uses message sequence numbers and ACK logs to compile a standardized fault mode library of messages that were not delivered during the fault period.

[0110] Step S69: Based on the preset quantitative evaluation model, calculate the recovery time target (RTO) (the time from the occurrence of the failure to the recovery of the core service), state recovery integrity (the percentage of checkpoints that match), and message recovery rate (the percentage of unacknowledged messages that were successfully delivered during the failure).

[0111] Step S70: If all three indicators reach the preset threshold (e.g., RTO≤20 seconds, state integrity≥99.9%, message recovery rate≥99.95%), the fault recovery performance (i.e., the fifth performance) of the target system is determined to be "normal"; if any indicator fails to meet the standard, it is marked as "abnormal".

[0112] In the above embodiments, by using a predefined standardized fault mode library, an automated fault injection mechanism, and a state checkpoint-based recovery verification process, the fault recovery test, which originally relied on manual experience and was difficult to reproduce, is transformed into a quantifiable and repeatable standardized evaluation process. This objectively measures key indicators such as recovery time targets, state integrity, and message recovery rates when facing typical faults such as broker downtime and network partitions, providing solid data support for the system's fault-tolerant design and high availability assurance.

[0113] To better understand the process of the above system performance verification method, the implementation flow of the above system performance verification method will be described below in conjunction with optional embodiments, but it is not intended to limit the technical solution of the embodiments of this application.

[0114] This embodiment provides a method for verifying system performance, taking the end-to-end stress and disaster recovery test of a large-scale live streaming event, "Peak Night," as an example, as follows:

[0115] Step 1: Preparation and Configuration;

[0116] The test platform integrates anonymized user behavior logs from similar large-scale live streaming events over the past three months to build user behavior profiles. It is configured with 100,000 virtual users to simulate real-world concurrency, with "die-hard fans" accounting for 1.5%. Simultaneously, distributed verification nodes are deployed in 12 representative cities across China (such as Shanghai, Beijing, Guangzhou, and Urumqi). Each node is configured with differentiated network parameters based on real-world network environments: for example, the Shanghai node simulates a low-latency network (average 20ms), while the Urumqi node simulates a high-latency, high-packet-loss edge network (average 110ms, packet loss rate 1.8%). Furthermore, a core failure scenario is preset: "During the peak of the event, the main MQTT Broker node suddenly crashes."

[0117] Step 2: Perform a stress test;

[0118] 1. Simulating Real-World Interactions: Based on user role profiles, the system dynamically generates a non-uniform message flow that conforms to real user behavior patterns, combining a Poisson-Gaussian mixture distribution with a Hawkes self-excitation process. At the 8-minute mark of the test, a "grand finale guest appearance" pulse event was triggered, instantly increasing the interaction intensity of the affected user group and successfully boosting the MQTT message sending rate by 320% within 3 seconds.

[0119] 2. State synchronization consistency verification: The verification node collects a snapshot of the local live room status (including popularity value, total amount of accumulated gifts, special effects status, etc.) once per second, and analyzes it in conjunction with the vector clock embedded in the message.

[0120] Step 3: Inject fault and verify recovery;

[0121] 1. Fault Injection: At the 12th minute of the test, a fault injection command is automatically executed to remotely terminate the main MQTT Broker process.

[0122] 2. Recovery Process Monitoring: The system monitors client reconnection behavior and service status changes in real time. Results show that each verification node seamlessly switches to the backup broker within an average of 15 seconds (RTO = 15s), and message sending and receiving returns to normal. After fault recovery, the current status of each node is compared with the global checkpoint generated before the fault, confirming that business statuses such as "cumulative gift value," "online audience list," and "fan club level" are consistent, achieving a message recovery rate of 99.99%.

[0123] Step 4: Generate an insight report.

[0124] A comprehensive assessment report was generated, clearly indicating that the system can stably withstand sudden interactive pressure exceeding three times the expected peak, and can achieve service recovery within 15 seconds in extreme scenarios where the core Broker node fails. However, the report also points out potential risks: when cross-regional network latency consistently exceeds 150ms (e.g., in remote western nodes), state synchronization latency approaches the system's tolerance threshold, posing a slight risk of consistency fluctuations. The report recommends optimizing the synchronization mechanism in subsequent architecture optimizations.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0126] This embodiment also provides a system performance verification device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0127] Figure 7 This is a structural block diagram of a system performance verification device according to an embodiment of this application, such as... Figure 7 As shown, the device includes:

[0128] The sending module 72 is used to send a synchronization acquisition instruction to multiple verification nodes, so that each verification node can obtain the status information corresponding to the real-time interactive event based on the synchronization acquisition instruction, and obtain the target vector clock stored by each verification node.

[0129] The acquisition module 74 is used to acquire the status information and target vector clock carried in the acquisition response information sent by each verification node, and determine the partial order relationship of the status information corresponding to the real-time interactive event between each pair of verification nodes based on the components in the target vector clock.

[0130] The verification module 76 is used to verify the first performance of the target system based on the status information of each verification node and the partial order relationship between the status information corresponding to the real-time interactive event obtained by each pair of verification nodes, wherein the target system is the interactive message system corresponding to the real-time interactive event.

[0131] In an exemplary embodiment, the verification module is configured to: determine whether the partial order relationship between the target vector clocks of each pair of verification nodes conforms to a preset partial order relationship; if the partial order relationship between the target vector clocks of each pair of verification nodes conforms to the preset partial order relationship, determine whether there is a difference in the state information of each pair of verification nodes; if there is no difference in the state information of each pair of verification nodes, determine that the first performance of the target system is normal; if there is a difference in the state information of each pair of verification nodes, determine that the first performance of the target system is abnormal; if the partial order relationship between the target vector clocks of each pair of verification nodes does not conform to the preset partial order relationship, determine whether the state difference information of the state information of each pair of verification nodes conforms to a preset state difference standard; if the state difference information of the state information of each pair of verification nodes conforms to the preset state difference standard, determine that the first performance of the target system is normal; if the state difference information of the state information of each pair of verification nodes does not conform to the preset state difference standard, determine that the first performance of the target system is abnormal.

[0132] In an exemplary embodiment, the verification module is further configured to acquire historical log data in the production environment corresponding to real-time interactive events, and cluster target objects in the production environment based on the historical log data to determine different types of target objects; determine the parameter values ​​of the interaction parameters of the different types of target objects, wherein the interaction parameters include at least one of the following: interaction timing parameters, interaction type parameters, interaction value parameters, and interaction concentration parameters; generate behavior parameter configuration files for the different types of target objects based on the parameter values ​​of the interaction parameters of the different types of target objects; generate interaction event sequences for the different types of target objects based on the behavior parameter configuration files, and run the interaction event sequences for the different types of target objects to verify the second performance of the target system.

[0133] In an exemplary embodiment, the verification module is further configured to sample parameter information in the behavior parameter configuration file using a target sampling algorithm to generate a first interactive event sequence of the different types of target objects; if a preset event exists in the real-time interactive events, determine a second interactive event sequence based on a conditional intensity function corresponding to the different types of target objects, wherein the conditional intensity function is the response intensity of the different types of target objects to the preset event; and generate the interactive event sequence according to the first interactive event sequence and the second interactive event sequence.

[0134] In an exemplary embodiment, the verification module is further configured to determine the basic response intensity of the different types of target objects to the preset event based on a basic response intensity function, and to determine the stimulus response intensity of the different types of target objects to the preset event based on an stimulus kernel function, wherein the conditional intensity function includes: the basic response intensity function and the stimulus kernel function; determine the response intensity of the target object within a target time period based on the basic response intensity and the stimulus response intensity, wherein the target time period is the time period after the occurrence of the preset event; and generate the second interactive event based on the response intensity.

[0135] In an exemplary embodiment, the verification module is further configured to, when a preset business rule triggers the real-time interactive event, determine whether an event corresponding to the preset business rule has occurred at multiple verification nodes in a first time window; if the events corresponding to the preset business rule have occurred at all multiple verification nodes in the first time window, determine that the third performance of the target system is normal; and if no event corresponding to the preset business rule has occurred at any verification node in the first time window, determine that the third performance of the target system is abnormal.

[0136] In an exemplary embodiment, the verification module is further configured to generate transmission strategies for different regions based on network parameter configuration files corresponding to different regions, wherein the network parameter configuration files include at least one of the following: network latency, packet loss rate, and jitter; transmit target state update information to target verification nodes in the corresponding regions based on the transmission strategies; and determine whether target verification nodes in different regions receive the target state update information within a second time window; if all target verification nodes in different regions receive the target state update information within the second time window, determine that the fourth performance of the target system is normal; if no target verification nodes in different regions receive the target state update information within the second time window, determine that the fourth performance of the target system is abnormal.

[0137] In an exemplary embodiment, the verification module is further configured to, in the event of a failure event, acquire failure recovery information of the verification node and the server instance in the target system, wherein the failure event is a network interruption event between the server instance and the verification node; verify the availability of the server instance if the network between the verification node and the server instance is determined to be restored; and determine a fifth performance of the target system based on the failure recovery information and the availability of the server instance.

[0138] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0139] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0140] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0141] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0142] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0143] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0144] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0145] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0146] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0147] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0148] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for verifying system performance, characterized in that, include: A synchronization acquisition command is sent to multiple verification nodes so that each verification node can obtain the status information corresponding to the real-time interactive event based on the synchronization acquisition command, and obtain the target vector clock stored by each verification node. The status information and target vector clock carried in the acquisition response information sent by each verification node are obtained, and the partial order relationship of the status information corresponding to the real-time interactive event obtained between each pair of verification nodes is determined according to the components in the target vector clock. The first performance of the target system is verified based on the state information of each verification node and the partial order relationship between the state information corresponding to the real-time interactive event obtained by each pair of verification nodes, wherein the target system is the interactive message system corresponding to the real-time interactive event.

2. The method according to claim 1, characterized in that, Verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between each pair of verification nodes includes: Determine whether the partial order relationship between the target vector clocks of each pair of verification nodes conforms to the preset partial order relationship; If the partial order relationship between the target vector clocks of the two pairs of verification nodes conforms to the preset partial order relationship, it is determined whether there is a difference in the state information of the two pairs of verification nodes; If there are no differences in the status information of each pair of verification nodes, the first performance of the target system is determined to be normal. If there are discrepancies in the status information of each pair of verification nodes, the first performance anomaly of the target system is determined.

3. The method according to claim 2, characterized in that, After determining whether the partial order relationship between the target vector clocks of each pair of verification nodes conforms to the preset partial order relationship, the method further includes: If the partial order relationship between the target vector clocks of the pairwise verification nodes does not conform to the preset partial order relationship, determine whether the state difference information of the state information of the pairwise verification nodes conforms to the preset state difference standard. If the state difference information of the state information of each pair of verification nodes meets the preset state difference standard, the first performance of the target system is determined to be normal. If the state difference information between the two pairs of verification nodes does not meet the preset state difference standard, the first performance anomaly of the target system is determined.

4. The method according to claim 1, characterized in that, Before verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between any two verification nodes, the method further includes: Obtain historical log data from the production environment corresponding to the real-time interactive event, and cluster the target objects in the production environment based on the historical log data to determine different types of target objects; Determine the parameter values ​​of the interaction parameters for the different types of target objects, wherein the interaction parameters include at least one of the following: interaction timing parameters, interaction type parameters, interaction value parameters, and interaction concentration parameters; Generate behavior parameter configuration files for the different types of target objects based on the parameter values ​​of the interaction parameters of the different types of target objects; Based on the behavior parameter configuration file, generate the interaction event sequences of the different types of target objects, and run the interaction event sequences of the different types of target objects to verify the second performance of the target system.

5. The method according to claim 4, characterized in that, Based on the behavior parameter configuration file, a sequence of interactive events for the different types of target objects is generated, including: The parameter information in the behavior parameter configuration file is sampled by a target sampling algorithm to generate the first interaction event sequence of the different types of target objects; In the case of a preset event in the real-time interactive event, a second interactive event sequence is determined based on the conditional intensity function corresponding to the different types of target objects, wherein the conditional intensity function is the response intensity of the different types of target objects to the preset event; The interactive event sequence is generated based on the first interactive event sequence and the second interactive event sequence.

6. The method according to claim 5, characterized in that, The second interactive event sequence is determined based on the conditional strength function corresponding to the different types of target objects, including: The basic response intensity of the different types of target objects to the preset event is determined based on the basic response intensity function, and the excitation response intensity of the different types of target objects to the preset event is determined based on the excitation kernel function, wherein the conditional intensity function includes: the basic response intensity function and the excitation kernel function; The response intensity of the target object within a target time period is determined based on the basic response intensity and the excitation response intensity, wherein the target time period is the time period after the occurrence of the preset event; The second interactive event is generated based on the response intensity.

7. The method according to claim 1, characterized in that, After verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between each pair of verification nodes, the method further includes: When a preset business rule triggers the real-time interactive event, it is determined whether the event corresponding to the preset business rule has occurred at multiple verification nodes in the first time window; If the events corresponding to the preset business rules occur at all the verification nodes in the first time window, it is determined that the third performance of the target system is normal. If no event corresponding to the preset business rule occurs at any verification node within the first time window, the third performance anomaly of the target system is determined.

8. The method according to claim 1, characterized in that, After verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between each pair of verification nodes, the method further includes: Different sending strategies are generated based on the network parameter configuration files corresponding to different regions. The network parameter configuration files include at least one of the following: network latency, packet loss rate, and jitter. Based on the sending strategy, the target status update information is sent to the target verification nodes in the corresponding areas, and it is determined whether the target verification nodes in different areas in the second time window receive the target status update information. If all target verification nodes in different regions within the second time window receive the target status update information, it is determined that the fourth performance of the target system is normal. If the target verification nodes in different regions do not receive the target status update information in the second time window, the fourth performance anomaly of the target system is determined.

9. The method according to claim 1, characterized in that, After verifying the first performance of the target system based on the state information of each verification node and the partial order relationship of the state information corresponding to the real-time interactive events obtained between each pair of verification nodes, the method further includes: In the event of a failure, obtain failure recovery information for the verification node and the server instance in the target system, wherein the failure event is a network interruption event between the server instance and the verification node; Verify the availability of the server instance if network recovery between the verification node and the server instance is confirmed. The fifth performance of the target system is determined based on the fault recovery information and the availability of the server instance.

10. A system performance verification device, characterized in that, include: The sending module is used to send synchronous acquisition instructions to multiple verification nodes, so that each verification node can obtain the status information corresponding to the real-time interactive event based on the synchronous acquisition instructions, and obtain the target vector clock stored by each verification node. The acquisition module is used to acquire the status information and target vector clock carried in the acquisition response information sent by each verification node, and determine the partial order relationship of the status information corresponding to the real-time interactive event between each pair of verification nodes based on the components in the target vector clock. The verification module is used to verify the first performance of the target system based on the status information of each verification node and the partial order relationship between the status information corresponding to the real-time interactive event obtained by each pair of verification nodes, wherein the target system is the interactive message system corresponding to the real-time interactive event.

Images