Message quality analysis method, system, device, storage medium and product

Abstract

The application provides a message quality analysis method, system, device, storage medium and product, and is applied to the technical field of network communication. The method comprises the following steps: obtaining the respective corresponding round-trip delay, retransmission rate and out-of-order rate of each message pair in a target service session; and determining the message transmission quality in the target service session according to the round-trip delay, the retransmission rate and the out-of-order rate. The technical scheme aims to solve the technical problem that the accuracy of the message quality calculation result obtained by the traditional method is relatively low.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, and in particular to a message quality analysis method, system, device, storage medium, and product. Background Technology

[0002] Since its development in 2006, mobile communication networks have evolved from traditional 2G (Second Generation) networks to today's 5G (Fifth Generation) networks. With this development, the complexity of mobile communication networks has also increased significantly.

[0003] Nowadays, the quality of mobile network access is usually assessed through subjective evaluation (such as evaluation based on the user's network usage experience) and objective evaluation (such as evaluation through speed test software).

[0004] However, since subjective and objective assessments are affected by inconsistent and uncommon assessment standards, the accuracy of message quality measurement results obtained through traditional assessment methods is low, making it difficult to accurately represent the actual Internet access quality. Summary of the Invention

[0005] This invention proposes a message quality analysis method, system, device, storage medium, and product, aiming to solve the technical problem of low accuracy of message quality measurement results obtained by traditional evaluation methods.

[0006] To address the above problems, this invention proposes a message quality analysis method, which includes:

[0007] Based on multiple message pairs in the target service session, the round-trip time, retransmission rate, and out-of-order rate corresponding to each message pair are obtained;

[0008] The message transmission quality in the target service session is determined by the round-trip delay, the retransmission rate, and the out-of-order rate.

[0009] Optionally, the step of determining the message transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate includes:

[0010] Each round-trip delay, each retransmission rate, and each out-of-order rate are input into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model. The transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the out-of-order rate. The out-of-order rate has a smaller impact on the transmission quality factor than the round-trip delay and the retransmission rate.

[0011] Based on the transmission quality factor, the message transmission quality in the target service session is obtained.

[0012] Optionally, the message pair includes an uplink message pair and a downlink message pair, and the transmission quality factor includes a left transmission quality factor and a right transmission quality factor;

[0013] Before the step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model, the method further includes:

[0014] The round-trip time and retransmission rate of the downlink packet pairs and the out-of-order rate of the uplink packet pairs are used as the left-side transmission quality dataset.

[0015] The round-trip time and retransmission rate of the uplink message pair and the out-of-order rate of the downlink message pair are used as the right-side transmission quality dataset.

[0016] The step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model includes:

[0017] Input the left-side transmission quality dataset into a preset user experience scoring model to obtain the left-side transmission quality factor output by the user experience scoring model;

[0018] The right-side transmission quality dataset is input into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model.

[0019] Optionally, before the step of obtaining the message transmission quality in the target service session based on the transmission quality factor, the method further includes:

[0020] Obtain the service status identifier of the target service session;

[0021] The step of obtaining the message transmission quality in the target service session based on the transmission quality factor includes:

[0022] The message transmission quality in the target service session is obtained based on the transmission quality factor and the service status identifier.

[0023] Optionally, when multiple target service sessions are detected, after the step of obtaining the message transmission quality in the target service session based on the transmission quality factor and the service status identifier, the method further includes:

[0024] The left-side transmission quality factors in each of the aforementioned message transmission quality factors are weighted and averaged to obtain the average left-side transmission quality factor, and the right-side transmission quality factors in each of the aforementioned message transmission quality factors are weighted and averaged to obtain the average right-side transmission quality factor.

[0025] The service status identifiers in each of the message transmission quality parameters are weighted, and the service status identifier with the highest weighting frequency is taken as the comprehensive service status identifier.

[0026] By combining the average left-side transmission quality factor, the average right-side transmission quality factor, and the integrated service status identifier, a mobile network quality monitoring result vector is obtained.

[0027] Optionally, after the step of obtaining the round-trip time, retransmission rate, and out-of-order rate corresponding to each of the multiple packet pairs in the target service session, the method further includes:

[0028] The end-to-end round-trip time is obtained by superimposing the round-trip times of each of the above-mentioned round-trip times;

[0029] The first difference between each of the retransmission rates and a preset threshold is determined, and the product between the preset threshold and the first difference is subtracted to obtain the end-to-end retransmission rate.

[0030] The second difference between each of the disorder rates and the preset threshold is determined, and the product between the preset threshold and the second difference is subtracted to obtain the end-to-end disorder rate;

[0031] The end-to-end round-trip time, end-to-end retransmission rate, and end-to-end out-of-order rate are input into the user experience scoring model to obtain the end-to-end transmission quality factor.

[0032] Optionally, after the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end out-of-order rate into the user experience scoring model to obtain the end-to-end transmission quality factor, the method further includes:

[0033] Calculate the total number of packets, actual end-to-end round-trip time, actual end-to-end retransmission rate, and actual end-to-end out-of-order rate for each collection time within the preset time period.

[0034] Each of the actual end-to-end round-trip times is taken as a new end-to-end round-trip time, each of the actual end-to-end retransmission rates is taken as a new end-to-end retransmission rate, and each of the actual end-to-end out-of-order rates is taken as a new end-to-end out-of-order rate. Then, the process of inputting the end-to-end round-trip times, the end-to-end retransmission rates, and the end-to-end out-of-order rates into the user experience scoring model to obtain the end-to-end transmission quality factor is repeated until the actual end-to-end transmission quality factor corresponding to each of the collection times is obtained.

[0035] Multiply the actual end-to-end transmission quality factor corresponding to each of the acquisition times by the total number of packets to obtain the full-cycle packet quality, and sum the total number of packets corresponding to each of the acquisition times to obtain the total number of packets in the full cycle.

[0036] The ratio of the full-cycle message quality to the total number of full-cycle messages is used as the full-cycle end-to-end transmission quality factor.

[0037] Optionally, before the step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model, the method further includes:

[0038] Obtain the user experience scoring model to be tested and the transmission quality dataset to be tested, and input the transmission quality dataset to be tested into the user experience scoring model to be tested to obtain the transmission quality factor test results;

[0039] When the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality dataset under test is detected to be greater than a set threshold, the adjustment factor in the user experience scoring model under test is optimized using a preset reference anchor dataset, and the user experience scoring model under test after optimization of the adjustment factor is used as the user experience scoring model.

[0040] Furthermore, to address the aforementioned problems, this invention also proposes a message quality analysis system, which includes:

[0041] The indicator value calculation module is used to obtain the round-trip delay, retransmission rate and out-of-order rate of each message pair based on multiple message pairs in the target service session.

[0042] The transmission quality determination module is used to determine the packet transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate.

[0043] In addition, to solve the above problems, the present invention also proposes a message quality analysis device, which includes: a memory, a processor, and a message quality analysis program stored in the memory and executable on the processor. When the message quality analysis program is executed by the processor, it implements the steps of the message quality analysis method as described above.

[0044] In addition, to solve the above problems, the present invention also proposes a computer storage medium storing a message quality analysis program, wherein the message quality analysis program, when executed by a processor, implements the steps of the message quality analysis method as described above.

[0045] In addition, to solve the above problems, the present invention also proposes a computer program product, which includes a message quality analysis program, and when the message quality analysis program is executed by a processor, it implements the steps of the message quality analysis method as described above.

[0046] In this embodiment, the present invention obtains the round-trip delay, retransmission rate, and out-of-order rate corresponding to each message pair based on multiple message pairs in the target service session. It can extract the corresponding message pairs in the session that needs to be measured for message transmission quality and obtain indicators to characterize the transmission quality level. Then, the message transmission quality in the target service session is determined by the round-trip delay, retransmission rate, and out-of-order rate. The message transmission quality in the target service session can be comprehensively determined based on the obtained multiple indicators.

[0047] Compared to traditional measurement schemes, this invention, by calculating round-trip time, retransmission rate, and out-of-order rate, avoids the problem of poor universality caused by setting the speed measurement device in the session terminal in objective evaluation. Furthermore, this application can calculate message transmission quality using objective data such as round-trip time, retransmission rate, and out-of-order rate, thereby avoiding the problem of inconsistent quantitative standards in subjective evaluation. Therefore, since the round-trip time and other data used in this invention are all real data from message transmission, and this invention is not affected by the session terminal or subjective judgment when measuring message transmission quality, this application improves the accuracy of message quality measurement results compared to traditional schemes. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the hardware operating environment of the message quality analysis device involved in the embodiments of the present invention;

[0051] Figure 2 This is a flowchart illustrating the first embodiment of the message quality analysis method of the present invention;

[0052] Figure 3 This is a schematic diagram of spectroscopic acquisition in an embodiment of the message quality analysis method of the present invention;

[0053] Figure 4This is a schematic diagram of the software-based monitoring data acquisition point acquisition in an embodiment of the message quality analysis method of the present invention;

[0054] Figure 5 This is a schematic diagram of the left and right data directions in an embodiment of the message quality analysis method of the present invention;

[0055] Figure 6 This is a schematic diagram of server disconnection in an embodiment of the message quality analysis method of the present invention;

[0056] Figure 7 This is a schematic diagram of the original model for internet access quality assessment, representing an embodiment of the packet quality analysis method of the present invention.

[0057] Figure 8 This is a second schematic diagram of an Internet access quality assessment model according to an embodiment of the packet quality analysis method of the present invention;

[0058] Figure 9 This is a fourth schematic diagram of an internet access quality assessment model according to an embodiment of the packet quality analysis method of the present invention;

[0059] Figure 10 This is the fifth schematic diagram of an Internet access quality assessment model according to an embodiment of the packet quality analysis method of the present invention;

[0060] Figure 11 This is the sixth schematic diagram of an Internet access quality assessment model according to an embodiment of the packet quality analysis method of the present invention;

[0061] Figure 12 This is a schematic diagram of the functional modules of an embodiment of the message quality analysis system of the present invention. Detailed Implementation

[0062] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0064] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0065] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0066] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0067] like Figure 1 As shown, Figure 1 This is a schematic diagram of the hardware operating environment of the message quality analysis device involved in the embodiments of the present invention.

[0068] like Figure 1 As shown, in the hardware operating environment of the packet quality analysis device, the packet quality analysis device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as a disk storage device. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0069] Those skilled in the art will understand that Figure 1 The structure of the message quality analysis device shown does not constitute a limitation on the message quality analysis device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0070] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a message quality analysis program.

[0071] exist Figure 1 In the device shown, network interface 1004 is mainly used to connect to the backend server and communicate with it; user interface 1003 is mainly used to connect to the client (user terminal) and communicate with it; while processor 1001 can be used to call the packet quality analysis program stored in memory 1005 and perform the following operations:

[0072] Based on multiple message pairs in the target service session, the round-trip time, retransmission rate, and out-of-order rate corresponding to each message pair are obtained;

[0073] The message transmission quality in the target service session is determined by the round-trip delay, the retransmission rate, and the out-of-order rate.

[0074] Based on the above hardware structure, the overall concept of various embodiments of the message quality analysis method of the present invention is proposed.

[0075] In this embodiment, the quality of mobile communication network access is now typically assessed through subjective evaluation (e.g., based on the user's network usage experience) or objective evaluation (e.g., through speed test software).

[0076] However, since subjective and objective assessments are affected by inconsistent and uncommon assessment standards, the accuracy of message quality measurement results obtained through traditional assessment methods is low, making it difficult to accurately represent the actual Internet access quality.

[0077] To address the above issues, a message quality analysis method is proposed.

[0078] Based on the overall concept of the various embodiments of the message quality analysis method of the present invention described above, various embodiments of the message quality analysis method of the present invention are proposed.

[0079] It should be noted that the execution subject of each embodiment of the message quality analysis method of the present invention is the aforementioned message quality analysis device, which can be a computer, server, or other terminal. For ease of explanation, the execution subject will be omitted in the following embodiments.

[0080] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the message quality analysis method of the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps of the message quality analysis method of the present invention can be performed in a different order.

[0081] In this embodiment, the message quality analysis method includes:

[0082] Step S10: Based on multiple message pairs in the target service session, obtain the round-trip delay, retransmission rate and out-of-order rate corresponding to each message pair.

[0083] It should be noted that the target service session refers to the current session for which message transmission quality needs to be measured. For example, if a user terminal has public or industry applications installed, and the current session includes a messaging app, and the message transmission quality of that messaging app needs to be measured, the session between the messaging app's terminal device (e.g., a mobile phone) and the server can be used as the target service session. To avoid the data collected by the service terminal being affected by the terminal itself, a target collection point can be set in the target service session. That is, the target collection point is a pre-determined message collection point in the message transmission path within the target service session. Round-trip time (RTT) refers to the total time required from the sender sending a data packet to the receiver receiving the data packet and sending an acknowledgment (ACK) signal, and then back to the sender receiving the acknowledgment signal. Retransmission rate refers to the proportion of data packets that need to be retransmitted due to various reasons (such as network congestion, data packet loss, etc.) that are not successfully received in network communication. Out-of-order rate refers to the proportion of data packets received by the receiver in a different order than those sent by the sender.

[0084] In this embodiment, when it is detected that the message transmission quality in a certain session needs to be measured, the session can be used as the target service session, and each message pair can be obtained at a pre-set message collection point in the message transmission path of the session, so as to calculate the round-trip delay, retransmission rate and out-of-order rate of each message pair.

[0085] In one feasible implementation, the message pair includes an uplink message pair and a downlink message pair. The uplink message pair refers to a first request message sent by a terminal device in the target service session to a server and a first response message corresponding to the first request message sent by the server. Similarly, the downlink message pair refers to a second request message sent by a server in the target service session to a terminal device and a second response message corresponding to the second request message sent by the terminal device. The target service session includes various uplink message pairs and various downlink message pairs. A typical "target service session" includes, but is not limited to, a set of messages with the same IP 5-tuple (source / destination IP address, source / destination Layer 4 port, and service type TOS).

[0086] When calculating round-trip time, for a single "uplink message pair" and "downlink message pair", if the uplink message pair includes a first request message and a first response message, and the downlink message pair includes a second request message and a second response message, the round-trip time corresponding to the uplink message pair can be obtained based on the difference between the timestamps of the first request message and the first response message. Similarly, the round-trip time corresponding to the downlink message pair can be obtained based on the difference between the timestamps of the second request message and the second response message.

[0087] For each acquired uplink and downlink message pair, the round-trip time of each pair is first calculated using the same method as for individual uplink and downlink message pairs. Then, the arithmetic mean is calculated for the respective delay sets of the calculated uplink and downlink message pairs to obtain the average round-trip time of the uplink and downlink message pairs in the current target business session. For each acquired uplink or downlink message pair, if only the first / second request message is acquired but the first / second response message is not acquired, the request message is recorded as a retransmission. The total number of first / second request messages for which no first / second response message was acquired is divided by the total number of all first / second request messages to obtain the retransmission rate statistics for the corresponding direction of a business session. For each "uplink message pair" or "downlink message pair," if the acquisition point only receives a certain first / second request message, and the order of this message is out of order compared to the other received first / second request messages, then this request message is recorded as an out-of-order event. The total number of out-of-order first / second request messages is divided by the total number of all first / second request messages to obtain the out-of-order rate statistics for the corresponding direction of a "business session." Thus, the round-trip time, retransmission rate, and out-of-order rate for each uplink and downlink message pair can be obtained.

[0088] Optionally, in a feasible embodiment, step S10 above further includes:

[0089] Step S101: At the target collection point in the target service session, acquire each uplink packet pair and each downlink packet pair by optical splitting.

[0090] Please refer to Figure 3 , Figure 3 This is a schematic diagram of optical acquisition based on an embodiment of the message quality analysis method of the present invention. Here, UE refers to User Equipment, SPN refers to network slicing in a 5G network, and PDN server includes, but is not limited to, service servers, CDN servers, edge CDNs, etc. Figure 3 In this context, the monitoring data collection points are also known as target collection points. These are designated for ease of data collection and storage. Figure 3 The system also includes a monitoring data processing platform for collecting and storing data, enabling the output of internet access log records (including monitoring results data and calculated monitoring indicators) at any time.

[0091] exist Figure 3 In the process of user equipment and server having a session, the monitoring data collection point is set between the N2 and N3 interfaces, so that uplink and downlink message pairs can be collected at the monitoring data collection point.

[0092] Step S102: At the target collection point in the target service session, obtain each uplink packet pair and each downlink packet pair through software-based monitoring data collection point.

[0093] Please refer to the following at the same time Figure 3 and Figure 4 , Figure 4 This is a schematic diagram illustrating the data acquisition points for software-based monitoring in an embodiment of the message quality analysis method of the present invention. Figure 3 In comparison, Figure 4 The message transmission path shown can also be used to deploy software-defined monitoring data collection points within the core network elements of the communication core network to collect uplink and downlink message pairs.

[0094] Step S20: Determine the message transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate.

[0095] Understandably, a higher round-trip time (RTD) indicates a longer time required to transmit data, resulting in poorer message transmission quality. Similarly, the retransmission rate and out-of-order rate can also characterize message transmission quality.

[0096] As an example, scores corresponding to specific values ​​of round-trip delay, retransmission rate, and out-of-order rate can be pre-set. Then, based on the weights corresponding to the round-trip delay, retransmission rate, and out-of-order rate, the scores are weighted and combined to obtain the comprehensive network quality score of the target service session. The message transmission quality can then be determined based on the comprehensive network quality score.

[0097] Optionally, in one feasible implementation, step S20 above includes:

[0098] Step S201: Input each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model. The transmission quality factor is inversely correlated with the round-trip delays, retransmission rates, and out-of-order rates. The out-of-order rate has less influence on the transmission quality factor than the round-trip delays and retransmission rates.

[0099] It should be noted that the user experience scoring model is a model that takes round-trip latency, retransmission rate, and out-of-order rate as input data and transmission quality factor as output data. The transmission quality factor characterizes the transmission quality of service data. In one feasible implementation, the user experience scoring model can be a pre-set formula or a pre-trained machine learning algorithm model; this application does not impose any restrictions on this.

[0100] Step S202: Obtain the message transmission quality in the target service session based on the transmission quality factor.

[0101] It should be noted that message transmission quality refers to the effectiveness of data packet transmission within the network. Higher message transmission quality means better data packet transmission.

[0102] It is understandable that, in one feasible implementation, after obtaining the transmission quality factor, the transmission quality factor can be used as the message transmission quality, or the quality standard corresponding to the numerical range of the transmission quality factor can be used as the message transmission quality.

[0103] It should also be noted that, since the target data collection point is located between the user terminal and the server, the statistically obtained data such as round-trip time (RTT), retransmission rate (RTR), and out-of-order rate (OFO) can actually be divided into two directions: for example... Figure 3 There are two message transmission directions: 1) RTT / RTR / OFO from the collection point to the user terminal (leftward); 2) RTT / RTR / OFO from the collection point to the server (rightward). Therefore, in addition to directly inputting each round-trip time, each retransmission rate, and each out-of-order rate into the model to obtain the transmission quality factor, the transmission quality factor can also be divided into left-side transmission quality factor and right-side transmission quality factor based on the message transmission direction, which includes left and rightward. Therefore, before step S20 above, the round-trip time, each retransmission rate, and each out-of-order rate obtained in step S10 above can also be classified.

[0104] Based on this, in one feasible implementation, the method further includes:

[0105] Step S200: The round-trip delay, retransmission rate and out-of-order rate of the downlink packet pair are used as the left transmission quality dataset.

[0106] Step S210: The round-trip delay, retransmission rate of the uplink message pair, and the out-of-order rate of the downlink message pair are used as the right-side transmission quality dataset.

[0107] As an example, please refer to Figure 5Taking a mobile phone as an example, for the uplink request message and corresponding server response message (uplink message pair) initiated by the mobile phone and obtained at the collection point, the round-trip time (RTT) and retransmission rate (RTR) calculated based on both actually reflect the message transmission quality of the right path (core network + bearer network + server) at the collection point, and are denoted as right-side delay (RTTR) and right-side retransmission rate (RTRR), respectively. The out-of-order rate (OFO) actually reflects the out-of-order status of incoming messages on the left path (transmission network + wireless base station + terminal), and is therefore denoted as left-side out-of-order rate (OFOL). For the downlink request message and corresponding mobile phone response message (downlink message pair) initiated by the server and obtained at the collection point, the round-trip time (RTT) and retransmission rate (RTR) calculated based on both actually reflect the message transmission quality of the left path at the collection point, and are denoted as left-side delay (RTTL) and left-side retransmission rate (RTRL), respectively. The out-of-order rate (OFO) actually reflects the out-of-order status of incoming messages on the right side, and is therefore denoted as right-side out-of-order rate (OFOR).

[0108] Please continue to refer to Figure 5 It should be noted that Req_UL refers to a request or uplink transmission request sent from the user equipment (UE) to the network side (first request message), Resp refers to the network side's response to the request (Req) sent by the user equipment (UE) (first response message), Req_DL refers to a request or downlink transmission request sent from the network side to the user equipment (second request message), and Resp refers to a request or downlink transmission request sent from the network side to the user equipment (second response message). Furthermore, in Figure 5 In the dataset, the transmission quality dataset on the left includes RTTL, RTRL, and OFOL. The transmission quality dataset on the right includes RTTR, RTRR, and OFOR.

[0109] Based on this, step S201 above also includes:

[0110] Step S2011: Input the left transmission quality dataset into a preset user experience scoring model to obtain the left transmission quality factor output by the user experience scoring model;

[0111] Step S2012: Input the right-side transmission quality dataset into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model.

[0112] Therefore, by processing the left and right transmission quality datasets separately using a user experience scoring model, the transmission quality factors for the left and right sides of the target collection point can be obtained. Compared to the method of directly inputting indicators such as round-trip delay into the model, this embodiment divides the indicators by transmission direction, which can more accurately reflect the message transmission situation of different transmission paths, thus obtaining a more accurate transmission quality factor.

[0113] As an example, to obtain a user experience rating model, boundary constraints are first required, namely:

[0114] The worst and best message transmission quality states are defined as follows:

[0115] The worst transmission quality: The receiver receives no messages from the sender at all;

[0116] The best transmission quality is achieved when the receiver receives all messages completely (without packet loss) and in order (without out-of-order delivery) the instant the sender sends the message (transmission delay is 0). At this time, all message sending and receiving are completed instantaneously, and the transmission quality is the best.

[0117] Then, the transmission quality factors for the "best" and "worst" message (session) transmission quality are defined as 1 and 0, respectively. At this point, the following correlations and boundary constraints between latency, retransmission rate, out-of-order rate, and quality factors can be obtained:

[0118] 1. Message transmission delay and quality factor (message transmission quality) show an inverse correlation:

[0119] 1) The greater the delay, the worse the transmission quality and the smaller the quality factor. The quality factor decreases monotonically as the delay increases.

[0120] 2) When the delay approaches infinity, the receiver approaches not receiving any messages from the sender, which meets the aforementioned definition. At this time, the transmission quality approaches the worst, and according to the aforementioned definition, the quality factor approaches 0.

[0121] 3) When the latency approaches zero, if the receiver can receive all messages completely and in order (retransmission rate and out-of-order rate are both 0), it meets the above definition. At this time, the transmission quality approaches the best. According to the above definition, the quality factor approaches 1.

[0122] 2. Message retransmission rate is inversely correlated with message transmission quality / quality factor:

[0123] 1) The higher the retransmission rate, the worse the transmission quality and the smaller the quality factor. The quality factor decreases monotonically as the retransmission rate increases.

[0124] 2) When the retransmission rate approaches 100%, it means that the receiver is close to not receiving any messages from the sender, which meets the aforementioned definition. At this time, the transmission quality approaches the worst, and according to the aforementioned definition, the quality factor approaches 0.

[0125] 3) When the retransmission rate approaches zero, if the receiver can receive all messages instantaneously and in order (delay and out-of-order rate are both 0), it meets the above definition. At this time, the transmission quality approaches the best. According to the above definition, the quality factor approaches 1.

[0126] 3. Message delivery out-of-order rate shows an inverse correlation with message transmission quality / quality factor:

[0127] 1) The higher the out-of-order rate, the worse the transmission quality, and the quality factor decreases monotonically as the out-of-order rate increases;

[0128] 2) When the out-of-order rate approaches 100%, it means that although the receiver can receive the message, the transmission quality is poor (not the worst). This is the "complete out-of-order point". Further defined: the quality factor when the out-of-order rate is 100%, the delay is 0, and the retransmission rate is 0% is 0.5.

[0129] 3) When the out-of-order rate approaches zero, if the receiver can receive all messages instantaneously and completely at the same time (delay and retransmission rate are both 0), it meets the above definition. At this time, the transmission quality approaches the best. According to the above definition, the quality factor approaches 1.

[0130] In summary, the boundary constraints between message transmission quality and latency, retransmission rate, and out-of-order rate are shown in Table 1 below:

[0131] Table 1

[0132] quality factor Round trip delay retransmission rate Disorder rate 1 0 0 0 0 any 100 any 0 +∞ any any 0.5 0 0 100

[0133] Then, the model can be constructed based on the constraint relationships defined in Table 1:

[0134] It should be noted that any functional relationship between the quality factor Q and the time delay RTT, retransmission rate RTR, and out-of-order rate OFO that satisfies the boundary constraints in Table 1 can be applied to the establishment of the functional relationship between the quality factor and RTT / RTR / OFO proposed in this application, that is:

[0135] Q = f(RTT, RTR, OFO);

[0136] Furthermore, the left / right transmission quality factors QL / QR (the quality factors corresponding to the left transmission quality dataset and the right transmission quality dataset) can be independently calculated based on the above function form using left / right RTTL / RTTR, RTRL / RTRR, and OFOL / OFOR respectively, i.e.:

[0137]

[0138] Furthermore, the general functional relationship between the quality factor Q and RTT / RTR / OFO, which satisfies the boundary constraints in Table 1, can also be expressed as follows: the transmission quality Q is equal to the product of the delay quality factor QRTT, the retransmission quality factor QRTR, and the out-of-order quality factor QOFO, i.e.:

[0139] Q(RTT,RTR,OFO)=Q RTT (RTT)*Q RTR (RTR)*Q OFO (OFO);

[0140] in,

[0141]

[0142] Based on the general functional relationship and value conditions of the above equation, we can construct (but are not limited to) the following functional factor expression:

[0143]

[0144] The user experience scoring model used to calculate the transmission quality factor (Q) can then be:

[0145]

[0146] Replacing RTR with RTRL and RTRR, RTT with RTTL and RTTR, and OFO with OFOL and OFOR in the above formula, we can obtain the left-side transmission quality factor and the right-side transmission quality factor. That is, the formulas for calculating the left-side and right-side transmission quality factors can be:

[0147]

[0148] The meanings and sources of values ​​for the different parameters in the formula:

[0149] RTTL and RTRL correspond to downlink round-trip time and downlink retransmission rate, respectively, while OFOL corresponds to uplink out-of-order rate, reflecting the message delivery status on the downlink path; RTTR and RTRR correspond to uplink round-trip time and downlink retransmission rate, respectively, while OFOR corresponds to downlink out-of-order rate, reflecting the message delivery status on the uplink path.

[0150] It is understandable that message transmission quality can also be a vector that can be used to characterize the message delivery status. Therefore, in this embodiment, when all path directions include left and right, QL and QR can be used as elements of a vector, that is, combining QL and QR yields vector Q, i.e.:

[0151] Q = (QL, QR);

[0152] Therefore, the transmission quality of packets in the target service session can be characterized by the transmission quality factor on the left and the quality factor transmitted on the right.

[0153] In this embodiment, compared with traditional measurement schemes, on the one hand, the present invention, by calculating the round-trip delay, retransmission rate, and out-of-order rate at the target collection point, can avoid the problem of poor universality caused by the speed measurement device being set in the session terminal in objective evaluation. On the other hand, the present application also inputs the transmission quality datasets of different path directions into a pre-set user experience scoring model, which can quantify the impact of round-trip delay, retransmission rate, and out-of-order rate on message transmission quality through the set model, thereby avoiding the problem of inconsistent quantification standards in subjective evaluation. In addition, since the round-trip delay and other data used in the present invention are all real data in message transmission, and the present invention is not affected by the session terminal or subjective judgment when measuring message transmission quality, the present application improves the accuracy of message quality measurement results compared with traditional schemes.

[0154] Based on the first embodiment of the message quality analysis method of the present invention described above, a second embodiment of the message quality analysis method of the present invention is proposed.

[0155] In this embodiment, before step S202 above, the method further includes:

[0156] Step S50: Obtain the service status identifier of the target service session;

[0157] It should be noted that a service status identifier refers to a status code information used to indicate the status or event of a specific service during network communication. Specifically, service status identifiers include, but are not limited to, successful completion of a service and abnormal termination.

[0158] For end-to-end service quality, in addition to the transmission quality of packets along the intermediate path, the behavior of the terminal / server itself will also affect the service quality. Analyzing only the transmission quality of packets along the path cannot fully reflect the overall service quality.

[0159] For example Figure 6 In the TCP message exchange between the terminal and the server, the forwarding latency, packet loss, and out-of-order delivery metrics are all normal. However, because the server actively sends an Rst message to reset the session after the TCP handshake, the service is interrupted. In this scenario, additional status code information reflecting "service interruption by the server" is needed to accurately reflect the service quality. This is understandable. Figure 6The business interruption scenario described is just one example. Other business scenarios also have the need to introduce status code information, and this invention does not limit this.

[0160] Therefore, in the message quality assessment model, for the "message (session) quality assessment vector", an additional "service status identifier" Status reflecting the end-to-end status of the service can be introduced. For different protocol stack types, different service status identifiers are given by enumeration values.

[0161] For example, for a single TCP session with the same IP 5-tuple as the smallest statistical granularity, the following eight common enumeration values ​​are:

[0162] Success – indicates that the TCP / UDP session started and ended normally, and all data transmission was completed;

[0163] Server connection establishment timeout - After the client sends the first SYN, it does not receive a SYN ACK message from the server;

[0164] Client connection establishment timeout - The server sent a SYN ACK message but did not receive an ACK message from the client;

[0165] Transmission timeout – If the TCP session does not end with RST / FIN / FIN ACK after a successful TCP three-way handshake, it is considered a timeout.

[0166] If a service is rejected by the server, a response code representing a server-rejected service is received from the application layer protocol; if a service is rejected by the client, a response code representing a client-rejected service is received from the application layer protocol.

[0167] Server-side abort—The server sends an RST to terminate the session;

[0168] Client aborts – The client sends an RST to abort the session.

[0169] Based on the above state classification, for a TCP packet or session, the "business state identifier" can be expressed as:

[0170]

[0171] It should be noted that the values ​​of the aforementioned service status identifiers are not limited to the eight types mentioned above. The values ​​of the service status identifiers can be flexibly set under different network environments, communication requirements, and communication targets, and this application does not impose any restrictions on this.

[0172] Based on this, step S202 above also includes:

[0173] Step S2021: Obtain the message transmission quality in the target service session based on the transmission quality factor and the service status identifier.

[0174] In this embodiment, the vector corresponding to message transmission quality can be:

[0175] Q = (QL, QR, Status);

[0176] Understandably, after adding the "status" service status identifier, the prototype of the message (session) quality assessment vector can be represented in the following form:

[0177]

[0178] in:

[0179] RTR L ∈[0,1]

[0180] RTT L ∈[0,+∞]

[0181] OFO L ∈[0,1]

[0182] RTR R ∈[0,1],

[0183] RTT R ∈[0,+∞]

[0184] OFO R ∈[0,1]

[0185]

[0186] Optionally, in a feasible embodiment, before the step of inputting each of the round-trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model, the method further includes:

[0187] Step X: Obtain the user experience scoring model to be tested and the transmission quality dataset to be tested, and input the transmission quality dataset to be tested into the user experience scoring model to be tested to obtain the transmission quality factor test results;

[0188] Step Y: When the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality dataset to be tested is greater than a set threshold, the adjustment factor in the user experience scoring model to be tested is optimized using a preset reference anchor dataset, and the user experience scoring model to be tested after optimization is used as the user experience scoring model.

[0189] It is understandable that the following formula is obtained:

[0190]

[0191] Then, the transmission quality factors on the left and right sides can be calculated based on the formula, and the formula can be optimized and used as a new user experience scoring model.

[0192] It should be noted that the user experience scoring model under test refers to the formula for calculating Q mentioned above, and the adjustment factor refers to the exponent or multiple of RTR, RTT, and OFO in the formula. In the above formula, the exponents of RTR, RTT, and OFO are all 1. To improve the accuracy of the calculated Q, the accuracy of the transmission quality obtained by the user experience scoring model under test can be judged based on the transmission quality dataset under test (including round-trip time, out-of-order rate, and retransmission rate). Then, by referring to the anchor dataset (including pre-set transmission quality factors, round-trip time, retransmission rate, and out-of-order rate), the exponents or multiples of RTR, RTT, and OFO can be re-determined, thereby optimizing each exponent or multiple and obtaining a new user experience scoring model.

[0193] As an example, there are two sets of data in the dataset to be tested for transmission quality:

[0194] A(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.02s / 0.1% / 0.1%, 0.01s / 1% / 1%;

[0195] B(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.2s / 1% / 1%, 0.1s / 1% / 1%;

[0196] You can input the above two sets of data, A and B, respectively:

[0197]

[0198] The transmission quality factor test results obtained are as follows:

[0199] QA (QL / QR): 0.9784 / 0.9881;

[0200] QB (QL / QR): 0.8168 / 0.8911;

[0201] It is evident that when latency, retransmission, and out-of-order delivery are generally 10 times worse in combination B than in combination A, the transmission quality factor only decreases by less than 20%. This means the difference between the tested transmission quality factor and the actual transmission quality factor exceeds the set threshold, causing the variation in the quality factor to fail to reflect the actual changes in service quality. Therefore, without altering the original boundary constraints, the exponents or multiples of RTR, RTT, and OFO are modified as follows:

[0202]

[0203] Among them, αL / βL / mL and αR / βR / mR are adjustment factors ranging from 0 to +∞.

[0204] By adding an adjustment factor, it is possible to achieve a differentiated representation of the distribution area of ​​indicators on the examined path (left / right path) based on the transmission quality factor.

[0205] Understandably, when calculating the value of the adjustment factor, the average latency, retransmission rate, and out-of-order rate of the specific network environment in which the user experience scoring model under test is applied can be used as a "reference anchor point". By inputting the above "reference anchor point" value (i.e., the reference anchor point dataset) into the adjustment factor model, the reasonable value of the "adjustment factor" in the specific network environment can be derived in reverse.

[0206] For example, taking a network environment in a first-tier city as an example, the data statistics obtained from the proposed method of collecting internet access logs in this network environment require the following values ​​for left / right latency, retransmission rate, and out-of-order rate:

[0207] RTTL = 0.03 (seconds), RTTR = 0.02 (seconds);

[0208] RTRL=0.013 (1.3%), RTRR=0.009 (0.9%);

[0209] OFOL=0.003 (0.3%), OFOR=0.012 (1.2%);

[0210] 1) Using QL and QR of 0.9 as reference anchor point 1 under the conditions of RTTL=0.03, RTTR=0.02, lossless (RTR=0), and ordered (OFO=0) transmission, we further obtained different m values ​​and obtained the transmission quality factors QL and QR on the left and right sides as follows:

[0211]

[0212] 2) For the same left and right RTT latency (RTTL=0.03, RTTR=0.02), considering lossy (RTR>0) and ordered (OFO=0) transmission scenarios, based on the same first-tier city network environment, the end-to-end retransmission rate statistics of the 5G network at the N3 port are as follows: The left / right retransmission rates are:

[0213] RTR L =0.013, 1.3%;

[0214] RTR R =0.009, 0.9%;

[0215] Let the transmission quality factor Q' of the corresponding path be equal to the network empirical average (RTTL = 0.013, RTTR = 0.009). This is equal to 90% of the quality factor Q value for lossless, ordered transmission with the same delay, where Q = 0.9. Therefore, we obtain:

[0216]

[0217] Will:

[0218] RTR L =0.013;

[0219] RTR R =0.009;

[0220] Substituting into the above equation, we get:

[0221] α L ≈0.53;

[0222] α R ≈0.49;

[0223] Therefore, we get:

[0224]

[0225] 3) For the same end-to-end RTT latency (RTT = 0.05), considering lossless (RTR = 0) and out-of-order (OFO > 0) delivery scenarios, based on the statistics of the end-to-end out-of-order rate of the existing 5G network at the N3 port, the out-of-order rates on the left and right sides are as follows:

[0226] OFO L =0.003, 0.3%;

[0227] OFO R =0.012, 1.2%;

[0228] Let the transmission quality factor Q' of the corresponding path be equal to the network empirical average (OFOL = 0.003, OFOR = 0.012). This is equal to 90% of the quality factor Q value for lossless, ordered transmission with the same delay, where Q = 0.9. Therefore, we obtain:

[0229]

[0230] Will:

[0231] OFO L =0.003;

[0232] OFO R =0.012;

[0233] Substituting into the above equation, we get:

[0234] β L ≈0.38;

[0235] β R ≈0.5;

[0236] Therefore, we get:

[0237]

[0238] It is understandable that, since the quality transfer factors on the left and right sides are divided into two formulas, after calculating the values ​​of the adjustment factors in the two formulas, the new user experience scoring model can also be divided into a user experience scoring model for calculating the left-side transmission quality factor and a user experience scoring model for calculating the right-side transmission quality factor.

[0239] Furthermore, the accuracy of the user experience scoring model after optimizing and adjusting the factors can be verified through the following steps:

[0240] Pass parameters to the two sets of messages at the beginning of this section:

[0241] A(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.02s / 0.1% / 0.1%, 0.01s / 1% / 1%;

[0242] B(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.2s / 1% / 1%, 0.1s / 1% / 1%;

[0243] Inputting the optimized user experience scoring model yields:

[0244] QA (QL / QR): 0.8459 / 0.8868;

[0245] QB(QL / QR): 0.4470 / 0.5217;

[0246] At this point, the transmission quality factor value of combination B is about 40% lower than that of combination A, meaning that the quality factor evaluated by the model is comparable to the actual quality factor. As a result, the user experience scoring model can better reflect the actual differences in internet browsing experience.

[0247] In summary, based on big data statistical analysis and after optimizing the adjustment factors, the transmission quality assessment vector results are as follows:

[0248]

[0249] in:

[0250] RTR L ∈[0,1]

[0251] RTT L ∈[0,+∞]

[0252] OFO L ∈[0,1]

[0253] RTR R ∈[0,1],

[0254] RTT R ∈[0,+∞]

[0255] OFO R ∈[0,1]

[0256]

[0257] In this embodiment, by introducing the variable Status and optimizing the model, the present invention can further accurately characterize the transmission quality of messages. Furthermore, by optimizing the adjustment factor, the accuracy of the calculated transmission quality factor in representing the actual transmission quality can be improved.

[0258] Based on the first and second embodiments of the message quality analysis method of the present invention described above, a third embodiment of the message quality analysis method of the present invention is proposed.

[0259] In this embodiment, when multiple target service sessions are detected, after step S2021 above, the method further includes:

[0260] Step S70: Perform a weighted average of the left transmission quality factors in each of the message transmission quality factors to obtain the average left transmission quality factor, and perform a weighted average of the right transmission quality factors in each of the message transmission quality factors to obtain the average right transmission quality factor.

[0261] Step S80: Weight each of the service status identifiers in the transmission quality of each message, and take the service status identifier with the most weighting times as the comprehensive service status identifier.

[0262] Step S90: Combine the average left transmission quality factor, the average right transmission quality factor, and the integrated service status identifier to obtain a mobile network quality monitoring result vector.

[0263] It should be noted that the mobile network quality monitoring result vector is a set of numerical values ​​or status codes used to quantify and describe the overall network performance and user experience experienced by users when using Internet services.

[0264] In this embodiment, for the numerical factors QL / QR in the evaluation vector of each packet transmission quality in each target service session, the average left-hand transmission quality factor (QLT) and average right-hand transmission quality factor (QRT) of the overall internet access can be obtained by calculating a weighted average. For the enumerated factor Status, the overall comprehensive service status identifier is obtained by statistically analyzing the Status enumeration value with the most weighted occurrences during the entire internet access quality statistical period (represented by Statusmost_common). The weighting factor can be represented by the total number of uplink / downlink packets Np in a single internet access record, with "service session flow" as the statistical unit. That is, the larger the traffic generated by a service session, the greater its impact on the overall service quality score within the statistical period.

[0265] Within a preset time period, different times are denoted as time 1, 2, 3...n, and the message (session) evaluation vector at time k is represented as:

[0266]

[0267] Simultaneously obtaining the total number of uplink / downlink packets Npk at time k allows us to reflect the overall packet evaluation vector (mobile network quality monitoring result vector) for the entire statistical period through the packet transmission quality within this preset time period. That is, by using the number of packets within this time period as weights, a weighted average of the transmission quality factors at different time points is obtained. The mobile network quality monitoring result vector is then obtained through the average value and the comprehensive service status identifier. In a feasible implementation, the mobile network quality monitoring result vector can be calculated using the following formula:

[0268]

[0269] As an example, if there are two sessions on a mobile phone: a chat session and a video chat session, assuming the chat session has 1 message and the video chat session has 10 messages, then each message in each session on the phone can be weighted using a preset weighting factor. Therefore, because the video chat session has more messages than the chat session, its status will be Statusmost_common.

[0270] Please refer to Figure 7 In this embodiment, when measuring Internet access quality, the present invention replaces the original macro-level overall service with the analysis of micro-level packets or sessions with the same IP five-tuple as the smallest unit. By quantitatively modeling the underlying packet transmission quality, independent quality assessment at the level of a single packet (single service session) is achieved. Then, the quality assessment of macro-level service behaviors (such as sending and receiving WeChat messages, watching short videos, etc.) is converted into a weighted average of multiple mobile network packet transmission quality monitoring result vectors, realizing the overall assessment process and thus effectively and quickly obtaining the mobile network quality monitoring result vector.

[0271] Based on this, combining the left-side transmission quality factor, right-side transmission quality factor, and service status identifier of a single session, an evaluation model for calculating the mobile network quality monitoring result vector can be obtained, as follows: Figure 8 As shown. Among them, Figure 8 Evaluation factor 1-1 is the left-side transmission quality factor of message 1, 1-2 is the right-side transmission quality factor of message 1, and 1-3 is the service status identifier of message 1.

[0272] Furthermore, when measuring internet quality, it is also possible to... Figure 9 The model shown is used for calculation. Among them, Figure 9 In this context, QL1 refers to the left-hand transmission quality factor of message 1, QR1 is the right-hand transmission quality factor of message 1, and Status1 is the service status identifier of message 1. The identifiers corresponding to messages 2 and 3 are similar to those of message 1, and will not be repeated here. By combining the left-hand transmission quality factor, the right-hand quality transfer factor, and the service status identifier, we can obtain the mobile network message transmission quality monitoring result vectors corresponding to messages 1 through 3. That is, we obtain Q1, Q2, and Q3. Furthermore, based on Q1, Q2, and Q3, we can obtain the mobile network quality monitoring result vector QT.

[0273] In this embodiment, the present invention transforms macroscopic to microscopic measurements, enabling rapid and accurate calculation of macroscopic internet quality. This avoids the problem of inconsistent evaluation standards inherent in traditional subjective assessments.

[0274] Based on the first, second, and third embodiments of the message quality analysis method of the present invention described above, a fourth embodiment of the message quality analysis method of the present invention is proposed.

[0275] In this embodiment, after step S10, the method further includes:

[0276] Step S100: The round-trip delays are summed to obtain the end-to-end round-trip delay;

[0277] Step S110: Determine the first difference between each of the retransmission rates and the preset threshold, and subtract the product between the preset threshold and the first difference to obtain the end-to-end retransmission rate;

[0278] Understandably, the first difference represents the probability that the data packet is successfully delivered to the receiver on the first transmission.

[0279] Step S120: Determine the second difference between each of the out-of-order rates and the preset threshold, and subtract the product between the preset threshold and the second difference to obtain the end-to-end out-of-order rate;

[0280] It should be noted that the second difference reflects the probability that the data packets successfully reach the receiver in the original sending order.

[0281] Step S130: Input the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end out-of-order rate into the user experience scoring model to obtain the end-to-end transmission quality factor.

[0282] It should be noted that the end-to-end transmission quality factor characterizes the transmission quality between two terminals transmitting messages.

[0283] As an example, end-to-end round-trip time, end-to-end retransmission rate, and end-to-end out-of-order rate can be calculated using the following formulas:

[0284]

[0285] The end-to-end round-trip time is RTT. E2E End-to-end retransmission rate is RTR E2E The end-to-end out-of-order rate is OFO. E2E Then RTT E2E RTR E2E OFO E2E By inputting the user experience rating model, the end-to-end transmission quality factor can be obtained.

[0286] Among them, RTT E2E RTR E2E OFO E2E After inputting the user experience rating model, the formula for calculating the end-to-end transmission quality factor can be:

[0287]

[0288] The end-to-end transmission quality is Q. E2E m E2E / α E2E / β E2E The value of can be calculated based on the actual network environment of the message quality to be tested.

[0289] Taking the statistical data of a certain operator's network environment in a first-tier central city as an example:

[0290] 1)RTTL / RTTR: 0.03s(30ms) / 0.02s(20ms)——》RTTE2E: 0.05s(50ms);

[0291] 2)RTRL / RTRR: 0.013(1.3%) / 0.009(0.9%)——》RTRE2E: 0.022(2.2%);

[0292] 3) OFOL / OFOR: 0.003 (0.3%) / 0.012 (1.2%)——》OFOE2E: 0.015 (1.5%);

[0293] Using the above data as anchor points, we obtain:

[0294]

[0295] Therefore, the formula for calculating the end-to-end transmission quality factor can also be:

[0296]

[0297] Based on this, please refer to Figure 10 The end-to-end message quality assessment model shown is illustrated. Figure 10 In this context, the end-to-end message transmission quality of message 1 can be determined based on the aforementioned status1 and RTT. E2E 1. RTR E2E 1 and OFO E2E 1 indicates that the end-to-end message transmission quality of message 2 can be based on the aforementioned status2 and RTT. E2E 2. RTR E2E 2 and OFO E2E 2 indicates that the end-to-end message transmission quality of message 3 can be based on the aforementioned status3 and RTT. E2E 3. RTR E2E 3 and OFO E2E 3 represents.

[0298] Optionally, in a feasible embodiment, after step S130 above, the method further includes:

[0299] Step S140: Calculate the total number of packets, actual end-to-end round-trip time, actual end-to-end retransmission rate and actual end-to-end out-of-order rate for each collection time within the preset time period.

[0300] Step S150: Take each of the actual end-to-end round-trip times as the new end-to-end round-trip times, take each of the actual end-to-end retransmission rates as the new end-to-end retransmission rates, and take each of the actual end-to-end out-of-order rates as the new end-to-end out-of-order rates. Return to the step of inputting the end-to-end round-trip times, the end-to-end retransmission rates, and the end-to-end out-of-order rates into the user experience scoring model to obtain the end-to-end transmission quality factors, until the actual end-to-end transmission quality factors corresponding to each of the collection times are obtained.

[0301] Step S160: Multiply the actual end-to-end transmission quality factor corresponding to each of the acquisition times by the total number of packets to obtain the full-cycle packet quality, and sum the total number of packets corresponding to each of the acquisition times to obtain the total number of packets in the full cycle.

[0302] Step S170: The ratio of the full-cycle message quality to the total number of full-cycle messages is used as the full-cycle end-to-end transmission quality factor.

[0303] It should be noted that the actual end-to-end round-trip time, actual end-to-end retransmission rate, and actual end-to-end out-of-order rate refer to the end-to-end round-trip time, end-to-end retransmission rate, and end-to-end out-of-order rate in actual scenarios, respectively. The total number of messages refers to the total number of uplink messages or downlink messages. The full-cycle end-to-end transmission quality factor characterizes the transmission quality of service data (various types of messages) between the two communicating ends within a certain time period.

[0304] Please refer to Figure 11 ,exist Figure 10 Based on this, after obtaining the end-to-end message transmission quality corresponding to each message, it is also possible to perform time and space aggregation based on the end-to-end message transmission quality corresponding to each message to obtain the full-cycle end-to-end transmission quality factor.

[0305] As an example, the formula for calculating the end-to-end transmission quality factor over the entire cycle can be:

[0306]

[0307] in, For end-to-end transmission quality throughout the entire cycle, K represents the acquisition time, and n represents the end time of the preset time period. N represents the end-to-end transmission quality at time k. pkThis represents the total number of messages.

[0308] In this embodiment, the present invention can calculate the full-cycle end-to-end quality factor with time dimension superimposed on spatial dimension based on the superposition result of end-to-end quality factor in time dimension, thereby achieving the effect of measuring end-to-end Internet quality within a preset time period and spatial dimension, thereby further improving the accuracy of the measured Internet quality.

[0309] In addition, the present invention also proposes a message quality analysis system.

[0310] Please refer to Figure 12 The message quality analysis system includes:

[0311] The indicator value calculation module 10 is used to obtain the round-trip delay, retransmission rate and out-of-order rate of each message pair based on multiple message pairs in the target service session.

[0312] The transmission quality determination module 20 is used to determine the message transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate.

[0313] Optionally, the transmission quality determination module 20 is also used for:

[0314] Each round-trip delay, each retransmission rate, and each out-of-order rate are input into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model. The transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the out-of-order rate. The out-of-order rate has a smaller impact on the transmission quality factor than the round-trip delay and the retransmission rate.

[0315] Based on the transmission quality factor, the message transmission quality in the target service session is obtained.

[0316] Optionally, the message acquisition module 10 is also used for:

[0317] At the target collection point in the target service session, uplink and downlink message pairs are acquired through optical splitting.

[0318] At the target collection point in the target business session, uplink and downlink message pairs are obtained through software-based monitoring of the data collection point.

[0319] Optionally, the message pair includes an uplink message pair and a downlink message pair, the message transmission direction includes left and right, and the transmission quality factor includes a left transmission quality factor and a right transmission quality factor.

[0320] The message quality analysis system also includes:

[0321] The first partitioning module is used to take the round-trip delay, retransmission rate and out-of-order rate of the downlink message pair as the left transmission quality dataset;

[0322] The second partitioning module is used to take the round-trip delay, retransmission rate and out-of-order rate of the uplink message pair as the right-side transmission quality dataset.

[0323] Based on this, the aforementioned transmission quality determination module 20 is also used for:

[0324] Input the left-side transmission quality dataset into a preset user experience scoring model to obtain the left-side transmission quality factor output by the user experience scoring model;

[0325] Input the right-side transmission quality dataset into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model.

[0326] Optionally, the message quality analysis system includes:

[0327] The first acquisition module is used to acquire the service status identifier of the target service session;

[0328] The transmission quality determination module 20 is also used for:

[0329] The message transmission quality in the target service session is obtained based on the transmission quality factor and the service status identifier.

[0330] Optionally, when multiple target service sessions are detected, the packet quality analysis system includes:

[0331] The weighting module is used to perform a weighted average of the left transmission quality factors in each of the message transmission quality factors to obtain an average left transmission quality factor, and to perform a weighted average of the right transmission quality factors in each of the message transmission quality factors to obtain an average right transmission quality factor.

[0332] The status determination module is used to weight each of the service status identifiers in the transmission quality of each message, and take the service status identifier with the most weighting times as the comprehensive service status identifier.

[0333] The Internet access quality assessment module is used to combine the average left transmission quality factor, the average right transmission quality factor, and the comprehensive service status identifier to obtain a mobile network quality monitoring result vector.

[0334] Optionally, the message quality analysis system includes:

[0335] The superposition module is used to superimpose the round-trip delays to obtain the end-to-end round-trip delay;

[0336] The first calculation module is used to determine the first difference between each of the retransmission rates and a preset threshold, and to obtain the end-to-end retransmission rate by subtracting the product between the preset threshold and the first difference.

[0337] The second calculation module is used to determine the second difference between each of the disorder rates and the preset threshold, and to subtract the product between the preset threshold and the second difference to obtain the end-to-end disorder rate.

[0338] The end-to-end calculation module is used to input the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end out-of-order rate into the user experience scoring model to obtain the end-to-end transmission quality factor.

[0339] Optionally, the message quality analysis system includes:

[0340] The second acquisition module is used to calculate the total number of packets, actual end-to-end round-trip time, actual end-to-end retransmission rate and actual end-to-end out-of-order rate at each acquisition time within the preset time period.

[0341] The loop module is used to take each of the actual end-to-end round-trip times as a new end-to-end round-trip time, each of the actual end-to-end retransmission rates as a new end-to-end retransmission rate, and each of the actual end-to-end out-of-order rates as a new end-to-end out-of-order rate, and return to execute the step of inputting the end-to-end round-trip time, the end-to-end retransmission rate, and the end-to-end out-of-order rate into the user experience scoring model to obtain the end-to-end transmission quality factor, until the actual end-to-end transmission quality factor corresponding to each of the acquisition times is obtained.

[0342] The data statistics module is used to multiply the actual end-to-end transmission quality factor corresponding to each of the acquisition times and the total number of packets to obtain the full-cycle packet quality, and to accumulate the total number of packets corresponding to each of the acquisition times to obtain the total number of packets in the full cycle.

[0343] The full-cycle quality calculation module is used to use the ratio of the full-cycle message quality to the total number of full-cycle messages as the full-cycle end-to-end transmission quality factor.

[0344] Optionally, the message quality analysis system further includes:

[0345] The model testing module is used to obtain the user experience scoring model to be tested and the transmission quality dataset to be tested, and input the transmission quality dataset to be tested into the user experience scoring model to be tested to obtain the transmission quality factor test results;

[0346] The model optimization module is used to optimize the adjustment factors in the user experience scoring model under test by using a preset reference anchor point dataset when the difference between the test result of the transmission quality factor and the actual transmission quality factor of the transmission quality dataset under test is greater than a set threshold, and to use the user experience scoring model under test after optimizing the adjustment factors as the user experience scoring model.

[0347] The functions of each module in the above message quality analysis system correspond to the steps in the above message quality analysis method embodiment, and their functions and implementation processes will not be described in detail here.

[0348] Furthermore, the present invention also proposes a message quality analysis device, which includes: a memory, a processor, and a message quality analysis program stored in the memory and executable on the processor. When the message quality analysis program is executed by the processor, it implements the steps of the message quality analysis method of the present invention as described above.

[0349] The specific embodiments of the message quality analysis device of the present invention are basically the same as the embodiments of the message quality analysis method described above, and will not be repeated here.

[0350] Furthermore, the present invention also proposes a computer storage medium storing a message quality analysis program, which, when executed by a processor, implements the steps of the message quality analysis method of the present invention as described above.

[0351] The specific embodiments of the computer storage medium of the present invention are basically the same as the embodiments of the above-described message quality analysis method, and will not be described in detail here.

[0352] Furthermore, the present invention also proposes a computer program product that stores a message quality analysis program, which, when executed by a processor, implements the steps of the message quality analysis method of the present invention as described above.

[0353] The specific embodiments of the computer program product of the present invention are basically the same as the embodiments of the above-described message quality analysis method, and will not be described in detail here.

[0354] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0355] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0356] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a message quality analysis device (which may be an in-vehicle computer, smartphone, computer, or server, etc.) to execute the methods described in the various embodiments of this application.

[0357] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A message quality analysis method, characterized in that, The message quality analysis method includes: Based on multiple message pairs in the target service session, the round-trip time, retransmission rate, and out-of-order rate corresponding to each message pair are obtained; The message transmission quality in the target service session is determined by the round-trip delay, the retransmission rate, and the out-of-order rate. The step of determining the message transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate includes: Each round-trip delay, each retransmission rate, and each out-of-order rate are input into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model. The transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the out-of-order rate. The out-of-order rate has a smaller impact on the transmission quality factor than the round-trip delay and the retransmission rate. Based on the transmission quality factor, the message transmission quality in the target service session is obtained; The user experience scoring model is as follows: Where, α L β L m L α R β R m R The adjustment factor has a value range of 0 to +∞, RTT is the round-trip time, RTR is the retransmission rate, OFO is the out-of-order rate, and Q is the retransmission time. L Q is the transmission quality factor on the left. R The right-hand transmission quality factor, RTT L For left-side delay, RTR L For left-side retransmission rate, OFO L Left-side out-of-order rate, RTT R For right-side delay, RTR R For right-side retransmission rate, OFO R The right-hand out-of-order rate is represented by Status, which is the business status identifier. The message pairs include uplink message pairs and downlink message pairs, and the transmission quality factors include left transmission quality factors and right transmission quality factors. Before the step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model, the method further includes: The round-trip time and retransmission rate of the downlink packet pairs and the out-of-order rate of the uplink packet pairs are used as the left-side transmission quality dataset. The round-trip time and retransmission rate of the uplink message pair and the out-of-order rate of the downlink message pair are used as the right-side transmission quality dataset. The step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model includes: Input the left-side transmission quality dataset into a preset user experience scoring model to obtain the left-side transmission quality factor output by the user experience scoring model; Input the right-side transmission quality dataset into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model; Prior to the step of obtaining the message transmission quality in the target service session based on the transmission quality factor, the method further includes: Obtain the service status identifier of the target service session; The step of obtaining the message transmission quality in the target service session based on the transmission quality factor includes: The message transmission quality in the target service session is obtained based on the transmission quality factor and the service status identifier. Wherein, when multiple target service sessions are detected, after the step of obtaining the message transmission quality in the target service session based on the transmission quality factor and the service status identifier, the method further includes: The left-side transmission quality factors in each of the aforementioned message transmission quality factors are weighted and averaged to obtain the average left-side transmission quality factor, and the right-side transmission quality factors in each of the aforementioned message transmission quality factors are weighted and averaged to obtain the average right-side transmission quality factor. The service status identifiers in each of the message transmission quality parameters are weighted, and the service status identifier with the highest weighting frequency is taken as the comprehensive service status identifier. By combining the average left-side transmission quality factor, the average right-side transmission quality factor, and the integrated service status identifier, a mobile network quality monitoring result vector is obtained.

2. The message quality analysis method as described in claim 1, characterized in that, After the step of obtaining the round-trip time, retransmission rate, and out-of-order rate for each packet pair based on multiple packet pairs in the target service session, the method further includes: The end-to-end round-trip time is obtained by superimposing the round-trip times of each of the above-mentioned round-trip times; The first difference between each of the retransmission rates and a preset threshold is determined, and the product between the preset threshold and the first difference is subtracted to obtain the end-to-end retransmission rate. The second difference between each of the out-of-order rates and the preset threshold is determined, and the product between the preset threshold and the second difference is subtracted to obtain the end-to-end out-of-order rate; The end-to-end round-trip time, end-to-end retransmission rate, and end-to-end out-of-order rate are input into the user experience scoring model to obtain the end-to-end transmission quality factor.

3. The message quality analysis method as described in claim 2, characterized in that, After the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end out-of-order rate into the user experience scoring model to obtain the end-to-end transmission quality factor, the method further includes: Calculate the total number of packets, actual end-to-end round-trip time, actual end-to-end retransmission rate, and actual end-to-end out-of-order rate for each collection time within the preset time period. Each of the actual end-to-end round-trip times is taken as a new end-to-end round-trip time, each of the actual end-to-end retransmission rates is taken as a new end-to-end retransmission rate, and each of the actual end-to-end out-of-order rates is taken as a new end-to-end out-of-order rate. Then, the process of inputting the end-to-end round-trip times, the end-to-end retransmission rates, and the end-to-end out-of-order rates into the user experience scoring model to obtain the end-to-end transmission quality factor is repeated until the actual end-to-end transmission quality factor corresponding to each of the collection times is obtained. Multiply the actual end-to-end transmission quality factor corresponding to each of the acquisition times by the total number of packets to obtain the full-cycle packet quality, and sum the total number of packets corresponding to each of the acquisition times to obtain the total number of packets in the full cycle. The ratio of the full-cycle message quality to the total number of full-cycle messages is used as the full-cycle end-to-end transmission quality factor.

4. The message quality analysis method as described in claim 1, characterized in that, Before the step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model, the method further includes: Obtain the user experience scoring model to be tested and the transmission quality dataset to be tested, and input the transmission quality dataset to be tested into the user experience scoring model to be tested to obtain the transmission quality factor test results; When the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality dataset under test is detected to be greater than a set threshold, the adjustment factor in the user experience scoring model under test is optimized using a preset reference anchor dataset, and the user experience scoring model under test after optimization of the adjustment factor is used as the user experience scoring model.

5. A message quality analysis system, characterized in that, The message quality analysis system includes: The indicator value calculation module is used to obtain the round-trip delay, retransmission rate and out-of-order rate of each message pair based on multiple message pairs in the target service session. The transmission quality determination module is used to determine the packet transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate. The step of determining the message transmission quality in the target service session based on the round-trip delay, the retransmission rate, and the out-of-order rate includes: Each round-trip delay, each retransmission rate, and each out-of-order rate are input into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model. The transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the out-of-order rate. The out-of-order rate has a smaller impact on the transmission quality factor than the round-trip delay and the retransmission rate. Based on the transmission quality factor, the message transmission quality in the target service session is obtained; The user experience scoring model is as follows: Where, α L β L m L α R β R m R The adjustment factor has a value range of 0 to +∞, RTT is the round-trip time, RTR is the retransmission rate, OFO is the out-of-order rate, and Q is the retransmission time. L Q is the transmission quality factor on the left. R The right-hand transmission quality factor, RTT L For left-side delay, RTR L For left-side retransmission rate, OFO L Left-side out-of-order rate, RTT R For right-side delay, RTR R For right-side retransmission rate, OFO R The right-hand out-of-order rate is represented by Status, which is the business status identifier. The message pairs include uplink message pairs and downlink message pairs, and the transmission quality factors include left transmission quality factors and right transmission quality factors. Before inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model, the method further includes: The round-trip time and retransmission rate of the downlink packet pairs and the out-of-order rate of the uplink packet pairs are used as the left-side transmission quality dataset. The round-trip time and retransmission rate of the uplink message pair and the out-of-order rate of the downlink message pair are used as the right-side transmission quality dataset. The step of inputting each of the round-trip delays, retransmission rates, and out-of-order rates into a preset user experience scoring model to obtain the transmission quality factor output by the user experience scoring model includes: Input the left-side transmission quality dataset into a preset user experience scoring model to obtain the left-side transmission quality factor output by the user experience scoring model; Input the right-side transmission quality dataset into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model; The step of obtaining the message transmission quality in the target service session based on the transmission quality factor further includes: Obtain the service status identifier of the target service session; The step of obtaining the message transmission quality in the target service session based on the transmission quality factor includes: The message transmission quality in the target service session is obtained based on the transmission quality factor and the service status identifier. Wherein, when multiple target service sessions are detected, after obtaining the message transmission quality in the target service session based on the transmission quality factor and the service status identifier, the method further includes: The left-side transmission quality factors in each of the aforementioned message transmission quality factors are weighted and averaged to obtain the average left-side transmission quality factor, and the right-side transmission quality factors in each of the aforementioned message transmission quality factors are weighted and averaged to obtain the average right-side transmission quality factor. The service status identifiers in each of the message transmission quality parameters are weighted, and the service status identifier with the highest weighting frequency is taken as the comprehensive service status identifier. By combining the average left-side transmission quality factor, the average right-side transmission quality factor, and the integrated service status identifier, a mobile network quality monitoring result vector is obtained.

6. A message quality analysis device, characterized in that, The message quality analysis device includes: a memory, a processor, and a message quality analysis program stored in the memory and executable on the processor. When the message quality analysis program is executed by the processor, it implements the steps of the message quality analysis method as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, The computer storage medium stores a message quality analysis program, which, when executed by a processor, implements the steps of the message quality analysis method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a message quality analysis program, which, when executed by a processor, implements the steps of the message quality analysis method as described in any one of claims 1 to 4.

Images