Message quality analysis method, system and device, storage medium and product

Abstract

The invention provides a message quality analysis method, system and device, a storage medium and a product, which are applied to the technical field of network communication, and the method comprises the following steps: according to a plurality of message pairs in a target service session, obtaining round-trip delay, retransmission rate and disorder rate corresponding to each message pair; and determining the message transmission quality in the target service session according to each round-trip delay, each retransmission rate and each out-of-order rate. The technical scheme of the invention aims to solve the technical problem of low accuracy of a message quality measurement result obtained by a traditional method.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a message quality analysis method, system, equipment, storage medium and product. Background Art

[0002] Mobile communication networks have developed from traditional 2G (Second Generation) networks to current 5G (Fifth Generation) networks since 2006. With the development of mobile communication networks, the complexity of mobile communication networks is also increasing.

[0003] Nowadays, the Internet access quality of mobile communication networks is usually evaluated through subjective evaluation (for example, evaluation based on users' network usage experience) and objective evaluation (for example, evaluation through speed test software).

[0004] However, since subjective and objective evaluations are affected by different evaluation standards and poor universality, the accuracy of the message quality measurement results obtained by traditional evaluation methods is low and it is difficult to correctly characterize the actual Internet access quality. Summary of the invention

[0005] The present invention provides a message quality analysis method, system, device, storage medium and product, aiming to solve the technical problem that the accuracy of message quality measurement results obtained by traditional evaluation methods is low.

[0006] To solve the above problem, the present invention proposes a message quality analysis method, which includes:

[0007] According to multiple message pairs in the target service session, the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs are obtained;

[0008] The message transmission quality in the target service session is determined based on the round-trip delays, the retransmission rates, and the out-of-order rates.

[0009] Optionally, the step of determining the message transmission quality in the target service session based on each of the round-trip delays, each of the retransmission rates, and each of the out-of-order rates includes:

[0010] Inputting each of the round-trip delays, each of the retransmission rates, and each of the disorder rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, wherein the transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the disorder rate, and the degree of influence of the disorder rate on the transmission quality factor is less than the degree of influence of the round-trip delay and the retransmission rate on the transmission quality factor;

[0011] The message transmission quality in the target service session is obtained according to the transmission quality factor.

[0012] Optionally, the message pair includes: an uplink message pair and a downlink message pair, and the transmission quality factor includes: a left transmission quality factor and a right transmission quality factor;

[0013] Before the step of inputting each of the round trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, the method further includes:

[0014] The round trip delay and retransmission rate corresponding to the downlink message pair and the disorder rate corresponding to the uplink message pair are used as the left transmission quality data set;

[0015] The round trip delay and retransmission rate corresponding to the uplink message pair and the disorder rate corresponding to the downlink message pair are used as the right transmission quality data set;

[0016] The step of inputting each of the round trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model includes:

[0017] Inputting the left transmission quality data set into a preset user experience scoring model to obtain a left transmission quality factor output by the user experience scoring model;

[0018] The right-side transmission quality data set is input into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model.

[0019] Optionally, before the step of obtaining the message transmission quality in the target service session according to the transmission quality factor, the method further includes:

[0020] Obtaining a service status identifier of the target service session;

[0021] The step of obtaining the message transmission quality in the target service session according to the transmission quality factor comprises:

[0022] The message transmission quality in the target service session is obtained according to the transmission quality factor and the service state identifier.

[0023] Optionally, when it is detected that the number of the target service sessions is multiple, after the step of obtaining the message transmission quality in the target service session according to the transmission quality factor and the service status identifier, the method further includes:

[0024] Performing a weighted average on the left transmission quality factors in the message transmission qualities to obtain an average left transmission quality factor, and performing a weighted average on the right transmission quality factors in the message transmission qualities to obtain an average right transmission quality factor;

[0025] Weighting each of the service status identifiers in the message transmission quality, and taking the service status identifier with the largest weighting times as the comprehensive service status identifier;

[0026] The average left transmission quality factor, the average right transmission quality factor and the comprehensive service state identifier are combined to obtain a mobile network quality monitoring result vector.

[0027] Optionally, after the step of obtaining the round-trip delay, retransmission rate and out-of-order rate corresponding to each of the message pairs according to the multiple message pairs in the target service session, the method further includes:

[0028] Adding the round trip delays to obtain the end-to-end round trip delay;

[0029] Determine a first difference between each of the retransmission rates and a preset threshold value, and subtract the product of the preset threshold value and the first difference to obtain an end-to-end retransmission rate;

[0030] Determine a second difference between each out-of-sequence rate and the preset threshold respectively, and subtract the product of the preset threshold and the second difference to obtain an end-to-end out-of-sequence rate;

[0031] The end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate are input into the user experience scoring model to obtain an end-to-end transmission quality factor.

[0032] Optionally, after the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain the end-to-end transmission quality factor, the method further includes:

[0033] Calculate the total number of messages, actual end-to-end round-trip delay, actual end-to-end retransmission rate and actual end-to-end disorder rate corresponding to each collection time within a preset time period;

[0034] Taking each of the actual end-to-end round-trip delays as a new end-to-end round-trip delay, taking each of the actual end-to-end retransmission rates as a new end-to-end retransmission rate, and taking each of the actual end-to-end disorder rates as a new end-to-end disorder rate, and returning to the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain an end-to-end transmission quality factor, until the actual end-to-end transmission quality factor corresponding to each of the collection moments is obtained;

[0035] The actual end-to-end transmission quality factor corresponding to each of the collection moments is multiplied by the total number of messages to obtain the message quality of the entire period, and the total number of messages corresponding to each of the collection moments is accumulated to obtain the total number of messages of the entire period;

[0036] The ratio of the full-cycle message quality to the total number of full-cycle messages is used as the full-cycle end-to-end transmission quality factor.

[0037] Optionally, before the step of inputting each of the round-trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, the method further includes:

[0038] Acquire a user experience scoring model to be tested and a transmission quality data set to be tested, and input the transmission quality data set to be tested into the user experience scoring model to be tested to obtain a transmission quality factor test result;

[0039] When it is detected that the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality data set to be tested is greater than a set threshold, the adjustment factor in the user experience scoring model to be tested is optimized through a preset reference anchor point data set, and the user experience scoring model to be tested after the optimized adjustment factor is used as the user experience scoring model.

[0040] In addition, to solve the above problem, the present invention also proposes a message quality analysis system, the message quality analysis system comprising:

[0041] An indicator value calculation module, used to obtain the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs according to multiple message pairs in the target service session;

[0042] The transmission quality determination module is used to determine the message transmission quality in the target service session based on each of the round-trip delays, each of the retransmission rates, and each of the disorder rates.

[0043] In addition, to solve the above-mentioned problems, the present invention also proposes a message quality analysis device, which includes: a memory, a processor, and a message quality analysis program stored in the memory and executable on the processor, and the message quality analysis program implements the steps of the message quality analysis method described above when executed by the processor.

[0044] In addition, to solve the above problem, the present invention also proposes a computer storage medium, on which a message quality analysis program is stored, and when the message quality analysis program is executed by a processor, the steps of the message quality analysis method described above are implemented.

[0045] In addition, to solve the above problem, the present invention also proposes a computer program product, which includes a message quality analysis program, and when the message quality analysis program is executed by a processor, the steps of the message quality analysis method described above are implemented.

[0046] In this embodiment, the present invention obtains the round-trip delay, retransmission rate and disorder rate corresponding to each message pair according to multiple message pairs in the target service session, and can extract the corresponding message pairs in the session where the message transmission quality needs to be measured, and obtain indicators for characterizing the transmission quality level; then, the message transmission quality in the target service session is determined by each of the round-trip delays, each of the retransmission rates and each of the disorder rates, and the message transmission quality in the target service session can be comprehensively determined based on the multiple indicators obtained.

[0047] Compared with the traditional measurement scheme, on the one hand, the present invention can avoid the problem of poor universality caused by the speed measurement equipment being set in the conversation terminal in the objective evaluation by calculating the round-trip delay, retransmission rate and disorder rate. On the other hand, the present application can also calculate the message transmission quality through objective data such as round-trip delay, retransmission rate and disorder rate, thereby avoiding the problem of different quantitative standards in subjective evaluation. Therefore, since the round-trip delay and other data used in the present invention are all real data in message transmission, and the present invention is not affected by the conversation terminal and subjective judgment when calculating the message transmission quality, compared with the traditional scheme, the present application improves the accuracy of the message quality measurement results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment of the message quality analysis device involved in the embodiment of the present invention;

[0051] Figure 2 It is a flow chart of the first embodiment of the message quality analysis method of the present invention;

[0052] Figure 3 A schematic diagram of spectral collection of an embodiment of a message quality analysis method of the present invention;

[0053] Figure 4A schematic diagram of software monitoring data collection points in accordance with an embodiment of a message quality analysis method of the present invention;

[0054] Figure 5 A schematic diagram of left and right data directions of an embodiment of a message quality analysis method of the present invention;

[0055] Figure 6 A schematic diagram of server disconnection in an embodiment of a message quality analysis method of the present invention;

[0056] Figure 7 A schematic diagram of an original model for Internet quality assessment according to an embodiment of a method for analyzing message quality according to the present invention;

[0057] Figure 8 A second schematic diagram of an Internet access quality assessment model according to an embodiment of a message quality analysis method of the present invention;

[0058] Fig. 9 A fourth schematic diagram of an Internet access quality assessment model according to an embodiment of a message quality analysis method of the present invention;

[0059] Fig.10 A fifth schematic diagram of an Internet access quality assessment model according to an embodiment of a message quality analysis method of the present invention;

[0060] Fig.11 A sixth schematic diagram of an Internet access quality assessment model according to an embodiment of a message quality analysis method of the present invention;

[0061] Fig.12 It is a schematic diagram of functional modules of an embodiment of a message quality analysis system of the present invention. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0065] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0066] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0067] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment of the message quality analysis device involved in the embodiment of the present invention.

[0068] like Figure 1 As shown, in the hardware operating environment of the message quality analysis device, the message quality analysis device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0069] Those skilled in the art will understand that Figure 1 The structure of the message quality analysis device shown in the figure does not constitute a limitation on the message quality analysis device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0070] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a message quality analysis program.

[0071] exist Figure 1 In the device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the message quality analysis program stored in the memory 1005 and perform the following operations:

[0072] According to multiple message pairs in the target service session, the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs are obtained;

[0073] The message transmission quality in the target service session is determined based on the round-trip delays, the retransmission rates, and the out-of-order rates.

[0074] Based on the above hardware structure, the overall concept of each embodiment of the message quality analysis method of the present invention is proposed.

[0075] In this embodiment, nowadays, the Internet access quality of the mobile communication network is usually evaluated by subjective evaluation (for example, evaluation based on the user's network usage experience) or objective evaluation (for example, evaluation through speed test software).

[0076] However, since subjective and objective evaluations are affected by different evaluation standards and poor universality, the accuracy of the message quality measurement results obtained by traditional evaluation methods is low and it is difficult to correctly characterize the actual Internet access quality.

[0077] In order to solve the above problems, a message quality analysis method is proposed.

[0078] Based on the overall concept of each embodiment of the message quality analysis method of the present invention, various embodiments of the message quality analysis method of the present invention are proposed.

[0079] It should be noted that the execution subject of each embodiment of the message quality analysis method of the present invention is the above-mentioned message quality analysis device, which can be a terminal such as a computer or a server. For ease of description, the execution subject is omitted in the following embodiments.

[0080] Please refer to Figure 2 , Figure 2 It should be noted that although the flowchart shows a logical sequence, in some cases, the steps of the message quality analysis method of the present invention can be performed in a different order than that shown here.

[0081] In this embodiment, the message quality analysis method includes:

[0082] Step S10, obtaining the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs according to the multiple message pairs in the target service session;

[0083] It should be noted that the target service session refers to the current session for which the message transmission quality needs to be measured. For example, if a public application or an industry application is installed in the user terminal, if the current session includes a certain message chat, and the message transmission quality of the certain message chat needs to be measured, the session between the terminal device (such as a mobile phone) of the certain message chat and the server can be used as the target service session. In order to prevent the data collected by the service terminal from being affected by the terminal itself, a target collection point can be set in the target service session. In other words, the target collection point is a predetermined message collection point in the message transmission path in the target service session. Round trip delay refers to the total time required from the sender sending a data packet to the receiver receiving the data packet and sending an acknowledgment (ACK) signal, and then returning to the sender to receive the acknowledgment signal. The retransmission rate refers to the proportion of data packets that cannot be successfully received due to various reasons (such as network congestion, data packet loss, etc.) in network communication, and thus need to be retransmitted. The out-of-order rate refers to the proportion of the order in which the receiver receives the data packets inconsistent with the order in which the sender sends the data packets.

[0084] In this embodiment, when it is detected that the message transmission quality in a certain session needs to be measured, the session can be used as a target service session, and each message pair can be obtained at a pre-set message collection point in the message transmission path of the session to calculate the round-trip delay, retransmission rate and disorder rate corresponding to each message pair.

[0085] In a feasible implementation, the message pair includes an uplink message pair and a downlink message pair, wherein the uplink message pair refers to a first request message sent by a terminal device in a target service session to a server and a first response message sent by the server corresponding to the first request message. Similarly, the downlink message pair refers to a second request message sent by a server in a target service session to a terminal device and a second response message sent by the terminal device corresponding to the second request message. Among them, the target service session includes each uplink message pair and each downlink message pair, and a typical "target service session" includes but is not limited to a message set with the same IP five-tuple (source / destination IP address, source / destination layer 4 port, service type TOS).

[0086] When calculating the round-trip delay, for a single "uplink message pair" and "downlink message pair", when the uplink message pair includes a first request message and a first response message, and the downlink message pair includes a second request message and a second response message, the round-trip delay corresponding to the uplink message pair can be obtained based on the difference between the timestamps of the first request message and the first response message. Similarly, the round-trip delay corresponding to the downlink message pair can be obtained based on the difference between the timestamps of the second request message and the second response message.

[0087] For each uplink message pair and each downlink message pair obtained, for each pair of "uplink message pair" or "downlink message pair", firstly refer to the method of calculating the round-trip delay of the above-mentioned single "uplink message pair" and "downlink message pair" to calculate the round-trip delay, and calculate the round-trip delay of each uplink message or downlink message pair. Then, by taking the arithmetic average of the delay set data of the calculated "uplink message pair" set and "downlink message pair" set, the average round-trip delay of the "uplink message pair" and "downlink message pair" in the current "target service session" is obtained; for each pair of "uplink message pair" or "downlink message pair" obtained, if only the first / second request message is obtained but the first / second response message is not obtained, the request message is recorded as a retransmission, and the total number of the first / second request messages that have not collected the first / second response message is divided by the total number of all first / second request messages to obtain the retransmission rate statistics of the corresponding direction of a "service session". For each pair of "uplink message pairs" or "downlink message pairs", if only a certain first / second request message is obtained at the collection point, and the order is misaligned with other received first / second request messages, then the request message is recorded as a disorder, and the total number of disordered first / second request messages is divided by the total number of all first / second request messages to obtain the disorder rate statistics of the corresponding direction of a "service session". Thus, the round-trip delay, retransmission rate and disorder rate corresponding to each uplink message pair and each downlink message pair can be obtained.

[0088] Optionally, in a feasible embodiment, the above step S10 further includes:

[0089] Step S101, at a target collection point in a target service session, each uplink message pair and each downlink message pair are acquired by optical spectrometry collection;

[0090] Please refer to Figure 3 , Figure 3 The figure is a schematic diagram of the spectral collection of an embodiment of the message quality analysis method of the present invention. UE refers to user equipment, SPN refers to the network slice supported in the 5G network, and PDN servers include but are not limited to service servers, CDN servers, edge CDNs, etc. Figure 3 In the monitoring data collection point, the target collection point is used to facilitate data collection and storage. Figure 3 A monitoring data processing platform is also provided for collecting and storing data, so that online log records (including monitoring result data and calculated monitoring indicators) can be output at any time.

[0091] exist Figure 3 In the example, when the user equipment and the server are in conversation, the monitoring data collection point is set at a position between the N2 interface and the N3 interface, so that the uplink message pair and the downlink message pair can be collected at the monitoring data collection point.

[0092] Step S102: at a target collection point in a target service session, each uplink message pair and each downlink message pair are acquired through software-based monitoring data collection points.

[0093] Please also refer to Figure 3 and Figure 4 , Figure 4 This is a schematic diagram of software monitoring data collection points in an embodiment of the message quality analysis method of the present invention. Figure 3 In comparison, Figure 4 In the message transmission path shown, software-based monitoring data collection points can also be deployed inside the network elements of the communication core network to realize the collection of uplink message pairs and downlink message pairs.

[0094] Step S20: determining the message transmission quality in the target service session according to the round-trip delays, the retransmission rates and the disorder rates.

[0095] It is understandable that a larger round trip delay value indicates that the time required to transmit data is longer, and the message transmission quality is worse. Similarly, the values ​​of the retransmission rate and the disorder rate can also indicate the message transmission quality.

[0096] As an example, scores corresponding to the specific values ​​of round-trip delay, retransmission rate and out-of-order rate can be pre-set, and then the scores can be weighted and synthesized based on the weights corresponding to the round-trip delay, retransmission rate and out-of-order rate, so as to obtain a comprehensive network quality score for the target service session, and then the message transmission quality can be determined based on the comprehensive network quality score.

[0097] Optionally, in a feasible implementation manner, the above step S20 includes:

[0098] Step S201, inputting each of the round-trip delays, each of the retransmission rates, and each of the disorder rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, wherein the transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the disorder rate, and the degree of influence of the disorder rate on the transmission quality factor is less than the degree of influence of the round-trip delay and the retransmission rate on the transmission quality factor;

[0099] It should be noted that the user experience scoring model is a model that takes round-trip delay, retransmission rate, and disorder rate as input data and transmission quality factor as output data. The transmission quality factor represents the transmission quality of the service data. In a feasible implementation, the user experience scoring model can be a pre-set formula or a pre-trained machine learning algorithm model, and this application does not limit this.

[0100] Step S202: obtaining the message transmission quality in the target service session according to the transmission quality factor.

[0101] It should be noted that the message transmission quality refers to the transmission effect of the data packet in the network. The higher the message transmission quality, the better the transmission effect of the data packet.

[0102] It can be understood that in a feasible implementation, after the transmission quality factor is obtained, the transmission quality factor can be used as the message transmission quality, or the quality standard corresponding to the value range of the transmission quality factor can be used as the message transmission quality.

[0103] It should also be noted that since the target collection point is located between the user terminal and the server, the round-trip time (RTT), retransmission rate (RTR), out-of-order rate (OFO) and other data collected can actually be divided into two directions: for example Figure 3 There are two message transmission directions: 1) RTT / RTR / OFO from the collection point to the user terminal (to the left); 2) RTT / RTR / OFO from the collection point to the server (to the right). Therefore, in addition to directly inputting each round-trip delay, each retransmission rate and each disorder rate into the model to obtain the transmission quality factor, the transmission quality factor can also be divided into a left transmission quality factor and a right transmission quality factor according to the message transmission direction, including left and right. Therefore, before the above step S20, the round-trip time, each retransmission rate and each disorder rate obtained in the above step S10 can also be classified.

[0104] Based on this, in a feasible implementation manner, the method further includes:

[0105] Step S200, taking the round trip delay and retransmission rate corresponding to the downlink message pair and the disorder rate corresponding to the uplink message pair as the left transmission quality data set;

[0106] Step S210: The round trip delay and retransmission rate corresponding to the uplink message pair and the disorder rate corresponding to the downlink message pair are used as a right transmission quality data set.

[0107] As an example, see Figure 5, taking the user equipment as a mobile phone as an example, for the uplink request message initiated by the mobile phone and the corresponding server response message (uplink message pair) obtained at the collection point, the round-trip delay RTT and retransmission rate RTR obtained based on the statistics of the two actually reflect the message transmission quality of the right path (core network + bearer network + server) of the collection point, which are recorded as right-side delay (RTTR) and right-side retransmission rate (RTRR) respectively; the out-of-order rate OFO obtained by statistics actually reflects the disorder of the incoming message on the left path (transmission network + wireless base station + terminal), so it is recorded as the left disorder rate (OFOL). For the downlink request message initiated by the server and the corresponding mobile phone response message (downlink message pair) obtained at the collection point, the round-trip delay RTT and retransmission rate RTR obtained based on the statistics of the two actually reflect the message transmission quality of the left path of the collection point, which are recorded as left-side delay (RTTL) and left-side retransmission rate (RTRL) respectively; the out-of-order rate OFO obtained by statistics actually reflects the disorder of the incoming message on the right direction, so it is recorded as the right disorder rate (OFOR).

[0108] Please continue to refer to Figure 5 It should be noted that Req_UL refers to a request or uplink transmission request (first request message) sent from the user equipment (UE) to the network side, Resp refers to the network side's response to the request (Req) sent by the user equipment (UE) (first response message), Req_DL refers to a request or downlink transmission request (second request message) sent from the network side to the user equipment, and Resp refers to a request or downlink transmission request (second response message) sent from the network side to the user equipment. In addition, Figure 5 In the figure, the transmission quality datasets on the left include RTTL, RTRL, and OFOL, and the transmission quality datasets on the right include RTTR, RTRR, and OFOR.

[0109] Based on this, the above step S201 further includes:

[0110] Step S2011, inputting the left transmission quality data set into a preset user experience scoring model to obtain a left transmission quality factor output by the user experience scoring model;

[0111] Step S2012: input the right transmission quality data set into the user experience scoring model to obtain the right transmission quality factor output by the user experience scoring model.

[0112] Thus, the transmission quality factors on the left and right sides of the target collection point can be obtained by processing the left transmission quality data set and the right transmission quality data set respectively through the user experience scoring model. Compared with the above method of directly inputting indicators such as round-trip delay into the model, the method of dividing the indicators by transmission direction in this embodiment can more accurately reflect the message transmission conditions of different transmission paths, thereby obtaining a more accurate transmission quality factor.

[0113] As an example, to obtain a user experience scoring model, boundary constraints are first required, namely:

[0114] The "worst" and "best" states of message transmission quality are defined as:

[0115] The worst transmission quality: the receiver cannot receive any message from the sender;

[0116] The best transmission quality: The receiver can receive all messages completely (without packet loss) and in order (without disorder) at the instant when the sender sends the message (transmission delay is 0). At this time, all message sending and receiving are completed instantly, and the transmission quality is the best.

[0117] Then define the transmission quality factors of the "best" and "worst" message (session) transmission quality as 1 and 0 respectively. At this time, it can be further obtained that the delay, retransmission rate, disorder rate and quality factor have the following correlation and boundary constraint relationship:

[0118] 1. The message transmission delay and quality factor (message transmission quality) show an inverse correlation:

[0119] 1) The greater the delay, the worse the transmission quality and the smaller the quality factor. The quality factor decreases monotonically as the delay increases.

[0120] 2) When the delay approaches infinity, the receiver approaches not receiving any messages from the sender, which meets the above definition. At this time, the transmission quality approaches the worst. According to the above definition, the quality factor approaches 0.

[0121] 3) When the delay approaches zero, if the receiver can receive all messages completely and in order (the retransmission rate and disorder rate are both 0), it meets the above definition. At this time, the transmission quality approaches the best. According to the above definition, the quality factor approaches 1;

[0122] 2. The message transmission retransmission rate and the message transmission quality / quality factor show an inverse correlation:

[0123] 1) The larger the retransmission rate, the worse the transmission quality and the smaller the quality factor. The quality factor decreases monotonically with the increase of the retransmission rate.

[0124] 2) When the retransmission rate approaches 100%, it means that the receiver is close to not receiving any messages from the sender, which meets the above definition. At this time, the transmission quality approaches the worst. According to the above definition, the quality factor approaches 0;

[0125] 3) When the retransmission rate approaches zero, if the receiver can receive all messages instantly and in order (delay and disorder rate are both 0), it meets the above definition. At this time, the transmission quality approaches the best. According to the above definition, the quality factor approaches 1;

[0126] 3. The message delivery disorder rate and the message transmission quality / quality factor show an inverse correlation:

[0127] 1) The greater the disorder rate, the worse the transmission quality, and the quality factor decreases monotonically with the increase of disorder rate;

[0128] 2) When the disorder rate approaches 100%, it means that the receiver can receive the message, but the transmission quality is poor (not the worst). This is the "complete disorder point". It is further defined as: the quality factor is 0.5 when the disorder rate is 100%, the delay is 0, and the retransmission rate is 0%;

[0129] 3) When the disorder rate approaches zero, if the receiver can receive all messages instantly and completely (delay and retransmission rate are both 0), it meets the above definition. At this time, the transmission quality approaches the best. According to the above definition, the quality factor approaches 1;

[0130] In summary, the boundary constraint relationship between message transmission quality and delay, retransmission rate, and disorder rate is shown in Table 1 below:

[0131] Table 1

[0132] Quality Factor Round Trip Delay Retransmission rate Disorder rate 1 0 0 0 0 any 100 any 0 +∞ any any 0.5 0 0 100

[0133] Then, the model can be constructed according to the constraints defined in Table 1:

[0134] It should be noted that any functional relationship between the quality factor Q and the delay RTT, retransmission rate RTR, and out-of-order rate OFO that meets the boundary constraints of Table 1 can be applied to the establishment of the functional relationship between the quality factor and RTT / RTR / OFO proposed in this application, that is:

[0135] Q = f(RTT,RTR,OFO);

[0136] In addition, the left / right transmission quality factors QL / QR (the quality factor corresponding to the left transmission quality data set, and the quality factor corresponding to the right transmission quality data set) can be independently calculated based on the above function form through the left / right RTTL / RTTR, RTRL / RTRR, OFOL / OFOR, that is:

[0137]

[0138] Furthermore, the general functional relationship between the quality factor Q and RTT / RTR / OFO that satisfies the boundary constraints in Table 1 can also be expressed as follows: the transmission quality Q is equal to the product of the delay quality factor QRTT, the retransmission quality factor QRTR and the out-of-order quality factor QOFO, that is:

[0139] Q(RTT,RTR,OFO)=Q RTT (RTT)*Q RTR (RTR)*Q OFO (OFO);

[0140] in,

[0141]

[0142] Based on the general function relationship and value conditions of the above formula, we can construct (but not limited to) the following function factor expression:

[0143]

[0144] Then the user experience scoring model used to calculate the transmission quality factor (Q) can be:

[0145]

[0146] Replace RTR in the above formula with RTRL and RTRR, replace RTT with RTTL and RTTR, and replace OFO with OFOL and OFOR, then the left transmission quality factor and the right transmission quality factor can be obtained. That is, the formula for calculating the left transmission quality factor and the right transmission quality factor can be:

[0147]

[0148] The meaning and value sources of different parameters in the formula:

[0149] RTTL and RTRL correspond to the downlink round-trip delay and downlink retransmission rate respectively, and OFOL corresponds to the uplink out-of-order rate, which reflects the message delivery situation on the downlink path; RTTR and RTRR correspond to the uplink round-trip delay and downlink retransmission rate respectively, and OFOR corresponds to the downlink out-of-order rate, which reflects the message delivery situation on the uplink path.

[0150] It can be understood that the message transmission quality can also be a vector that can be used to characterize the message delivery situation. Therefore, in this embodiment, when all path directions include left and right, QL and QR can be used as elements of the vector, that is, QL and QR are combined to obtain the vector Q, that is:

[0151] Q = (QL, QR);

[0152] Therefore, the transmission quality of the message in the target service session can be represented by the transmission quality factor on the left and the quality factor transferred on the right.

[0153] In this embodiment, compared with the traditional measurement scheme, on the one hand, the present invention can avoid the problem of poor universality caused by the speed measurement equipment being set in the conversation terminal in the objective evaluation by calculating the round-trip delay, retransmission rate and disorder rate at the target collection point. On the other hand, the present application also inputs the transmission quality data sets of different path directions into the pre-set user experience scoring model, and can quantify the impact of the round-trip delay, retransmission rate and disorder rate on the message transmission quality through the set model, thereby avoiding the problem of different quantitative standards in subjective evaluation. In addition, since the round-trip delay and other data used in the present invention are all real data in message transmission, and the present invention is not affected by the conversation terminal and subjective judgment when measuring the message transmission quality, compared with the traditional scheme, the present application improves the accuracy of the message quality measurement results.

[0154] Based on the first embodiment of the message quality analysis method of the present invention, a second embodiment of the message quality analysis method of the present invention is proposed.

[0155] In this embodiment, before the above step S202, the method further includes:

[0156] Step S50, obtaining a service status identifier of the target service session;

[0157] It should be noted that the service status identifier refers to a status code information used to indicate a specific service status or event during network communication. Specifically, the service status identifier includes but is not limited to successful completion, abnormal termination, and the like of the service.

[0158] The end-to-end service quality is affected not only by the transmission quality of the message on the intermediate path, but also by the behavior of the terminal / server itself. Analyzing only the transmission quality of the message on the path cannot fully reflect the overall service quality.

[0159] For example Figure 6 In the TCP message interaction between the terminal and the server, the forwarding delay, packet loss, disorder and other indicators of the message in the path are normal. However, after the TCP handshake, the server actively sends the Rst message to reset the session, resulting in service interruption. In this scenario, it is necessary to introduce additional status code information reflecting "service terminated by the server" to accurately reflect the service quality. It is understandable that Figure 6The business interruption scenario is only an example. In other business scenarios, there is also a need to introduce status code information, and the present invention is not limited to this.

[0160] Therefore, in the message quality assessment model, for the "message (session) quality assessment vector", an additional "service status identifier" Status that reflects the end-to-end status of the service can be introduced. For different protocol stack types, different service status identifiers are given in the form of enumerated values.

[0161] For example, for a single TCP session with the same IP quintuple as the minimum statistical granularity, the following eight common enumeration values ​​can be used:

[0162] Success - indicates that the TCP / UDP session process starts and ends normally, and the data transmission is completed;

[0163] Server link establishment timeout - after the client sends the first SYN, it does not receive the SYN ACK message from the server;

[0164] Client link establishment timeout - after the server sends a SYN ACK message, it does not receive the client's ACK message;

[0165] Transmission timeout - After the TCP three-way handshake is successful, if the TCP session does not end with RST / FIN / FIN ACK, it is considered to have timed out;

[0166] The service is rejected by the server - the response code of the application layer protocol representing the server's refusal of service is obtained; the service is rejected by the client - the response code of the application layer protocol representing the client's refusal of service is obtained;

[0167] Server abort - the server sends RST to terminate the session;

[0168] Client abort - The client sends a RST to abort the session.

[0169] According to the above status classification, for a TCP message or session, the "service status identifier" at this time can be expressed as:

[0170]

[0171] It should be noted that the value of the service status identifier is not limited to the above 8. Under different network environments, different communication requirements and different communication objects, the value of the service status identifier can be flexibly set, and this application does not limit this.

[0172] Based on this, the above step S202 further includes:

[0173] Step S2021: Obtain the message transmission quality in the target service session according to the transmission quality factor and the service status identifier.

[0174] In this embodiment, the vector corresponding to the message transmission quality may be:

[0175] Q = (QL, QR, Status);

[0176] It can be understood that after adding the service status identifier status, the prototype of the message (session) quality evaluation vector can be expressed as follows:

[0177]

[0178] in:

[0179] RTR L ∈[0,1]

[0180] RTT L ∈[0,+∞]

[0181] OFO L ∈[0,1]

[0182] RTR R ∈[0,1],

[0183] RTT R ∈[0,+∞]

[0184] OFO R ∈[0,1]

[0185]

[0186] Optionally, in a feasible embodiment, before the step of inputting each of the round-trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, the method further includes:

[0187] Step X, obtaining a user experience scoring model to be tested and a transmission quality data set to be tested, and inputting the transmission quality data set to be tested into the user experience scoring model to be tested to obtain a transmission quality factor test result;

[0188] Step Y, when it is detected that the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality data set to be tested is greater than a set threshold, the adjustment factor in the user experience scoring model to be tested is optimized through a preset reference anchor point data set, and the user experience scoring model to be tested after the optimized adjustment factor is used as the user experience scoring model.

[0189] It can be understood that, when the following formula is obtained:

[0190]

[0191] Afterwards, the transmission quality factors on the left and right sides can be calculated according to the formula, and the formula can also be optimized and the optimized formula can be used as a new user experience scoring model.

[0192] It should be noted that the user experience scoring model to be tested refers to the above-mentioned formula for calculating Q, and the adjustment factor refers to the exponent or multiple of RTR, RTT, and OFO in the formula. In the above formula, the exponents of RTR, RTT, and OFO are all 1. In order to improve the accuracy of the calculated Q, the accuracy of the transmission quality obtained by the user experience scoring model to be tested can also be judged according to the transmission quality data set to be tested (including round-trip delay, disorder rate, and retransmission rate), and then the exponent or multiple of RTR, RTT, and OFO can be re-determined by referring to the anchor point data set (including the pre-set transmission quality factor, round-trip delay, retransmission rate, and disorder rate), so as to optimize each exponent or multiple and obtain a new user experience scoring model.

[0193] As an example, there are two sets of data in the transmission quality data set to be tested:

[0194] A(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.02s / 0.1% / 0.1%, 0.01s / 1% / 1%;

[0195] B(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.2s / 1% / 1%, 0.1s / 1% / 1%;

[0196] The above two groups of data A and B can be entered separately:

[0197]

[0198] The transmission quality factor test results are:

[0199] QA(QL / QR):0.9784 / 0.9881;

[0200] QB(QL / QR):0.8168 / 0.8911;

[0201] It can be seen that when the delay / retransmission / disorder in combination B is generally 10 times worse than that in combination A, the change in the transmission quality factor is only less than 20%. In other words, the difference between the tested transmission quality factor and the actual transmission quality factor is greater than the set threshold, resulting in the change in the quality factor unable to reflect the change in the actual service quality. Therefore, without changing the original boundary constraints, the exponents or multiples of RTR, RTT, and OFO are modified as follows:

[0202]

[0203] Among them, αL / βL / mL and αR / βR / mR are adjustment factors with a value range of 0 to +∞.

[0204] By adding the adjustment factor, the transmission quality factor can be differentiated to present the concentrated area of ​​the indicator distribution on the investigated path (left / right path).

[0205] It can be understood that when calculating the value of the adjustment factor, the average value of the delay, retransmission rate, and disorder rate of the specific network environment to which the user experience scoring model to be tested is applied can be used as a "reference anchor point". By inputting the above-mentioned "reference anchor point" value (i.e., the reference anchor point data set) into the model of the adjustment factor, the reasonable "adjustment factor" value under the specific network environment can be reversely deduced.

[0206] For example, taking a first-tier city network environment as an example, in this network environment, the data statistics obtained by the Internet log collection method proposed in the proposal require the following values ​​for left / right delay, retransmission rate, and disorder rate:

[0207] RTTL=0.03(seconds), RTTR=0.02(seconds);

[0208] RTRL=0.013 (1.3%), RTRR=0.009 (0.9%);

[0209] OFOL=0.003 (0.3%), OFOR=0.012 (1.2%);

[0210] 1) Taking RTTL=0.03, RTTR=0.02, lossless (RTR=0), and ordered (OFO=0) transmission under QL and QR equal to 0.9 as reference anchor point 1, further obtain different m values, and obtain the left and right transmission quality factors QL and QR as follows:

[0211]

[0212] 2) For the same left and right RTT delays (RTTL = 0.03, RTTR = 0.02), considering lossy (RTR>0) and ordered (OFO = 0) transmission scenarios, based on the same first-tier city network environment, the end-to-end retransmission rate statistics of the 5G network at the N3 port are as follows:

[0213] RTR L =0.013,1.3%;

[0214] RTR R =0.009,0.9%;

[0215] When the RTR on the corresponding path is equal to the network empirical average value (RTTL = 0.013, RTTR = 0.009), the transmission quality factor Q' is equal to 90% of the value of the quality factor Q when the same delay is lossless and in-order transmission, and the latter Q = 0.9: Therefore, we get:

[0216]

[0217] Will:

[0218] RTR L =0.013;

[0219] RTR R =0.009;

[0220] Substituting into the above formula, we get:

[0221] α L ≈0.53;

[0222] α R ≈0.49;

[0223] Thus we get:

[0224]

[0225] 3) For the same end-to-end RTT delay (RTT = 0.05), considering the lossless (RTR = 0) and out-of-order (OFO>0) transmission scenarios, based on the statistics of the end-to-end out-of-order rate at the N3 port of the existing 5G network, the left / right out-of-order rates are:

[0226] OFO L =0.003,0.3%;

[0227] OFO R =0.012,1.2%;

[0228] When OFO on the corresponding path is equal to the network empirical average (OFOL = 0.003, OFOR = 0.012), the transmission quality factor Q' is equal to 90% of the quality factor Q value when the same delay is lossless and in-order transmission, and the latter Q = 0.9: Therefore, we get:

[0229]

[0230] Will:

[0231] OFO L =0.003;

[0232] OFO R =0.012;

[0233] Substituting into the above formula, we get:

[0234] β L ≈0.38;

[0235] β R ≈0.5;

[0236] Thus we get:

[0237]

[0238] It can be understood that since the quality transfer factors on the left and right sides are divided into two formulas, after calculating the values ​​of the adjustment factors in the two formulas, the new user experience scoring model obtained can also be divided into a user experience scoring model for calculating the transmission quality factor on the left, and a user experience scoring model for calculating the transmission quality factor on the right.

[0239] Furthermore, the accuracy of the user experience scoring model after optimizing the adjustment factor can be verified by the following steps:

[0240] Pass the two groups of messages at the beginning of this section with parameters:

[0241] A(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.02s / 0.1% / 0.1%, 0.01s / 1% / 1%;

[0242] B(RTTL / RTRL / OFOL,RTTR / RTRR / OFOR): 0.2s / 1% / 1%, 0.1s / 1% / 1%;

[0243] Input the optimized user experience scoring model and get:

[0244] QA(QL / QR):0.8459 / 0.8868;

[0245] QB(QL / QR): 0.4470 / 0.5217;

[0246] At this time, the transmission quality factor value of combination B is about 40% lower than that of combination A, that is, the quality factor evaluated by the model is equivalent to the actual quality factor, so the user experience scoring model obtained can better reflect the actual differences in Internet perception.

[0247] In summary, based on big data statistical analysis, the transmission quality evaluation vector obtained after optimizing the adjustment factor is as follows:

[0248]

[0249] in:

[0250] RTR L ∈[0,1]

[0251] RTT L ∈[0,+∞]

[0252] OFO L ∈[0,1]

[0253] RTR R ∈[0,1],

[0254] RTT R ∈[0,+∞]

[0255] OFO R ∈[0,1]

[0256]

[0257] In this embodiment, the present invention can further correctly characterize the transmission quality of the message by introducing the variable Status and optimizing the model, and can also improve the accuracy of the calculated transmission quality factor in characterizing the actual transmission quality by optimizing the adjustment factor.

[0258] Based on the first embodiment and the second embodiment of the message quality analysis method of the present invention, a third embodiment of the message quality analysis method of the present invention is proposed.

[0259] In this embodiment, when it is detected that the number of the target service sessions is multiple, after the above step S2021, the method further includes:

[0260] Step S70, performing weighted averaging on the left transmission quality factors in the message transmission qualities to obtain an average left transmission quality factor, and performing weighted averaging on the right transmission quality factors in the message transmission qualities to obtain an average right transmission quality factor;

[0261] Step S80, weighting each of the service status identifiers in the message transmission quality, and taking the service status identifier with the largest number of weightings as the comprehensive service status identifier;

[0262] Step S90: combining the average left transmission quality factor, the average right transmission quality factor and the comprehensive service state identifier to obtain a mobile network quality monitoring result vector.

[0263] It should be noted that the mobile network quality monitoring result vector is a set of numerical values ​​or status codes used to quantify and describe the overall network performance and user experience experienced by users when using Internet services.

[0264] In this embodiment, for the numerical factors QL / QR in the evaluation vector of the transmission quality of each message in each target service session, the average left transmission quality factor (QLT) and the average right transmission quality factor (QRT) of the overall Internet access can be obtained by calculating the weighted average value; for the enumeration factor Status, the overall comprehensive service status identifier is obtained by counting the Status enumeration value with the most weighted times in the entire Internet access quality statistical time period (represented by Statusmost_common). Among them, for the weight factor used for weighting, it can be represented by the total number of uplink / downlink messages Np in a single Internet access document record with "service session flow" as the statistical unit, that is, the greater the traffic generated by a certain service session, the greater its impact on the overall service quality score within the statistical period.

[0265] In the preset time period, different moments are recorded as moments 1, 2, 3, ... n, and the message (session) evaluation vector at moment k is expressed as:

[0266]

[0267] At the same time, the total number of uplink / downlink messages Npk at time k is obtained, and the overall message evaluation vector (mobile network quality monitoring result vector) in the entire statistical period can be reflected by the message transmission quality in the preset time period. That is, the number of messages in the time period is used as the weight, and the transmission quality factors at different time points are weighted averaged to obtain each average value. And the mobile network quality monitoring result vector is obtained by the average value and the comprehensive service status identifier. That is, in a feasible implementation, the mobile network quality monitoring result vector can be calculated by the following formula:

[0268]

[0269] As an example, when there are two sessions on a mobile phone, chat and watch videos, assuming that the number of messages in the chat session is 1, and the number of messages in the watch videos session is 10, each message in each session of the mobile phone can be weighted by a preset weighting factor. Therefore, since the number of messages in the watch videos session is more than that in the chat session, the status in the watch videos session is Statusmost_common.

[0270] Please refer to Figure 7 In this embodiment, when measuring the Internet access quality, the present invention replaces the original macro-overall business with the analysis of micro-messages or sessions with the same IP quintuple as the smallest particles, and realizes independent quality evaluation at the level of single message (single business session) by quantitatively modeling the underlying message transmission quality. Then, the quality evaluation of macro-business behaviors (such as sending and receiving WeChat messages, watching short videos, etc.) is converted into the weighted average of multiple mobile network message transmission quality monitoring result vectors that have been obtained, thereby realizing the overall evaluation process, so that the mobile network quality monitoring result vector can be obtained effectively and quickly.

[0271] Based on this, combined with the transmission quality factor on the left side, the transmission quality factor on the right side and the service status identifier of the above single session, an evaluation model for calculating the mobile network quality monitoring result vector can be obtained as follows: Figure 8 As shown. Among them, Figure 8 The evaluation factor 1-1 is the left transmission quality factor of message 1, 1-2 is the right transmission quality factor of message 1, and 1-3 is the service status identifier of message 1.

[0272] Furthermore, when measuring Internet access quality, Fig. 9 The model shown is used for calculation. Fig. 9 QL1 in the figure refers to the left transmission quality factor of message 1, QR1 is the right transmission quality factor of message 1, and Status1 is the service status identifier of message 1. The identifiers corresponding to message 2 and message 3 are similar to the identifier corresponding to message 1, and will not be repeated here. After combining the left transmission quality factor, the right quality transfer factor, and the service status identifier, the mobile network message transmission quality monitoring result vector corresponding to messages 1 to 3 can be obtained. That is, Q1, Q2, and Q3 are obtained. Then, the mobile network quality monitoring result vector QT can be obtained based on Q1, Q2, and Q3.

[0273] In this embodiment, the present invention can quickly and accurately measure the macroscopic Internet access quality by converting macroscopic information into microscopic information, thereby avoiding the problem of different evaluation standards in traditional subjective evaluation.

[0274] Based on the first embodiment, the second embodiment, and the third embodiment of the message quality analysis method of the present invention, a fourth embodiment of the message quality analysis method of the present invention is proposed.

[0275] In this embodiment, after the above step S10, the method further includes:

[0276] Step S100, adding the round trip delays to obtain the end-to-end round trip delay;

[0277] Step S110, respectively determining a first difference between each of the retransmission rates and a preset threshold, and subtracting the product of the preset threshold and the first difference to obtain an end-to-end retransmission rate;

[0278] It can be understood that the first difference represents the probability that the data packet is successfully delivered to the receiver during the first transmission.

[0279] Step S120, respectively determining a second difference between each out-of-order rate and the preset threshold, and subtracting the product of the preset threshold and the second difference to obtain an end-to-end out-of-order rate;

[0280] It should be noted that the second difference reflects the probability that the data packet successfully reaches the receiver according to the original sending order.

[0281] Step S130: input the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain an end-to-end transmission quality factor.

[0282] It should be noted that the end-to-end transmission quality factor represents the transmission quality between two terminals that transmit messages.

[0283] As an example, the end-to-end round-trip delay, end-to-end retransmission rate, and end-to-end out-of-order rate can be calculated using the following formulas:

[0284]

[0285] Among them, the end-to-end round-trip delay is RTT E2E , the end-to-end retransmission rate is RTR E2E , the end-to-end out-of-order rate is OFO E2E , then RTT E2E ,RTR E2E OFO E2E By inputting the user experience scoring model, the end-to-end transmission quality factor can be obtained.

[0286] Among them, RTT E2E ,RTR E2E OFO E2E After inputting the user experience scoring model, the calculation formula of the end-to-end transmission quality factor can be:

[0287]

[0288] Among them, the end-to-end transmission quality is Q E2E . m E2E / α E2E / β E2E The value of can be calculated based on the actual network environment of the message quality to be measured.

[0289] Take the statistical data of a certain operator in a first-tier central city network environment as an example:

[0290] 1)RTTL / RTTR: 0.03s(30ms) / 0.02s(20ms)——》RTTE2E: 0.05s(50ms);

[0291] 2)RTRL / RTRR: 0.013(1.3%) / 0.009(0.9%)——》RTRE2E: 0.022(2.2%);

[0292] 3) OFOL / OFOR: 0.003 (0.3%) / 0.012 (1.2%)——》OFOE2E: 0.015 (1.5%);

[0293] Taking the above data as anchor point, we get:

[0294]

[0295] Therefore, the calculation formula of the end-to-end transmission quality factor can also be:

[0296]

[0297] Based on this, please refer to Fig.10 The end-to-end message quality assessment model shown in Figure 1 is Fig.10 In the example, the end-to-end message transmission quality of message 1 can be based on the above status1, RTT E2E 1. RTR E2E 1 and OFO E2E 1 means that the end-to-end message transmission quality of message 2 can be based on the above status2, RTT E2E 2. RTR E2E 2 and OFO E2E 2 indicates that the end-to-end message transmission quality of message 3 can be based on the above status3, RTT E2E 3. RTR E2E 3 and OFO E2E 3 means.

[0298] Optionally, in a feasible embodiment, after the above step S130, the method further includes:

[0299] Step S140, calculating the total number of messages, the actual end-to-end round-trip delay, the actual end-to-end retransmission rate and the actual end-to-end disorder rate corresponding to each collection time within the preset time period;

[0300] Step S150, taking each of the actual end-to-end round-trip delays as a new end-to-end round-trip delay, taking each of the actual end-to-end retransmission rates as a new end-to-end retransmission rate, and taking each of the actual end-to-end disorder rates as a new end-to-end disorder rate, and returning to execute the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain an end-to-end transmission quality factor, until the actual end-to-end transmission quality factor corresponding to each of the collection moments is obtained;

[0301] Step S160, multiplying the actual end-to-end transmission quality factor corresponding to each of the collection moments and the total number of messages to obtain the message quality of the entire period, and accumulating the total number of messages corresponding to each of the collection moments to obtain the total number of messages of the entire period;

[0302] Step S170: taking the ratio of the full-cycle message quality to the total number of full-cycle messages as the full-cycle end-to-end transmission quality factor.

[0303] It should be noted that the actual end-to-end round-trip delay, actual end-to-end retransmission rate and actual end-to-end disorder rate refer to the end-to-end round-trip delay, end-to-end retransmission rate and end-to-end disorder rate in actual scenarios. The total number of messages refers to the total number of uplink messages or the total number of downlink messages. The full-cycle end-to-end transmission quality factor represents the transmission quality of service data (various messages) between the two ends of the communication within a certain period of time.

[0304] Please refer to Fig.11 ,exist Fig.10 On this basis, after obtaining the end-to-end message transmission quality corresponding to each message, it is possible to perform temporal and spatial aggregation based on the end-to-end message transmission quality corresponding to each message to obtain the full-cycle end-to-end transmission quality factor.

[0305] As an example, the formula for calculating the full-cycle end-to-end transmission quality factor may be:

[0306]

[0307] in, is the end-to-end transmission quality of the entire cycle, K represents the acquisition time, n represents the end time of the preset time period, represents the end-to-end transmission quality at time k, N pkThe total number of messages.

[0308] In this embodiment, the present invention can calculate the full-cycle end-to-end quality factor of the time dimension superimposed on the space dimension based on the superposition result of the end-to-end quality factor in the time dimension, so as to achieve the effect of measuring the end-to-end Internet access quality within a preset time period and space dimension, thereby further improving the accuracy of the measured Internet access quality.

[0309] In addition, the present invention also provides a message quality analysis system.

[0310] Please refer to Fig.12 , the message quality analysis system comprises:

[0311] The indicator value calculation module 10 is used to obtain the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs according to the multiple message pairs in the target service session;

[0312] The transmission quality determination module 20 is used to determine the message transmission quality in the target service session based on each of the round-trip delays, each of the retransmission rates, and each of the out-of-order rates.

[0313] Optionally, the transmission quality determination module 20 is further configured to:

[0314] Inputting each of the round-trip delays, each of the retransmission rates, and each of the disorder rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, wherein the transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the disorder rate, and the degree of influence of the disorder rate on the transmission quality factor is less than the degree of influence of the round-trip delay and the retransmission rate on the transmission quality factor;

[0315] The message transmission quality in the target service session is obtained according to the transmission quality factor.

[0316] Optionally, the message acquisition module 10 is further used for:

[0317] At a target collection point in a target service session, an uplink message pair and a downlink message pair are obtained by optical splitting collection;

[0318] At a target collection point in a target service session, an uplink message pair and a downlink message pair are acquired through software-based monitoring of the data collection point.

[0319] Optionally, the message pair includes: an uplink message pair and a downlink message pair, the message transmission direction includes: left and right; the transmission quality factor includes: a left transmission quality factor and a right transmission quality factor;

[0320] The message quality analysis system also includes:

[0321] A first partitioning module is used to use the round-trip delay and retransmission rate corresponding to the downlink message pair and the disorder rate corresponding to the uplink message pair as a left transmission quality data set;

[0322] The second division module is used to use the round-trip delay and retransmission rate corresponding to the uplink message pair and the disorder rate corresponding to the downlink message pair as the right transmission quality data set;

[0323] Based on this, the transmission quality determination module 20 is further used to:

[0324] Inputting the left transmission quality data set into a preset user experience scoring model to obtain a left transmission quality factor output by the user experience scoring model;

[0325] The right-side transmission quality data set is input into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model.

[0326] Optionally, the message quality analysis system includes:

[0327] A first acquisition module, used to acquire a service status identifier of the target service session;

[0328] The transmission quality determination module 20 is further used for:

[0329] The message transmission quality in the target service session is obtained according to the transmission quality factor and the service state identifier.

[0330] Optionally, when it is detected that the number of the target service sessions is multiple, the message quality analysis system includes:

[0331] A weighting module, used to perform weighted averaging on the left transmission quality factors of each of the message transmission qualities to obtain an average left transmission quality factor, and perform weighted averaging on the right transmission quality factors of each of the message transmission qualities to obtain an average right transmission quality factor;

[0332] A state determination module, used for weighting each of the service state identifiers in each of the message transmission qualities, and taking the service state identifier with the largest number of weightings as the comprehensive service state identifier;

[0333] The Internet access quality assessment module is used to combine the average left transmission quality factor, the average right transmission quality factor and the comprehensive service status identifier to obtain a mobile network quality monitoring result vector.

[0334] Optionally, the message quality analysis system includes:

[0335] A superposition module, used for superimposing the round-trip delays to obtain an end-to-end round-trip delay;

[0336] A first calculation module, used for respectively determining a first difference between each of the retransmission rates and a preset threshold, and subtracting a product of the preset threshold and the first difference to obtain an end-to-end retransmission rate;

[0337] A second calculation module is used to respectively determine a second difference between each out-of-sequence rate and the preset threshold, and to obtain an end-to-end out-of-sequence rate by subtracting a product of the preset threshold and the second difference;

[0338] The end-to-end calculation module is used to input the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain an end-to-end transmission quality factor.

[0339] Optionally, the message quality analysis system includes:

[0340] The second acquisition module is used to calculate the total number of messages corresponding to each collection time within a preset time period, the actual end-to-end round-trip delay, the actual end-to-end retransmission rate and the actual end-to-end disorder rate;

[0341] a loop module, configured to use each of the actual end-to-end round-trip delays as a new end-to-end round-trip delay, each of the actual end-to-end retransmission rates as a new end-to-end retransmission rate, and each of the actual end-to-end disorder rates as a new end-to-end disorder rate, and return to execute the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain an end-to-end transmission quality factor, until the actual end-to-end transmission quality factor corresponding to each of the collection moments is obtained;

[0342] A data statistics module, used to multiply the actual end-to-end transmission quality factor corresponding to each of the collection moments and the total number of messages to obtain the message quality of the entire period, and to accumulate the total number of messages corresponding to each of the collection moments to obtain the total number of messages of the entire period;

[0343] The full-cycle quality calculation module is used to use the ratio of the full-cycle message quality to the total number of full-cycle messages as the full-cycle end-to-end transmission quality factor.

[0344] Optionally, the message quality analysis system further includes:

[0345] A model testing module is used to obtain a user experience scoring model to be tested and a transmission quality data set to be tested, and input the transmission quality data set to be tested into the user experience scoring model to be tested to obtain a transmission quality factor test result;

[0346] A model optimization module is used to optimize the adjustment factor in the user experience scoring model to be tested through a preset reference anchor point data set when it is detected that the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality data set to be tested is greater than a set threshold, and use the user experience scoring model to be tested after optimizing the adjustment factor as the user experience scoring model.

[0347] Among them, the functional implementation of each module in the above-mentioned message quality analysis system corresponds to each step in the above-mentioned message quality analysis method embodiment, and its functions and implementation processes are no longer repeated here one by one.

[0348] In addition, the present invention also proposes a message quality analysis device, which includes: a memory, a processor, and a message quality analysis program stored on the memory and executable on the processor, wherein the message quality analysis program implements the steps of the message quality analysis method of the present invention as described above when executed by the processor.

[0349] The specific embodiments of the message quality analysis device of the present invention are basically the same as the embodiments of the above-mentioned message quality analysis method, and will not be described in detail here.

[0350] In addition, the present invention also proposes a computer storage medium on which a message quality analysis program is stored. When the message quality analysis program is executed by a processor, the steps of the message quality analysis method of the present invention as described above are implemented.

[0351] The specific embodiments of the computer storage medium of the present invention are basically the same as the embodiments of the above-mentioned message quality analysis method, and will not be described in detail here.

[0352] In addition, the present invention also proposes a computer program product, on which a message quality analysis program is stored. When the message quality analysis program is executed by a processor, the steps of the message quality analysis method of the present invention as described above are implemented.

[0353] The specific embodiments of the computer program product of the present invention are basically the same as the embodiments of the above-mentioned message quality analysis method, and will not be described in detail here.

[0354] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0355] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0356] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a message quality analysis device (which can be a vehicle-mounted computer, a smart phone, a computer, or a server, etc.) to execute the methods described in each embodiment of the present application.

[0357] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A message quality analysis method, characterized in that: The message quality analysis method comprises: According to multiple message pairs in the target service session, the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs are obtained; The message transmission quality in the target service session is determined based on the round-trip delays, the retransmission rates, and the out-of-order rates.

2. The message quality analysis method according to claim 1, characterized in that: The step of determining the message transmission quality in the target service session based on the round-trip delays, the retransmission rates, and the out-of-order rates comprises: Inputting each of the round-trip delays, each of the retransmission rates, and each of the disorder rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, wherein the transmission quality factor is inversely correlated with the round-trip delay, the retransmission rate, and the disorder rate, and the degree of influence of the disorder rate on the transmission quality factor is less than the degree of influence of the round-trip delay and the retransmission rate on the transmission quality factor; The message transmission quality in the target service session is obtained according to the transmission quality factor.

3. The message quality analysis method according to claim 2, characterized in that: The message pair includes: an uplink message pair and a downlink message pair, and the transmission quality factor includes: a left transmission quality factor and a right transmission quality factor; Before the step of inputting each of the round trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, the method further includes: The round trip delay and retransmission rate corresponding to the downlink message pair and the disorder rate corresponding to the uplink message pair are used as the left transmission quality data set; The round trip delay and retransmission rate corresponding to the uplink message pair and the disorder rate corresponding to the downlink message pair are used as the right transmission quality data set; The step of inputting each of the round trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model includes: Inputting the left transmission quality data set into a preset user experience scoring model to obtain a left transmission quality factor output by the user experience scoring model; The right-side transmission quality data set is input into the user experience scoring model to obtain the right-side transmission quality factor output by the user experience scoring model.

4. The message quality analysis method according to claim 2, characterized in that: Before the step of obtaining the message transmission quality in the target service session according to the transmission quality factor, the method further includes: Obtaining a service status identifier of the target service session; The step of obtaining the message transmission quality in the target service session according to the transmission quality factor comprises: The message transmission quality in the target service session is obtained according to the transmission quality factor and the service state identifier.

5. The message quality analysis method according to claim 4, characterized in that: When it is detected that the number of the target service sessions is multiple, after the step of obtaining the message transmission quality in the target service session according to the transmission quality factor and the service state identifier, the method further includes: Performing a weighted average on the left transmission quality factors in the message transmission qualities to obtain an average left transmission quality factor, and performing a weighted average on the right transmission quality factors in the message transmission qualities to obtain an average right transmission quality factor; Weighting each of the service status identifiers in the message transmission quality, and taking the service status identifier with the largest weighting times as the comprehensive service status identifier; The average left transmission quality factor, the average right transmission quality factor and the comprehensive service state identifier are combined to obtain a mobile network quality monitoring result vector.

6. The message quality analysis method according to claim 2, characterized in that: After the step of obtaining the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs according to the plurality of message pairs in the target service session, the method further includes: Adding the round trip delays to obtain the end-to-end round trip delay; Determine a first difference between each of the retransmission rates and a preset threshold value, and subtract the product of the preset threshold value and the first difference to obtain an end-to-end retransmission rate; Determine a second difference between each out-of-sequence rate and the preset threshold value, and subtract the product of the preset threshold value and the second difference to obtain an end-to-end out-of-sequence rate; The end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate are input into the user experience scoring model to obtain an end-to-end transmission quality factor.

7. The message quality analysis method according to claim 6, characterized in that: After the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain the end-to-end transmission quality factor, the method further includes: Calculate the total number of messages, actual end-to-end round-trip delay, actual end-to-end retransmission rate and actual end-to-end disorder rate corresponding to each collection time within a preset time period; Taking each of the actual end-to-end round-trip delays as a new end-to-end round-trip delay, taking each of the actual end-to-end retransmission rates as a new end-to-end retransmission rate, and taking each of the actual end-to-end disorder rates as a new end-to-end disorder rate, and returning to the step of inputting the end-to-end round-trip delay, the end-to-end retransmission rate, and the end-to-end disorder rate into the user experience scoring model to obtain an end-to-end transmission quality factor, until the actual end-to-end transmission quality factor corresponding to each of the collection moments is obtained; The actual end-to-end transmission quality factor corresponding to each of the collection moments is multiplied by the total number of messages to obtain the message quality of the entire period, and the total number of messages corresponding to each of the collection moments is accumulated to obtain the total number of messages of the entire period; The ratio of the full-cycle message quality to the total number of full-cycle messages is used as the full-cycle end-to-end transmission quality factor.

8. The message quality analysis method according to claim 2, characterized in that: Before the step of inputting each of the round trip delays, each of the retransmission rates, and each of the out-of-order rates into a preset user experience scoring model to obtain a transmission quality factor output by the user experience scoring model, the method further includes: Acquire a user experience scoring model to be tested and a transmission quality data set to be tested, and input the transmission quality data set to be tested into the user experience scoring model to be tested to obtain a transmission quality factor test result; When it is detected that the difference between the transmission quality factor test result and the actual transmission quality factor of the transmission quality data set to be tested is greater than a set threshold, the adjustment factor in the user experience scoring model to be tested is optimized through a preset reference anchor point data set, and the user experience scoring model to be tested after the optimized adjustment factor is used as the user experience scoring model.

9. A message quality analysis system, characterized in that: The message quality analysis system comprises: An indicator value calculation module, used to obtain the round-trip delay, retransmission rate and disorder rate corresponding to each of the message pairs according to multiple message pairs in the target service session; The transmission quality determination module is used to determine the message transmission quality in the target service session based on each of the round-trip delays, each of the retransmission rates, and each of the disorder rates.

10. A message quality analysis device, characterized in that: The message quality analysis device comprises: a memory, a processor, and a message quality analysis program stored in the memory and executable on the processor, wherein the message quality analysis program implements the steps of the message quality analysis method according to any one of claims 1 to 8 when executed by the processor.

11. A computer storage medium, characterized in that: The computer storage medium stores a message quality analysis program, and when the message quality analysis program is executed by the processor, the steps of the message quality analysis method according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that The computer program product comprises a message quality analysis program, and when the message quality analysis program is executed by a processor, the steps of the message quality analysis method according to any one of claims 1 to 8 are implemented.

Images