Method and apparatus for predicting mos value in a communication scenario
By constructing a MOS value prediction model based on alarm information of existing network elements and using an LSTM network for MOS value prediction, the problems of high cost and untimely operation of traditional methods are solved, and low-cost, real-time MOS value monitoring and full network coverage are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XINYANG BRANCH HENAN CO LTD OF CHINA MOBILE COMM CORP
- Filing Date
- 2024-09-05
- Publication Date
- 2026-06-19
AI Technical Summary
Traditional MOS value acquisition methods are costly and untimely, unable to achieve real-time monitoring, and difficult to fully cover all cells in the network.
By utilizing real-time and historical alarm information from existing network elements, an alarm information time series is constructed. The MOS value is then predicted using an LSTM network, reducing costs and enabling dynamic updates.
It enables low-cost, real-time MOS value monitoring and dynamic updates, covering all cells in the network, and improves the overall monitoring of network performance.
Smart Images
Figure CN119109824B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to a method and apparatus for predicting MOS values in a communication scenario. Background Technology
[0002] In network maintenance and optimization, the Mean Opinion Score (MOS) is widely used in network quality assessment as an important indicator for measuring voice or video quality. The MOS value converts voice or video quality into a numerical value from 1 to 5 through subjective testing, where 5 represents the best quality and 1 represents the worst quality. This indicator accurately reflects the user's perceived experience and is therefore crucial in network quality management.
[0003] Traditional MOS (Mean Offset of Memory) acquisition methods rely on carrying dedicated equipment to test the transmission and reception of voice or video over an actual network. This approach typically involves cell-by-cell, time-period testing, followed by data analysis using quality assessment software to obtain MOS values. While this method provides detailed network quality data, it suffers from high manpower and material costs, and in remote mountainous areas, data can only be collected once when the network is operational, with almost no updates afterward, making real-time monitoring of MOS value changes impossible. Comprehensive MOS value acquisition covering all cells in the network presents significant challenges, necessitating a low-cost, timely, and comprehensive solution to overcome the limitations of traditional methods. Summary of the Invention
[0004] This disclosure provides a method and apparatus for predicting MOS values in communication scenarios, so as to at least solve the problems of high cost and untimely response in related technologies.
[0005] The first aspect of this application proposes a method for predicting MOS values in a communication scenario, comprising: after detecting real-time alarm information corresponding to a network element in the existing network that is related to the mean opinion score (MOS) value, parsing the real-time alarm information to extract the target cell identifier carried by the real-time alarm information; obtaining the most recent N historical alarm information related to the target cell identifier, including the real-time alarm information, wherein all N historical alarm information are alarm information related to the MOS value; constructing an alarm information time series based on the most recent N historical alarm information; inputting the alarm information time series into a pre-trained MOS value prediction model, and obtaining the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model, wherein the MOS value prediction model is a voice MOS value prediction model or a video MOS value prediction model.
[0006] According to one embodiment of this application, obtaining the most recent N historical alarm information, including real-time alarm information, related to the target cell identifier includes: extracting historical alarm information related to the target cell identifier from the alarm information database; and tracing the historical alarm information backward in chronological order from the real-time alarm information to obtain the most recent N historical alarm information, including the real-time alarm information.
[0007] According to one embodiment of this application, constructing an alarm information time series based on the most recent N historical alarm information includes: extracting key fields from each of the most recent N historical alarm information to obtain the key field information corresponding to each of the most recent N historical alarm information; and constructing an alarm information time series based on the key field information corresponding to each of the most recent N historical alarm information.
[0008] According to one embodiment of this application, a training method for a MOS value prediction model includes: acquiring multiple sets of drive test data, wherein each set of drive test data includes one drive test cell identifier, one drive test collection time, and one actual drive test MOS value; for any set of drive test data, acquiring historical alarm information samples corresponding to the drive test cell identifier included in the set of drive test data, and tracing back the historical alarm information samples from the drive test collection time included in the set of drive test data to acquire N traced historical alarm information samples; for any set of drive test data, constructing training samples based on the set of drive test data and the N historical alarm information samples corresponding to the set of drive test data; acquiring the training samples corresponding to each of the multiple sets of drive test data to form a training sample set, and training the initial MOS value prediction model based on the training sample set until training is completed, and acquiring the trained MOS value prediction model.
[0009] According to one embodiment of this application, for any set of road test data, a training sample is constructed based on the set of road test data and N historical alarm information samples corresponding to the set of road test data, including: for any set of road test data, constructing an alarm information time series sample based on the N historical alarm information samples corresponding to the set of road test data, and using the actual values of the road test MOS included in the set of road test data as the sample labels of the alarm information time series samples; for any set of road test data, forming a training sample based on the alarm information time series samples and sample labels corresponding to the set of road test data.
[0010] According to one embodiment of this application, for any set of drive test data, training samples are constructed based on the set of drive test data and N historical alarm information samples corresponding to the set of drive test data, including: for any set of drive test data, constructing alarm information time series samples based on the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to the set of drive test data, and using the drive test cell identifier and actual drive test MOS value included in the set of drive test data as sample labels for the alarm information time series samples; for any set of drive test data, training samples are formed based on the alarm information time series samples and sample labels corresponding to the set of drive test data.
[0011] According to one embodiment of this application, the method for predicting MOS values in a communication scenario further includes: comparing the predicted MOS value corresponding to a target cell identifier with a preset threshold to obtain a first comparison result; and in response to the first comparison result indicating that the predicted MOS value corresponding to the target cell identifier is less than the preset threshold, sending a drive test command and / or a network maintenance command to the terminal device of the maintenance personnel corresponding to the target cell identifier.
[0012] According to one embodiment of this application, the method for predicting MOS values in a communication scenario further includes: in response to a first comparison result indicating that the predicted MOS value corresponding to a target cell identifier is less than a preset threshold, obtaining the model output time of the predicted MOS value corresponding to the target cell identifier; determining a target time period formed by tracing back a preset duration from the model output time; obtaining at least one target user number related to the target cell identifier within the target time period; and sending a preset text to the target user number.
[0013] According to one embodiment of this application, a method for predicting MOS values in a communication scenario further includes: obtaining the previous MOS prediction value corresponding to a target cell identifier; comparing the previous MOS prediction value corresponding to the target cell identifier with the MOS prediction value corresponding to the target cell identifier at the current time to obtain a second comparison result; in response to the second comparison result indicating that the values of the previous MOS prediction value corresponding to the target cell identifier and the MOS prediction value corresponding to the target cell identifier at the current time are different, obtaining the target color corresponding to the MOS prediction value corresponding to the target cell identifier at the current time; and updating the color of the target location corresponding to the target cell identifier on the geographic information system GIS map based on the target color.
[0014] A second aspect of this application proposes a device for predicting MOS values in a communication scenario, comprising: a monitoring module, configured to parse real-time alarm information to extract the target cell identifier carried by the real-time alarm information after detecting real-time alarm information corresponding to a network element related to the mean opinion score (MOS) value; an acquisition module, configured to acquire the most recent N historical alarm information related to the target cell identifier, including the real-time alarm information, wherein all N historical alarm information are alarm information related to the MOS value; a construction module, configured to construct an alarm information time series based on the most recent N historical alarm information; and a prediction module, configured to input the alarm information time series into a pre-trained MOS value prediction model to obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model, wherein the MOS value prediction model is a voice MOS value prediction model or a video MOS value prediction model.
[0015] According to one embodiment of this application, the acquisition module is further configured to: extract historical alarm information related to the target cell identifier from the alarm information database; trace back the historical alarm information in chronological order starting from the real-time alarm information, and acquire the most recent N historical alarm information, including the real-time alarm information.
[0016] According to one embodiment of this application, the construction module is further configured to: extract key fields from each of the most recent N historical alarm messages to obtain the key field information corresponding to each of the most recent N historical alarm messages; and construct an alarm information time series based on the key field information corresponding to each of the most recent N historical alarm messages.
[0017] According to one embodiment of this application, the MOS value prediction device in a communication scenario further includes a training module, configured to: acquire multiple sets of drive test data, wherein each set of drive test data includes one drive test cell identifier, one drive test acquisition time, and one actual drive test MOS value; for any set of drive test data, acquire historical alarm information samples corresponding to the drive test cell identifier included in the set of drive test data, and trace back the historical alarm information samples from the drive test acquisition time included in the set of drive test data to acquire N traced historical alarm information samples; for any set of drive test data, construct training samples based on the set of drive test data and the N historical alarm information samples corresponding to the set of drive test data; acquire the training samples corresponding to each of the multiple sets of drive test data to form a training sample set, and train the initial MOS value prediction model based on the training sample set until training is completed, and acquire the trained MOS value prediction model.
[0018] According to one embodiment of this application, the training module is further configured to: construct an alarm information time series sample based on N historical alarm information samples corresponding to any set of road test data, and use the actual values of the road test MOS included in the set of road test data as the sample labels of the alarm information time series sample; and form a training sample based on the alarm information time series sample and the sample labels corresponding to any set of road test data.
[0019] According to one embodiment of this application, the training module is further configured to: for any set of drive test data, construct an alarm information time series sample based on the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to the set of drive test data, and use the drive test cell identifier and actual drive test MOS value included in the set of drive test data as sample labels for the alarm information time series sample; and for any set of drive test data, form a training sample based on the alarm information time series sample and sample labels corresponding to the set of drive test data.
[0020] According to one embodiment of this application, the MOS value prediction device in a communication scenario further includes a post-processing module, configured to: compare the predicted MOS value corresponding to the target cell identifier with a preset threshold to obtain a first comparison result; and in response to the first comparison result indicating that the predicted MOS value corresponding to the target cell identifier is less than the preset threshold, send a drive test command and / or a network maintenance command to the terminal device of the maintenance personnel corresponding to the target cell identifier.
[0021] According to one embodiment of this application, the post-processing module is further configured to: in response to a first comparison result indicating that the MOS prediction value corresponding to the target cell identifier is less than a preset threshold, obtain the model output time of the MOS prediction value corresponding to the target cell identifier; determine a target time period formed by tracing back a preset duration from the model output time; obtain at least one target user number related to the target cell identifier within the target time period; and send a preset text to the target user number.
[0022] According to one embodiment of this application, the post-processing module is further configured to: obtain the previous MOS prediction value corresponding to the target cell identifier; compare the previous MOS prediction value corresponding to the target cell identifier with the MOS prediction value corresponding to the target cell identifier at the current time, and obtain a second comparison result; in response to the second comparison result indicating that the values of the previous MOS prediction value corresponding to the target cell identifier and the MOS prediction value corresponding to the target cell identifier at the current time are different, obtain the target color corresponding to the MOS prediction value corresponding to the target cell identifier at the current time; and update the color of the target location corresponding to the target cell identifier on the geographic information system GIS map based on the target color.
[0023] A third aspect of this application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to implement a method for predicting MOS values in a communication scenario as described in a first aspect of this application.
[0024] A fourth aspect of this application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to implement a method for predicting MOS values in a communication scenario as described in a first aspect of this application.
[0025] A fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements a method for predicting MOS values in a communication scenario as described in a first aspect of this application.
[0026] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:
[0027] This application uses existing alarm information in the network to predict MOS values, eliminating the need for on-site measurements and thus significantly reducing costs.
[0028] The prediction of MOS value in this application is based on real-time alarm information and is triggered by alarm information to achieve dynamic updating of MOS value. Compared with the traditional road test MOS collection method of quarterly or monthly, this method provides higher real-time performance.
[0029] Traditional drive tests typically only cover a limited area. This application uses alarm information for MOS prediction, which can cover all network cells and ensure overall network performance monitoring.
[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0032] Figure 1 This is a schematic diagram illustrating an exemplary implementation of a method for predicting MOS values in a communication scenario, as shown in one embodiment of this application.
[0033] Figure 2 This is a schematic diagram illustrating an embodiment of the present application of obtaining the most recent N historical alarm information, including real-time alarm information, related to the target cell identifier for MOS value prediction.
[0034] Figure 3 This is a schematic diagram illustrating an exemplary implementation of a method for predicting MOS values in a communication scenario, as shown in one embodiment of this application.
[0035] Figure 4 This is a schematic diagram illustrating an exemplary implementation of a method for predicting MOS values in a communication scenario, as shown in one embodiment of this application.
[0036] Figure 5 This is a schematic diagram of a device for predicting MOS values in a communication scenario, as shown in one embodiment of this application.
[0037] Figure 6 This is a schematic diagram of an electronic device according to one embodiment of this application. Detailed Implementation
[0038] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0039] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.
[0040] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.
[0041] The following is an introduction to some of the technical terms used in this application:
[0042] The Mean Opinion Score (MOS) is a commonly used subjective evaluation metric for assessing the quality of communication services. ( Qua li t y o fS erv i ce , Q o S ) 。 MOS Values are commonly used to measure the quality of voice and video communications. 。
[0043] The Voice MOS (Mean Offset Score) is a metric used to evaluate the quality of voice communication. It is typically based on the user's subjective evaluation of voice call quality, and generally ranges from 1 to 5. 1 represents very poor voice quality, while 5 represents excellent voice quality.
[0044] The MOS (Mean Offset of Quality) score is a metric used to evaluate the quality of video communication. It is also based on the user's subjective evaluation of video quality, typically ranging from 1 to 5. 1 represents very poor video quality, while 5 represents excellent video quality.
[0045] Figure 1 This is a schematic diagram illustrating an exemplary implementation of a method for predicting MOS values in a communication scenario as shown in this application. Figure 1 As shown, the method for predicting the MOS value in this communication scenario includes the following steps:
[0046] S101 After detecting real-time alarm information corresponding to existing network elements related to the average opinion score (MOS) value, the real-time alarm information is parsed to extract the target cell identifier carried by the real-time alarm information.
[0047] The operational status of existing network elements, such as mobile stations (MS), base stations (BS), transmission resource adapters (TRA), media gateways (MGW), and IP multimedia subsystems (IMS), plays a crucial role in the network. Their operational status directly affects the Quality of Service (QoS), particularly the Mean Opinion Score (MOS). The operational status of network elements can be reflected through network element alarm information. This application proposes to predict the MOS value of each cell based on the alarm information of existing network elements.
[0048] In this application, the entire network alarm status can be monitored in real time, and the original alarm information stream can be filtered to identify and filter only the key alarms. Subsequent alarms are traced back N historical alarm messages based on these key alarms. A key alarm is an alarm that causes the model to update the MOS value; it can also be understood as an alarm related to the MOS value. After detecting a real-time alarm message (a key alarm) corresponding to a network element related to the MOS value, the real-time alarm message is parsed to extract the target cell identifier carried by the real-time alarm message.
[0049] In the initial stage, this invention defines alarms of the Cell network element type as FocusAlarms. For alarms of other network element types, the model will ignore these alarms and not include them in the calculation and update of the Mean Opinion Score (MOS). This strategy aims to simplify the model's complexity and ensure that the model can run efficiently with limited computing resources.
[0050] As computing power gradually increases, the scope of focused alarms will be gradually expanded. This is because, theoretically, alarms from different network element types are correlated and may also cause changes in MOS values. For example, transmission interruptions may lead to cell failures, thus there is a causal relationship between transmission interruption alarms and cell alarms. By expanding the scope of focused alarms, the actual state of the network can be reflected more accurately, improving the accuracy of MOS value prediction.
[0051] From a network element perspective, in the initial stage, only cell alarms corresponding to specific MOS values are selected for model training. This strategy ensures that the model focuses on key factors directly affecting MOS values in the early stages. Later, the scope of network elements can be expanded, as those affecting MOS values are not limited to the Radio Access Network (RAN), but also include the Core Network, Transport Network, and Backhaul Network. In the future, alarms from all network elements and the entire network will be used to train the model to more comprehensively reflect the overall network condition.
[0052] From an alarm status perspective, the current state of network devices inevitably affects the MOS value. An alarm generation may cause the MOS value to decrease, while an alarm recovery may cause it to increase. Therefore, both active alarms and cleared alarms should be considered within the selected data range. By comprehensively considering both alarm generation and recovery states, the impact of network device status changes on the MOS value can be more accurately assessed.
[0053] S102, Obtain the most recent N historical alarm messages related to the target cell identifier, including real-time alarm information.
[0054] The Mean Opinion Score (MOS) is not affected by a single alarm message, but rather by a combination of alarm messages. These alarms are not only independent but also have distinct temporal characteristics. For example, some alarm messages have a temporal relationship, meaning that the occurrence of some alarms may trigger other alarms. Furthermore, over time, active alarms and cleared alarms together reflect changes in cell quality, and these changes all affect the cell's MOS.
[0055] In this application, after determining the target cell identifier as described above, the most recent N historical alarm information related to the target cell identifier, including real-time alarm information, are obtained for the next step of MOS value prediction. Optionally, N can be set to 100.
[0056] Among them, all N historical alarm messages are alarm messages related to MOS value, that is, they are all focus alarms.
[0057] S103, construct an alarm information time series based on the most recent N historical alarm information.
[0058] As one feasible approach, an alarm information time series can be constructed directly based on N historical alarm information.
[0059] Understandably, the original alarm information in the live network contains a large number of fields, with each alarm record typically containing 118 fields. Among these, some fields have overlapping meanings, leading to information redundancy, while others are unrelated to the operating status of network elements and therefore have no direct impact on changes in the MOS value.
[0060] Therefore, in order to reduce the complexity of subsequent models and improve processing efficiency, as another feasible approach, key fields are extracted from each of the most recent N historical alarm messages to obtain the key field information corresponding to each of the most recent N historical alarm messages, and an alarm information time series is constructed based on the key field information corresponding to each of the most recent N historical alarm messages.
[0061] For example, the key fields are set as message identifier, network element identifier, network element type, location object, location object type, event occurrence time, clearing time, first-level specialty, second-level specialty, standardized alarm category, clearing status, alarm flag, alarm event level, and equipment manufacturer, totaling 14 fields.
[0062] It is easy to understand that key fields can be set according to the actual situation. The 14 key fields exemplified in this application are merely examples and do not constitute a limitation on this application.
[0063] If a field in a historical alarm message is empty, then that field is set to the default value to indicate that the field in that historical alarm message is empty.
[0064] S104, input the alarm information time series into the trained MOS value prediction model, and obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model. The MOS value prediction model is either a voice MOS value prediction model or a video MOS value prediction model.
[0065] The MOS value prediction model in this application uses a Long Short-Term Memory (LSTM) network for data fitting. Compared to other types of neural networks, LSTM networks are specifically designed to solve long-term dependency problems and have strong memory capabilities. Through its unique gating mechanism, LSTM can effectively capture and maintain dependencies in long sequences, making it excellent at processing time series data.
[0066] For example, if the MOS value prediction model is a voice MOS value prediction model, then the alarm information time series is input into the trained voice MOS value prediction model to obtain the voice MOS prediction value corresponding to the target cell identifier output by the voice MOS value prediction model.
[0067] For example, if the MOS value prediction model is a video MOS value prediction model, then the alarm information time series is input into the trained video MOS value prediction model to obtain the video MOS prediction value corresponding to the target cell identifier output by the video MOS value prediction model.
[0068] For ease of understanding, Figure 2 This application illustrates a method for predicting MOS values by acquiring the most recent N historical alarm messages, including real-time alarm information, related to a target cell identifier. Figure 2 As shown, after detecting the latest real-time alarm information corresponding to the existing network element related to the MOS value, the most recent N alarms, including real-time alarm information, related to the target cell identifier carried by the element are obtained. Figure 2 (Taking N as 100 as an example) Historical alarm information is predicted using the input MOS value prediction model to obtain the MOS prediction value corresponding to the latest real-time alarm information.
[0069] This application proposes a method for predicting MOS values in a communication scenario, comprising: after detecting real-time alarm information corresponding to existing network elements related to the mean opinion score (MOS) value, parsing the real-time alarm information to extract the target cell identifier carried by the real-time alarm information; obtaining the most recent N historical alarm information related to the target cell identifier, including the real-time alarm information; constructing an alarm information time series based on the most recent N historical alarm information; inputting the alarm information time series into a pre-trained MOS value prediction model to obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model, wherein the MOS value prediction model is a voice MOS value prediction model or a video MOS value prediction model.
[0070] This application uses existing alarm information in the network to predict MOS values, eliminating the need for on-site measurements and thus significantly reducing costs.
[0071] The prediction of MOS value in this application is based on real-time alarm information and is triggered by alarm information to achieve dynamic updating of MOS value. Compared with the traditional road test MOS collection method of quarterly or monthly, this method provides higher real-time performance.
[0072] Traditional drive tests typically only cover a limited area. This application uses alarm information for MOS prediction, which can cover all network cells and ensure overall network performance monitoring.
[0073] In S102, obtaining the most recent N historical alarm information related to the target cell identifier, including real-time alarm information, includes: extracting historical alarm information related to the target cell identifier from the alarm information database; and tracing the historical alarm information backward in chronological order from the real-time alarm information to obtain the most recent N historical alarm information, including real-time alarm information.
[0074] Figure 3 This is a schematic diagram illustrating an exemplary implementation of a method for predicting MOS values in a communication scenario as shown in this application. Figure 3 As shown, the method for predicting the MOS value in this communication scenario includes the following steps:
[0075] S301 acquires multiple sets of drive test data, where each set of drive test data includes one drive test cell identifier, one drive test acquisition time, and one actual drive test MOS value.
[0076] Among them, the drive test cell identifier (CELL_ID) is a unique identifier used to represent a drive test cell.
[0077] The road test data collection time record specifies the exact time when the road test data was collected.
[0078] The actual MOS value in drive testing refers to the actual MOS value of the cell corresponding to the drive test cell identifier, collected during the drive test data collection period. It's important to note that if training a speech MOS prediction model, the actual speech MOS value is collected here; if training a video MOS prediction model, the actual video MOS value is collected here.
[0079] For example, a set of road test data can be recorded as: CELL_ID1, August 1, 2024, 09:00, Voice MOS value 1. This means that the voice MOS value collected at 09:00 on August 1, 2024, in the road test cell with cell identifier 1 is 1.
[0080] S302, for any set of drive test data, obtain the historical alarm information samples corresponding to the drive test cell identifiers included in the set of drive test data, and trace back the historical alarm information samples from the drive test collection time included in the set of drive test data to obtain N traced historical alarm information samples.
[0081] For example, if the drive test data is: CELL_ID1, August 1, 2024, 09:00, voice MOS value 1, then the historical alarm information corresponding to the drive test cell with cell ID 1 is obtained as a historical alarm information sample. Based on the collection timestamp of all historical alarm information samples corresponding to the drive test cell with cell ID 1, starting from August 1, 2024, 09:00, N historical alarm information samples are traced back.
[0082] S303: For any set of road test data, construct training samples based on the set of road test data and the N historical alarm information samples corresponding to the set of road test data.
[0083] The first method for constructing training samples:
[0084] For any set of road test data, construct an alarm information time series sample based on the N historical alarm information samples corresponding to the set of road test data (key fields need to be extracted), and use the actual values of the road test MOS included in the set of road test data as the sample labels of the alarm information time series sample.
[0085] For any set of road test data, after obtaining the alarm information time series samples and the sample labels of the alarm information time series samples corresponding to the set of road test data, training samples are formed based on the alarm information time series samples and sample labels corresponding to the set of road test data.
[0086] Thus, if there are 10,000 sets of road test data, then 10,000 training samples will be obtained.
[0087] The second method for constructing training samples:
[0088] For any set of drive test data, an alarm information time series sample is constructed by combining the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to the drive test data (key fields also need to be extracted). The drive test cell identifier and actual drive test MOS value included in the drive test data are used as the sample labels of the alarm information time series sample.
[0089] For any set of road test data, after obtaining the alarm information time series samples and the sample labels of the alarm information time series samples corresponding to the set of road test data, training samples are formed based on the alarm information time series samples and sample labels corresponding to the set of road test data.
[0090] Similarly, if there are 10,000 sets of road test data, then 10,000 training samples will be obtained.
[0091] Compared to the first training sample construction method, the second training sample construction method adds the drive test cell identifier and drive test collection time as training features, making the trained MOS value prediction model more accurate.
[0092] Taking the second training sample construction method as an example, the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to this set of drive test data together form a matrix data, which serves as the alarm information time series sample, and can be represented as follows:
[0093] CELL_ID1, Drive Test Acquisition Time 1, Historical Alarm Information 1 (14 key fields were extracted, including message identifier, network element identifier, network element type, location object, location object type, event occurrence time, clearing time, first-level specialty, second-level specialty, standardized alarm category, clearing status, alarm flag, alarm (event) level, and equipment manufacturer). ............
[0095] CELL_ID1, Drive Test Acquisition Time 1, Historical Alarm Information 2 (14 key fields were extracted, including message identifier, network element identifier, network element type, location object, location object type, event occurrence time, clearing time, first-level specialty, second-level specialty, standardized alarm category, clearing status, alarm flag, alarm (event) level, and equipment manufacturer).
[0096] CELL_ID1, drive test acquisition time 1, historical alarm information N (14 key fields extracted, including message identifier, network element identifier, network element type, location object, location object type, event occurrence time, clearing time, first-level specialty, second-level specialty, standardized alarm category, clearing status, alarm flag, alarm (event) level, and equipment manufacturer).
[0097] S304: Obtain training samples corresponding to multiple sets of road test data to form a training sample set, and train the initial MOS value prediction model based on the training sample set until the training is completed, and obtain the MOS value prediction model generated by the training.
[0098] If the training samples constructed using the first training sample construction method described above are used for training, the number of input features of the MOS value prediction model is 14 (14 key fields); the number of output features is 1 (MOS prediction value); and the length of the input sequence is N.
[0099] If the training samples constructed using the second training sample construction method described above are used for training, the number of input features for the MOS value prediction model is 16 (1 drive test cell identifier, 1 drive test acquisition time, and 14 key fields); the number of output features is 2 (drive test cell identifier and MOS prediction value); and the length of the input sequence is N.
[0100] Furthermore, the number of hidden layer units in the MOS value prediction model is set to 64, with each hidden layer containing 64 LSTM units.
[0101] Furthermore, the activation function for the MOS value prediction model is Softmax.
[0102] Furthermore, the loss function for the MOS value prediction model is categorical crossentropy. Categorical crossentropy is suitable for multi-class classification problems. After the model output passes through the Softmax activation function, the categorical cross-entropy can evaluate the difference between the predicted probability distribution and the true label distribution.
[0103] This application uses model.fit(X_train,y_train,epochs=1,batch_size=1,verbose=1)# to train the model. Here, each training epoch has a batch size of 1 (i.e., 1 sample is input each time) and outputs detailed information (verbose=1).
[0104] `model.fit` is a fitting function used to train neural network models. By training the model using training data, it fits model parameters that match the training data, enabling the model to predict or classify new input data. Through multiple iterations of training data, the `fit` function can automatically adjust the model parameters to minimize the loss function, thus obtaining a well-performing model.
[0105] S305, after detecting real-time alarm information corresponding to existing network elements related to the mean opinion score (MOS) value, parses the real-time alarm information to extract the target cell identifier carried by the real-time alarm information.
[0106] S306, retrieve the most recent N historical alarm messages related to the target cell identifier, including real-time alarm information.
[0107] S307, construct an alarm information time series based on the most recent N historical alarm information.
[0108] S308, input the alarm information time series into the trained MOS value prediction model, and obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model. The MOS value prediction model is either a voice MOS value prediction model or a video MOS value prediction model.
[0109] This application uses existing alarm information in the network to predict MOS values, eliminating the need for on-site measurements and significantly reducing costs. The MOS value prediction in this application is based on real-time alarm information and is triggered by alarm information, achieving dynamic updates of the MOS value. Compared to traditional drive test MOS collection methods that are performed quarterly or monthly, this method provides higher real-time performance. Traditional drive tests typically only cover limited areas; this application, using alarm information for MOS prediction, can cover all network cells, ensuring overall network performance monitoring.
[0110] Figure 4 This is a schematic diagram illustrating an exemplary implementation of a method for predicting MOS values in a communication scenario as shown in this application. Figure 4 As shown, the method for predicting the MOS value in this communication scenario includes the following steps:
[0111] The system monitors the entire network alarm status in real time, preprocesses the real-time alarm stream from the existing network, and filters out only alarms related to the MOS value. After detecting real-time alarm information corresponding to network elements related to the MOS value, the system parses the real-time alarm information to extract the target cell identifier carried in the real-time alarm information, and obtains the N most recent historical alarm information related to the target cell identifier, including the real-time alarm information (all N historical alarm information are alarms related to the MOS value).
[0112] Key fields are extracted from the most recent N historical alarm messages to construct an alarm message time series. The alarm message time series is then input into a pre-trained MOS value prediction model to obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model. The MOS value prediction model can be either a voice MOS value prediction model or a video MOS value prediction model.
[0113] Furthermore, such as Figure 4As shown, in terms of drive test auditing, after obtaining the MOS prediction value corresponding to the target cell identifier, the MOS prediction value corresponding to the target cell identifier is compared with a preset threshold (set to 2) to obtain a first comparison result; if the first comparison result indicates that the MOS prediction value corresponding to the target cell identifier is less than the preset threshold, then a drive test command and / or network maintenance command is sent to the terminal device of the maintenance personnel corresponding to the target cell identifier; similarly, it can also be understood that if the MOS prediction value corresponding to the target cell identifier is identified as 1, then a drive test command and / or network maintenance command is sent to the terminal device of the maintenance personnel corresponding to the target cell identifier.
[0114] Furthermore, if maintenance personnel conduct actual drive tests on the cell corresponding to the target cell identifier after receiving the drive test instruction, the system receives the actual MOS value corresponding to the target cell identifier collected by the maintenance personnel, and determines whether the actual MOS value corresponding to the target cell identifier is consistent with the predicted MOS value corresponding to the target cell identifier. If the actual MOS value corresponding to the target cell identifier is inconsistent with the predicted MOS value corresponding to the target cell identifier, new sample data is generated based on the alarm record time series and the actual MOS value to fit and update the MOS value prediction model.
[0115] Furthermore, such as Figure 4 As shown, in terms of customer care, in response to the first comparison result indicating that the MOS prediction value corresponding to the target cell identifier is less than a preset threshold, the model output time of the MOS prediction value corresponding to the target cell identifier is obtained; a target time period is determined by tracing back a preset duration from the model output time; at least one target user number related to the target cell identifier is obtained within the target time period; and a preset text is sent to the target user number.
[0116] For example, the specific implementation of obtaining at least one target user number associated with the target cell identifier within the target time period and sending a preset text to the target user number is as follows:
[0117] For 2G users: Query all 2G-MC interface XDRs within the target time period, filter by CELL_ID, and obtain the user number.
[0118] For 4G users: Query all 4G-S1MME interface XDRs within the target time period, filter by CELL_ID, and obtain the user number.
[0119] For 5G users: Query all 5G-N1N2 interface XDRs within the target time period, filter by CELL_ID, and obtain the user number.
[0120] The retrieved user numbers are deduplicated, and the SMS sending records of each deduplicated user number are retrieved to ensure that no care SMS messages have been sent within the past 24 hours. User numbers that have not sent care SMS messages within the past 24 hours are then selected as target user numbers, and a preset text message (preset care SMS message) is sent to these target user numbers. This method helps to proactively care for users who have recently used communities with poor quality of service, thereby reducing complaints.
[0121] Furthermore, such as Figure 4 As shown, in terms of map coloring in a Geographic Information System (GIS), the previous MOS prediction value corresponding to the target cell identifier is obtained; the previous MOS prediction value corresponding to the target cell identifier is compared with the MOS prediction value corresponding to the target cell identifier at the current time to obtain a second comparison result; in response to the second comparison result indicating that the values of the previous MOS prediction value corresponding to the target cell identifier and the MOS prediction value corresponding to the target cell identifier at the current time are different, the target color corresponding to the MOS prediction value corresponding to the target cell identifier at the current time is obtained; and the target location corresponding to the target cell identifier on the GIS map is updated with color based on the target color.
[0122] In this diagram, MOS=5 corresponds to dark green; MOS=4 corresponds to light green; MOS=3 corresponds to yellow; MOS=2 corresponds to orange; and MOS=1 corresponds to red.
[0123] Furthermore, this application supports displaying the CELL_ID and MOS values when the mouse hovers over the cell's coverage area on a GIS map.
[0124] Generally, monitoring personnel will focus on red areas. Red areas indicate poor network service quality; no red areas mean the network is operating well; many red areas mean the network is operating poorly.
[0125] Figure 5 This is a schematic diagram of a MOS value prediction device in a communication scenario shown in this application, as follows: Figure 5 As shown, the MOS value prediction device 500 in this communication scenario includes a monitoring module 501, an acquisition module 502, a construction module 503, and a prediction module 504, wherein:
[0126] The monitoring module 501 is used to parse the real-time alarm information after detecting the real-time alarm information corresponding to the existing network element related to the average opinion score (MOS) value, so as to extract the target cell identifier carried by the real-time alarm information.
[0127] The acquisition module 502 is used to acquire the most recent N historical alarm information related to the target cell identifier, including real-time alarm information. Among them, the N historical alarm information are all alarm information related to the MOS value.
[0128] Module 503 is used to construct a time series of alarm information based on the most recent N historical alarm information.
[0129] The prediction module 504 is used to input the alarm information time series into the trained MOS value prediction model and obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model. The MOS value prediction model is either a voice MOS value prediction model or a video MOS value prediction model.
[0130] This device uses existing alarm information in the network to predict MOS values, eliminating the need for on-site measurements and thus greatly reducing costs.
[0131] The prediction of the MOS value of this device is based on real-time alarm information and is triggered by alarm information to update, realizing dynamic updating of the MOS value. Compared with the traditional road test MOS acquisition method of quarterly or monthly, this method provides higher real-time performance.
[0132] Traditional drive tests typically only cover a limited area. This device uses alarm information to predict MOS (Mean Offset of Morphology), which can cover all network cells and ensure overall network performance monitoring.
[0133] Furthermore, the acquisition module 502 is also used to: extract historical alarm information related to the target cell identifier from the alarm information database; and trace back the historical alarm information in chronological order starting from the real-time alarm information to obtain the most recent N historical alarm information, including the real-time alarm information.
[0134] Furthermore, the construction module 503 is also used to: extract key fields from each of the most recent N historical alarm messages to obtain the key field information corresponding to each of the most recent N historical alarm messages; and construct an alarm information time series based on the key field information corresponding to each of the most recent N historical alarm messages.
[0135] Furthermore, the MOS value prediction device 500 in the communication scenario also includes a training module, used for: according to an embodiment of this application, acquiring multiple sets of drive test data, wherein each set of drive test data includes one drive test cell identifier, one drive test acquisition time, and one drive test MOS actual value; for any set of drive test data, acquiring historical alarm information samples corresponding to the drive test cell identifier included in the set of drive test data, and tracing back the historical alarm information samples from the drive test acquisition time included in the set of drive test data to acquire N traced historical alarm information samples; for any set of drive test data, constructing training samples based on the set of drive test data and the N historical alarm information samples corresponding to the set of drive test data; acquiring the training samples corresponding to each of the multiple sets of drive test data to form a training sample set, and training the initial MOS value prediction model based on the training sample set until training is completed, and acquiring the trained MOS value prediction model.
[0136] Furthermore, the training module is also used to: construct alarm information time series samples based on N historical alarm information samples corresponding to any set of road test data, and use the actual values of road test MOS included in the set of road test data as the sample labels of the alarm information time series samples; and to form training samples based on the alarm information time series samples and sample labels corresponding to any set of road test data.
[0137] Furthermore, the training module is also used to: construct an alarm information time series sample based on the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to the drive test data for any set of drive test data, and use the drive test cell identifier and actual drive test MOS value included in the drive test data as the sample label of the alarm information time series sample; and form a training sample based on the alarm information time series sample and sample label corresponding to the drive test data for any set of drive test data.
[0138] Furthermore, the MOS value prediction device 500 in the communication scenario also includes a post-processing module, used to: compare the predicted MOS value corresponding to the target cell identifier with a preset threshold to obtain a first comparison result; and in response to the first comparison result indicating that the predicted MOS value corresponding to the target cell identifier is less than the preset threshold, send a drive test command and / or a network maintenance command to the terminal device of the maintenance personnel corresponding to the target cell identifier.
[0139] Furthermore, the post-processing module is also used to: in response to the first comparison result indicating that the MOS prediction value corresponding to the target cell identifier is less than a preset threshold, obtain the model output time of the MOS prediction value corresponding to the target cell identifier; determine the target time period formed by tracing back a preset duration from the model output time; obtain at least one target user number related to the target cell identifier within the target time period; and send a preset text to the target user number.
[0140] Furthermore, the post-processing module is also used to: obtain the previous MOS prediction value corresponding to the target cell identifier; compare the previous MOS prediction value corresponding to the target cell identifier with the MOS prediction value corresponding to the target cell identifier at the current time, and obtain a second comparison result; in response to the second comparison result indicating that the values of the previous MOS prediction value corresponding to the target cell identifier and the MOS prediction value corresponding to the target cell identifier at the current time are different, obtain the target color corresponding to the MOS prediction value corresponding to the target cell identifier at the current time; and update the color of the target location corresponding to the target cell identifier on the geographic information system GIS map based on the target color.
[0141] To implement the above embodiments, this application also proposes an electronic device 600, such as... Figure 6 As shown, the electronic device 600 includes a processor 601 and a memory 602 communicatively connected to the processor. The memory 602 stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor 601 to implement the method for predicting MOS values in the communication scenario as shown in the above embodiment.
[0142] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to implement a method for predicting MOS values in the communication scenario shown in the above embodiments.
[0143] To implement the above embodiments, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements a method for predicting MOS values in the communication scenario shown in the above embodiments.
[0144] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0145] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0146] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0147] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.
[0148] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.
Claims
1. A method for predicting MOS value in a communication scenario, characterized in that, include: After detecting real-time alarm information corresponding to existing network elements that are related to the mean opinion score (MOS) value, the real-time alarm information is parsed to extract the target cell identifier carried by the real-time alarm information. Obtain the most recent N historical alarm messages related to the target cell identifier, including the real-time alarm information. All N historical alarm messages are alarm messages related to the MOS value, and all N historical alarm messages are focus alarms. Alarms with the network element type of cell are set as focus alarms. Construct an alarm information time series based on the N most recent historical alarm information; The alarm information time series is input into the trained MOS value prediction model to obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model, wherein the MOS value prediction model is a voice MOS value prediction model or a video MOS value prediction model. The training method for the MOS value prediction model includes: Acquire multiple sets of drive test data, wherein each set of drive test data includes one drive test cell identifier, one drive test acquisition time, and one actual drive test MOS value; For any set of drive test data, obtain historical alarm information samples corresponding to the drive test cell identifiers included in the set of drive test data, and trace back the historical alarm information samples from the drive test collection time included in the set of drive test data to obtain N traced historical alarm information samples. For any set of road test data, construct training samples based on the set of road test data and the N historical alarm information samples corresponding to the set of road test data; The training samples corresponding to each of the multiple sets of road test data are obtained to form a training sample set, and the initial MOS value prediction model is trained based on the training sample set until the training is completed, and the MOS value prediction model generated by the training is obtained. For any set of road test data, a training sample is constructed based on that set of road test data and N historical alarm information samples corresponding to that set of road test data, including: For any set of drive test data, an alarm information time series sample is constructed based on the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to the drive test data. The drive test cell identifier and actual drive test MOS value included in the drive test data are used as the sample label of the alarm information time series sample. For any set of road test data, training samples are formed based on the time series samples of the alarm information corresponding to that set of road test data and the sample labels.
2. The method of claim 1, wherein, The acquisition of the most recent N historical alarm information related to the target cell identifier, including the real-time alarm information, includes: Extract historical alarm information related to the target cell identifier from the alarm information database; Starting from the real-time alarm information, the historical alarm information is traced backward in chronological order to obtain the most recent N historical alarm information, including the real-time alarm information.
3. The method according to claim 2, characterized in that, The construction of the alarm information time series based on the most recent N historical alarm information includes: Extract key fields from each of the N most recent historical alarm messages to obtain the key field information corresponding to each of the N most recent historical alarm messages; The alarm information time series is constructed based on the key field information corresponding to each of the N most recent historical alarm information.
4. The method according to claim 1, characterized in that, The step of constructing training samples based on any set of road test data and N historical alarm information samples corresponding to that set of road test data also includes: For any set of drive test data, construct an alarm information time series sample based on N historical alarm information samples corresponding to the set of drive test data, and use the actual drive test MOS values included in the set of drive test data as the sample label of the alarm information time series sample; For any set of road test data, training samples are formed based on the time series samples of the alarm information corresponding to that set of road test data and the sample labels.
5. The method according to any one of claims 1-3, characterized in that, The method further includes: The predicted MOS value corresponding to the target cell identifier is compared with a preset threshold to obtain a first comparison result; In response to the first comparison result indicating that the MOS prediction value corresponding to the target cell identifier is less than the preset threshold, a drive test command and / or network maintenance command are sent to the terminal device of the maintenance personnel corresponding to the target cell identifier.
6. The method according to claim 5, characterized in that, The method further includes: In response to the first comparison result indicating that the MOS prediction value corresponding to the target cell identifier is less than the preset threshold, the model output time of the MOS prediction value corresponding to the target cell identifier is obtained; Determine the target time period formed by tracing back a preset duration from the time output by the model; Obtain at least one target user number associated with the target cell identifier within the target time period; Send a preset text to the target user number.
7. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain the previous MOS prediction value corresponding to the target cell identifier; The previous MOS prediction value corresponding to the target cell identifier is compared with the current MOS prediction value corresponding to the target cell identifier to obtain a second comparison result; In response to the second comparison result indicating that the previous MOS prediction value corresponding to the target cell identifier and the MOS prediction value corresponding to the target cell identifier at the current time are different, the target color corresponding to the MOS prediction value corresponding to the target cell identifier at the current time is obtained; The target location corresponding to the target cell identifier on the GIS map is updated with color based on the target color.
8. A device for predicting MOS values in a communication scenario, characterized in that, include: The monitoring module is used to parse the real-time alarm information to extract the target cell identifier carried by the real-time alarm information after detecting the real-time alarm information corresponding to the existing network element related to the mean opinion score (MOS) value. The acquisition module is used to acquire the most recent N historical alarm information related to the target cell identifier, including the real-time alarm information. Among them, the N historical alarm information are all alarm information related to the MOS value, and the N historical alarm information are all focus alarms. The alarms with the network element type of cell are set as focus alarms. The construction module is used to construct an alarm information time series based on the most recent N historical alarm information; The prediction module is used to input the alarm information time series into a pre-trained MOS value prediction model and obtain the MOS prediction value corresponding to the target cell identifier output by the MOS value prediction model, wherein the MOS value prediction model is a voice MOS value prediction model or a video MOS value prediction model. The training method for the MOS value prediction model includes: Acquire multiple sets of drive test data, wherein each set of drive test data includes one drive test cell identifier, one drive test acquisition time, and one actual drive test MOS value; For any set of drive test data, obtain historical alarm information samples corresponding to the drive test cell identifiers included in the set of drive test data, and trace back the historical alarm information samples from the drive test collection time included in the set of drive test data to obtain N traced historical alarm information samples. For any set of road test data, construct training samples based on the set of road test data and the N historical alarm information samples corresponding to the set of road test data; The training samples corresponding to each of the multiple sets of road test data are obtained to form a training sample set, and the initial MOS value prediction model is trained based on the training sample set until the training is completed, and the MOS value prediction model generated by the training is obtained. For any set of road test data, a training sample is constructed based on that set of road test data and N historical alarm information samples corresponding to that set of road test data, including: For any set of drive test data, an alarm information time series sample is constructed based on the drive test cell identifier, drive test collection time, and N historical alarm information samples corresponding to the drive test data. The drive test cell identifier and actual drive test MOS value included in the drive test data are used as the sample label of the alarm information time series sample. For any set of road test data, training samples are formed based on the time series samples of the alarm information corresponding to that set of road test data and the sample labels.
9. An electronic device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.
11. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-7.