Data processing method and apparatus, and electronic device

Abstract

The present disclosure relates to a data processing method and device and electronic equipment, and relates to the technical field of communication networks, wherein the method comprises: acquiring XDR data messages, historical rule information and rule information of a data processing process; determining comprehensive root cause evaluation data in a quality difference positioning process according to the XDR data messages and the rule information; and adaptively correcting the rule information according to the comprehensive root cause evaluation data and the historical rule information. In this way, the rule information of the data processing process is acquired, the rule information is intelligently optimized in combination with the historical rule information and the comprehensive root cause evaluation data, the rule information can be adaptively iteratively optimized, the requirements of system evolution and business feature change for quality of service analysis can be met, the accuracy of quality difference identification and positioning results and the intelligent closed loop of the entire system are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus and electronic device. Background Technology

[0002] For the identification and localization of poor mobile internet service quality, existing technical solutions usually pre-define specific processing procedures and parameters. However, with the continuous enrichment and development of mobile internet services, these pre-define processing procedures and parameters may be difficult to adapt to new business needs, resulting in low accuracy of the identification and localization results during the identification and localization process. Summary of the Invention

[0003] In view of this, this application provides a data processing method, apparatus and electronic device, the main purpose of which is to solve the technical problem of low accuracy of quality difference identification and positioning results in the process of quality difference identification and positioning.

[0004] According to a first aspect of this disclosure, a data processing method is provided, the method comprising:

[0005] Obtain XDR data packets, historical rule information, and rule information for the data processing process;

[0006] Based on the XDR data message and the rule information, the comprehensive root cause evaluation data in the quality defect localization process is determined;

[0007] Based on the comprehensive root cause evaluation data and the historical rule information, the rule information is adaptively corrected.

[0008] According to a second aspect of this disclosure, a data processing apparatus is provided, the apparatus comprising:

[0009] The acquisition module is used to acquire XDR data packets, historical rule information, and rule information for the data processing process;

[0010] The determination module is used to determine the comprehensive root cause evaluation data in the quality defect localization process based on the XDR data message and the rule information.

[0011] The optimization module is used to adaptively correct the rule information based on the comprehensive root cause evaluation data and the historical rule information.

[0012] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect described above.

[0013] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method of the first aspect described above.

[0014] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0015] Compared with existing technologies, the data processing method, apparatus, and electronic equipment disclosed herein acquire XDR data messages, historical rule information, and rule information of the data processing process; determine comprehensive root cause evaluation data in the quality defect localization process based on the XDR data messages and rule information; and adaptively correct the rule information based on the comprehensive root cause evaluation data and historical rule information. In this way, rule information of the data processing process is acquired, and combined with historical rule information and comprehensive root cause evaluation data, the rule information is intelligently optimized, enabling adaptive iterative optimization. This meets the business quality analysis requirements of continuously evolving systems and changing business characteristics, while improving the accuracy of quality defect identification and localization results and the intelligent closed loop of the entire system.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical applications in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of a data processing method provided in an embodiment of the present disclosure;

[0020] Figure 2 This is a schematic flowchart of a data processing method provided in an embodiment of the present disclosure;

[0021] Figure 3 This is a system structure block diagram provided in the embodiments of this disclosure;

[0022] Figure 4 This is a clustering diagram of perception metrics provided in the embodiments of this disclosure;

[0023] Figure 5 This is a schematic diagram of primary / backup cache switching provided in an embodiment of this disclosure;

[0024] Figure 6 This is a threshold optimization diagram for the delimitation index quality difference provided in the embodiments of this disclosure;

[0025] Figure 7 This is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of the present disclosure. Detailed Implementation

[0026] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments of this disclosure and the features described therein can be combined with each other.

[0027] The data processing methods, apparatus, and electronic devices of this disclosure are described below with reference to the accompanying drawings.

[0028] This disclosure provides a data processing method, apparatus, and electronic device. By acquiring rule information from the data processing process and combining it with historical rule information and comprehensive root cause evaluation data, the rule information is intelligently optimized, enabling it to adaptively and iteratively optimize the rule information. This meets the business quality analysis requirements of continuous system evolution and changing business characteristics, while improving the accuracy of quality defect identification and location results, as well as the intelligent closed loop of the entire system.

[0029] like Figure 1 As shown, embodiments of this disclosure provide a data processing method, which may include:

[0030] Step 101: Obtain XDR data packets, historical rule information, and rule information for the data processing process.

[0031] XDR data packets can be transmitted in a Structured Data Record (SDR) format. In network transmission protocols, XDR can be a method for encapsulating and transmitting structured data. Rule information can be rules that are adjusted and changed within a certain range, used to guide and control the identification and handling of poor service quality. Rule information may include preset perception scoring thresholds, baseline thresholds for perception indicators, challenge thresholds, negative impact degree, negative deviation degree, boundary indicators, preset negative impact degree thresholds, preset negative deviation thresholds, and preset boundary indicator thresholds, etc. Historical rule information can be the experience and rules accumulated during problem diagnosis and resolution over a past period.

[0032] Different XDR data packets can be distinguished by their header. The header may contain information identifying the packet type and format. The packet content can be structured XDR records, typically using the TLV (Type-Length-Value) transmission format. This supports encapsulating multiple XDR records into a single independent data packet, improving transmission efficiency, especially when multiple records need to be sent together. Within the same XDR data packet, all encapsulated XDR records must have the same service type. This ensures that the receiving end can perform unified processing logic based on the service type. Furthermore, the length of the encapsulated XDR records within the same packet must be consistent, simplifying the receiving end's processing logic and improving efficiency. The header typically includes information such as service type, record length, and number of records.

[0033] In the embodiments of this disclosure, the executing entity can be a data processing device or equipment. This application primarily abstracts the logic of the business quality defect identification and location processing process, extracts parameterizable rules (i.e., rule information), and combines rule status data (i.e., historical rule information) and root cause evaluation data to intelligently optimize the rules (i.e., rule information). This enables the rules to adaptively and iteratively optimize, unifies the management of optimal rules to form a rule base, and applies it to the defect identification and location processing process. This application can meet the business quality analysis requirements of continuously evolving systems and changing business characteristics, while improving the accuracy of defect identification and location results and the intelligent closed loop of the entire system.

[0034] Step 102: Determine the comprehensive root cause evaluation data in the quality defect localization process based on the XDR data message and rule information.

[0035] Among them, the comprehensive root cause evaluation data can be used to diagnose faults, anomalies and other problems in the network, find the root cause and evaluate it; the poor quality localization process can be used in network optimization to diagnose problems and analyze causes for poor network quality (such as in a cell).

[0036] In this embodiment of the disclosure, during the defect localization process, comprehensive root cause evaluation data can be determined based on XDR data packets and rule information. The XDR data packets and rule information can be used to collect data related to quality problems, which can then be used to determine the root causes of the quality problems (i.e., comprehensive root cause evaluation data). Comprehensive root cause evaluation data is a crucial part of the defect localization process, used to determine the causes of quality problems and how to resolve them.

[0037] Step 103: Based on the comprehensive root cause evaluation data and historical rule information, adaptively correct the rule information.

[0038] In specific application scenarios, such as network security, quality monitoring, and anomaly detection, comprehensive root cause evaluation data can provide information about the effectiveness and efficiency of rule information, such as whether the rules can accurately identify problems and effectively filter out valuable information. By analyzing comprehensive root cause evaluation data, problems with the rule information can be identified, such as thresholds being set too high or too low, or conditions being set too broadly or too narrowly. Historical rule information can provide the evolution of rule information, including adjustments made to rules over time and the effects of these adjustments. By analyzing historical rule information, adjustment trends can be discovered, and how these adjustments affect system performance.

[0039] By integrating root cause evaluation data and historical rule information, we can better understand the effectiveness and efficiency of rule information, thereby adaptively correcting the rule information to improve system performance and efficiency.

[0040] The embodiments of this disclosure can intelligently complete rule optimization by combining real-time data, root cause evaluation information and historical rule information, continuously adapt to changes in the network environment, improve rule accuracy, avoid the problem that the expert database or knowledge base cannot perceive environmental changes and update in real time, and avoid the bottleneck of human experience analysis conclusions.

[0041] In summary, compared with existing technologies, the data processing method provided in this disclosure acquires XDR data messages, historical rule information, and rule information of the data processing process; determines comprehensive root cause evaluation data in the quality defect localization process based on the XDR data messages and rule information; and adaptively corrects the rule information based on the comprehensive root cause evaluation data and historical rule information. In this way, rule information of the data processing process is acquired, and combined with historical rule information and comprehensive root cause evaluation data, the rule information is intelligently optimized, enabling adaptive iterative optimization. This meets the business quality analysis requirements of continuously evolving systems and changing business characteristics, while improving the accuracy of quality defect identification and localization results and the intelligent closed loop of the entire system.

[0042] Furthermore, as a refinement and extension of the above embodiments, and in order to fully illustrate the specific implementation process of the method disclosed herein, this disclosure provides the following... Figure 2 The specific method shown includes:

[0043] Step 201: Obtain XDR data packets, historical rule information, and rule information for the data processing process.

[0044] In the embodiments disclosed herein, each unit of this application involves reading a general database. Since the database is not a claim of this system, this part will not be described in detail in the description of the invention.

[0045] like Figure 3 As shown, the data acquisition unit can be an external system, generally referring to a DPI (Deep Packet Inspection) system, which can process mirrored or split optical data from the switch, is responsible for acquiring service quality signaling data and generating real call detail records (XDRs), and interacts with this system based on a common transmission interface, pushing XDR data to this system in real time.

[0046] The data preprocessing unit is responsible for real-time preprocessing of XDR data, including receiving XDR data packets pushed by external systems, splitting them into complete XDR records, performing field conversion, backfilling, and verification on each XDR record, generating memory records (i.e., XDR memory records) that the system can process efficiently, and sending them to the quality defect identification unit.

[0047] The poor quality identification unit is responsible for real-time XDR processing (which can be XDR memory recording), clustering business quality indicators based on business dimensions, calculating indicator scores through baseline and challenge thresholds of perceived indicators, and then comprehensively calculating perceived scores based on indicator weights. It identifies poor quality businesses through perceived score thresholds. The indicators and perceived score thresholds are the poor quality identification rules, which come from the rule management unit. It extracts XDR indicator distribution data and sends it to the rule status output unit, and sends the XDR of poor quality businesses to the poor quality location unit.

[0048] The quality defect localization unit can be used to delineate and locate quality defect problems. First, it will cluster the XDR (which can be recorded in XDR memory) in different domain dimensions, count the perceived score of each object in each dimension, and make horizontal comparisons in related domains. Then, it will screen the quality defect objects by negative impact degree and negative deviation degree. Finally, it will determine the root cause object by delineation index. The rule state data of the delineation and localization process is extracted and sent to the rule state acquisition unit.

[0049] The root cause output unit can be used to receive poor quality location root causes, count the frequency of root cause objects, fill in information such as the root cause processing period, and then send it to the external root cause processing unit.

[0050] The root cause handling unit can be an external system responsible for resolving the root causes of poor quality. After implementing the solution, it fills in the expected resolution time of the problem and feeds back the status information to the root cause evaluation unit.

[0051] The root cause evaluation unit is responsible for receiving feedback on root cause treatment, verifying the optimization effect, providing a comprehensive evaluation (i.e., comprehensive root cause evaluation data), and then sending it to the rule-based intelligent optimization unit.

[0052] The rule-based intelligent optimization unit is responsible for the intelligent optimization of rules for identifying and locating quality defects. Based on algorithms such as variance, correlation coefficient, multiple correlation system, GMM, least squares method, polynomial curve fitting, and incentive strategy, it analyzes rule state data and root cause evaluation data to find the optimal rule.

[0053] The rule management unit is responsible for managing the optimal rules for identifying and locating quality defects, and supports updating and querying the rules.

[0054] The rule status acquisition unit is responsible for loading rule status data from the database, performing transformations such as histogram probability distribution, and then sending it to the rule intelligent optimization unit.

[0055] Step 202: Perform concurrent processing on the XDR data packets to obtain XDR memory records. The XDR memory records are transmitted in the form of XDR messages through message queues.

[0056] In this embodiment of the disclosure, in order for the data preprocessing unit to support real-time processing of massive amounts of data, the system will start multiple threads on each host node to concurrently execute data preprocessing tasks, making full use of computing resources.

[0057] Specifically, the data preprocessing unit can perform concurrent processing on XDR data packets to obtain XDR memory records. The concurrent processing can involve parsing multiple received XDR data packets concurrently and converting each XDR record into a fast-processable memory format (i.e., an XDR memory record). Then, these XDR fields will undergo concurrent preprocessing, which may include default backfilling, content conversion, and anomaly verification.

[0058] To improve preprocessing efficiency, these field cleaning logics are dynamically compiled into directly executable binary dynamic libraries and provided with calling interfaces. Each business type of XDR record corresponds to a dynamic library. When an XDR record needs to be processed, this dynamic library interface is called, passing the original XDR record memory block and context state information. The dynamic library returns the XDR cleaning result as a memory block pointer, assembles the cleaned XDR records into XDR messages that can be quickly transmitted within the system, with each message supporting multiple XDR records. Finally, the XDR messages are placed in message queue Qu1.

[0059] Message queues are a first-in, first-out (FIFO) data structure that ensures data order and integrity. Placing XDR messages in a message queue allows subsequent data processing and analysis to proceed in a predetermined order, eliminating the need for reordering and improving efficiency. Furthermore, message queues can also cache and distribute data, further enhancing system processing power and flexibility.

[0060] Step 203: Read the XDR messages in the message pair column, and perform perception index clustering based on business dimensions on the XDR records in each XDR message until the preset time granularity is reached to obtain the perception index.

[0061] In this embodiment of the disclosure, the quality defect identification unit will retrieve information from message queue Q. u1 The system reads XDR (Extended Data Record) messages and performs business-dimensional perception metric clustering on the XDR records in each message. This clustering process continues until a preset time granularity is reached, ultimately yielding the business-dimensional perception metrics. The preset time granularity can be a pre-defined time interval, such as 5 minutes, 15 minutes, 30 minutes, 1 hour, or 1 day. Within this time granularity, the system continuously performs clustering analysis of the perception metrics to obtain more refined business insights.

[0062] One possible approach is to use a pre-defined business quality perception index model to cluster the XDR records in each XDR message based on perception indicators from different business dimensions, such as product, region, or customer type. The business quality perception index model can be a model composed of multiple perception indicators, which can be used to measure business quality, such as customer satisfaction, transaction success rate, and response time.

[0063] Step 204: Calculate the comprehensive score of the perception index based on the rule information, and filter out poor-quality services whose comprehensive score is less than the preset perception score threshold.

[0064] The rule information may include preset perception scoring thresholds, baseline thresholds and challenge thresholds for perception indicators, etc.; the comprehensive perception score may be a comprehensive score calculated based on the scores of multiple perception indicators and their corresponding weights, used to represent the overall quality status of the business.

[0065] In this embodiment of the disclosure, based on pre-defined rule information, a comprehensive score for each perception indicator can be calculated. Then, services with a comprehensive score below a preset perception score threshold are selected to identify poor-quality services. These services can then be improved and optimized to enhance the overall quality of the services. The preset perception score threshold can be a pre-defined perception score standard used to determine whether the quality of the services meets the requirements.

[0066] For embodiments of this disclosure, the comprehensive score of the perception index calculated based on the rule information may include the following specific process:

[0067] The perception index score is calculated based on the baseline threshold and challenge threshold of the perception index; the comprehensive score of the perception index is then calculated based on the perception weight corresponding to the perception index score. The baseline threshold can be a pre-set standard value for the perception index, used to determine whether the quality of the service has reached a basic, acceptable level. For example, only when the perception index score is higher than this threshold can the quality of the service be considered acceptable.

[0068] The challenge threshold can be a pre-set challenge value for a perceived indicator, typically higher than the baseline threshold, used to determine whether the quality of the business has reached an excellent level. For example, the quality of the business can only be considered excellent if the score of the perceived indicator is higher than this threshold.

[0069] like Figure 4 As shown, the perception index score is calculated based on the baseline threshold and challenge threshold of the perception index; the comprehensive score of the perception index is calculated according to the perception weight corresponding to the perception index score, and the specific formula is as follows:

[0070]

[0071] and

[0072] In the formula, QoE represents the overall score, W i This represents the perceived weight, and KQI represents the perceived index score.

[0073] The baseline thresholds and perception weights of the perceived indicators are derived from the rule management unit. Rules are used to automatically filter and identify a set of indicators reflecting users' perceptions of business quality and their corresponding threshold standards. These rules can be refined to specific business applications to meet the diverse requirements of different business behaviors and characteristics. In this system, perceived indicators are divided into three intervals based on pre-set baseline and challenge thresholds, and linear scoring is performed within these intervals. The baseline and challenge thresholds are two reference points used to determine whether business quality meets or exceeds expected standards. By default, the system uses a five-point scale for scoring, but it is not limited to five points; a percentage scale or other scoring systems can also be used. A scoring system facilitates the establishment of a unified and intuitive standard, and perceived indicators can be scored based on the following formula.

[0074]

[0075] In the formula, baseline represents the baseline threshold, challenge represents the challenge threshold, and direction represents the direction of the perception indicator. It can take the value 1 or -1. 1 represents a positive indicator (the larger the indicator value, the better), and -1 represents a negative indicator (the smaller the indicator value, the better).

[0076] For each business application, a comprehensive perception score (QoE) is calculated. Then, services with a QoE below a preset threshold (TQoE) are identified as having poor quality. The system defaults to TQoE of 4. These poor-quality services are encapsulated into messages and sent to message queue Qu4 for root cause analysis by the poor quality localization unit. Simultaneously, this unit forwards XDR messages to message queue Qu2.

[0077] The quality defect identification unit simultaneously performs perception indicator statistics for each business application based on the user identifier dimension. During this process, the system's default statistical granularity is 1 hour. Once the statistical granularity is reached, the statistical data is saved to the database. To obtain sufficient samples while conserving resources, users need to be sampled at a certain ratio before the statistics are compiled. This sampling ratio can be adjusted according to actual needs. After the statistics are completed, the results are filtered to remove users with insufficient business volume, ensuring the accuracy and effectiveness of the statistical results.

[0078] Upon startup, the poor quality identification unit reads the poor quality identification rules from the rule management unit, generates a rule cache within its unit, and sets this cache as the primary cache. Simultaneously, the poor quality identification unit periodically reads the poor quality identification rules from the rule management unit to generate a backup cache. After the backup cache initialization is complete, a switch between the primary and backup caches occurs: the original primary cache becomes the backup cache, and the backup cache becomes the primary cache, as follows: Figure 5 As shown. Since the primary / standby switchover is completed through atomic operations on flag bits, it does not affect the normal processing flow, nor is it necessary to add a mutual exclusion mechanism to the primary / standby cache read / write operations.

[0079] Specifically, this may include: reading rule information and generating a primary cache and a backup cache, wherein the primary cache is used to switch between the primary cache and the backup cache.

[0080] Step 205: Determine the comprehensive root cause evaluation data based on the XDR memory records of poor-quality services.

[0081] For embodiments of this disclosure, determining comprehensive root cause evaluation data based on XDR memory records of poor-quality services may specifically include:

[0082] The XDR memory records are delimited and located to obtain the root cause object and root cause data.

[0083] Based on the root cause data, determine whether the frequency of occurrence of the root cause object exceeds a preset threshold. If it exceeds the preset threshold, a root cause record is generated. Based on the evaluation scores and weights of different evaluation dimensions in the root cause record, determine the comprehensive root cause evaluation data.

[0084] The delimitation and localization process involves precisely analyzing and locating XDR memory records to determine the specific cause and location of the problem. The root cause object refers to the key factors that caused the problem, while the root cause data consists of data that provides a detailed description and analysis of these factors.

[0085] In specific application scenarios, the root cause output unit can periodically query root cause data from the database. Root cause data can be data that records the causes and circumstances that may lead to problems.

[0086] The system determines whether the frequency of recent root cause objects exceeds a preset threshold. This threshold can be configured as needed. If it does, a root cause record is generated, including the root cause conclusion and processing deadline. The system notifies registered external root cause processing units of this record, allowing them to promptly understand and address the issue. Simultaneously, the root cause record is saved to the system's database for later querying and analysis. Expired root cause objects are cleaned up. External root cause processing units register with the system's root cause output unit to receive and process root cause records issued by the system.

[0087] Root cause recording is a form of detailed documentation of the root cause objects and root cause data that led to the problem, which can be used for further analysis and problem solving.

[0088] In specific application scenarios, after processing the root cause problem, the external root cause processing unit will provide feedback to the root cause evaluation unit, including the implementation effective time and root cause evaluation information. Upon receiving the feedback, the root cause evaluation unit will update the root cause record information in the database.

[0089] The system periodically queries the database for root cause records whose implementation effective time has arrived. Then, the system queries the database for historical data related to these root cause records and data after the root cause resolution is implemented to verify the effectiveness of the problem resolution.

[0090] Root cause evaluation will be conducted from the following dimensions: scope of impact, difficulty of remediation, cost impact, customer satisfaction, and resolution effectiveness. Each evaluation dimension is worth 1 point, for a total of 5 points. If no feedback is provided on the root cause, the default score for each dimension is 0.6 points. This scoring system helps the system establish a unified and intuitive standard. The scoring system in this application includes, but is not limited to, a five-point system, and may also use a percentage system, etc.

[0091] Then, based on the weight of each dimension (the system defaults to a weight of 0.2 for each dimension), a comprehensive evaluation score is calculated. Finally, the root cause information and the comprehensive evaluation results are saved to the database.

[0092] The impact can be categorized as follows: Scope of impact: whether the quality issues of the substandard object will have a significant impact on the entire production process or the final product; Repair difficulty: whether the quality issues of the substandard object can be resolved through simple repair methods, and how difficult the repair is; Cost impact: whether the quality issues of the substandard object will affect costs, and to what extent; Customer satisfaction: whether the substandard object will affect customer satisfaction, and to what extent; Resolution effect: the recovery of indicators after the problem is resolved, which is automatically evaluated through the root cause assessment unit.

[0093] In this embodiment of the disclosure, delimiting and locating XDR memory records to obtain root cause objects and root cause data may specifically include:

[0094] Calculate the contribution of each perception indicator to the poor quality business; cluster the XDR memory records according to different dimension objects to generate statistical tables under preset dimensions and preset granularity; calculate the perception indicator score of each dimension object from the statistical tables based on the contribution value; filter the target poor quality objects under the same dimension object based on the negative impact degree and negative deviation degree of the perception indicator score, and determine the root cause object according to the boundary index of the target poor quality object; determine the root cause data based on the negative impact degree, negative deviation degree, and root cause object.

[0095] Among them, the negative impact degree can be the degree of negative impact of perceived indicators on business quality; the negative deviation degree can be the gap between actual performance and expectations or standards.

[0096] In this embodiment of the disclosure, target poor-quality objects under the same dimension are screened based on the negative impact degree and negative deviation degree of the perceived index score, and the root cause object is determined according to the delimitation index of the target poor-quality objects. Specifically, this may include:

[0097] The system filters out perception objects whose perception index scores do not reach the preset scores; it identifies perception objects whose negative impact exceeds the preset negative impact threshold as initial poor quality objects, and identifies initial poor quality objects whose negative deviation exceeds the preset negative deviation threshold as target poor quality objects; it identifies target poor quality objects that meet preset conditions as root cause objects, wherein the preset conditions include: target poor quality objects whose boundary indicators exceed the preset boundary indicator threshold, and target poor quality objects whose boundary indicators all exceed the preset boundary indicator threshold.

[0098] In specific application scenarios, the poor quality localization unit can read XDR messages from message queue Qu2, cluster the XDRs across various domain dimensions, and generate statistical tables with preset dimensions and granularities. These domains can include terminals, wireless cells, core network elements, and content sources. The statistical tables only calculate basic indicators; each composite indicator (KQI) is broken down into numerator and denominator basic indicators. The granularity of the statistical tables can be preset, and the statistical results need to be stored in a database.

[0099] Read the list of poor-quality services from message queue Qu4 and calculate the contribution of each KQI to the QoE quality difference. The calculation formula is as follows: Δc i = (5-KQI) i )*w i

[0100] In the formula, KQIi is the KQI score, when Δc i If the value is greater than 0, root cause localization is performed for that KQI.

[0101] KQI indicator data for the corresponding services are selected from the statistical tables of each domain, and root cause analysis is performed in the order of core network elements, content sources, wireless cells, and terminals, based on the principle of large-to-small pipeline. Horizontal comparisons are conducted within each domain. For content sources, data that has passed through poor-quality core network elements is removed before comparison, and objects with perception indicator scores that are not full (i.e., perception objects that have not reached the preset score) are selected.

[0102] For each object, calculate its negative impact score, and then filter out the objects with the greatest negative impact on the overall performance index using a negative impact score threshold. Next, calculate the negative deviation score for each object, and then filter out the objects with the largest negative deviation from the overall performance index using a negative deviation score threshold.

[0103] For each poor-quality object, iterate through its boundary indicators and filter out those whose boundary indicators exceed the poor-quality threshold. For a poor-quality core network, it is necessary to further determine that other similar business applications on this core network also have poor-quality boundary indicators. The poor-quality objects filtered out by the above judgment steps are the root cause objects.

[0104] The perception indicator baseline threshold, challenge threshold, negative impact threshold, negative deviation threshold, boundary indicator, and quality defect threshold are derived from the rule management unit, and the loading and caching mechanism is the same as that of the quality defect identification unit. The quality defect localization unit organizes information such as root cause object, business application, domain type, negative impact degree, and negative deviation degree into records and saves them to the database to form root cause data.

[0105] Step 206: Using the index weight optimization algorithm combined with comprehensive root cause evaluation and historical rule information, determine the weight of each perception index, use the k-means clustering method to determine the best optimization result among the perception index weights, and use the best optimization result to adaptively correct the rule information.

[0106] In this embodiment of the disclosure, the weights of each perception indicator are determined by combining an indicator weight optimization algorithm with comprehensive root cause evaluation and historical rule information. The optimal optimization result among the perception indicator weights is determined using the k-means clustering method, and the rule information is adaptively corrected using the optimal optimization result. Specifically, the implementation process is as follows: For each business application, an objective perception indicator system needs to be constructed, and different weights are assigned to each indicator to meet the requirements of comprehensive business quality evaluation. Based on the indicator value data, indicators are grouped according to business characteristics, and the weight of each group is assigned. Then, within each group, the Pearson correlation coefficient between indicators and the standardized variance of each indicator value are calculated to further determine the indicator weights within each group. The weight calculation formula is as follows:

[0107]

[0108] In the formula, Wi represents the weight of feature i, Xi represents the i-th feature in the dataset, Y represents the target variable, m represents the total number of features, var(Xi) represents the variance of the i-th feature, corr(Xi,Y) represents the correlation coefficient between the i-th feature and the target variable, |dot| represents the absolute value, and a is a constant between 0 and 1 used to adjust the relative importance of variance and correlation coefficient. In this formula, the first term measures the proportion of the variance of feature i in the entire dataset, and the second term measures the proportion of the correlation between feature i and the target variable in the entire dataset. The value of a determines the relative importance of variance and correlation coefficient in weight calculation. If a is close to 0, the variance has a greater impact on the weight; if a is close to 1, the correlation coefficient has a greater impact on the weight. The system default value of a is 0.5. Multiplying Wi by the group weights yields the final weight of each indicator, which needs to be verified. If the value is not equal to 1, then a minor adjustment to the weights is required. When the optimization cycle arrives, k-means clustering is used to find the index weight closest to the cluster center as the best optimization result, and the rules are updated through the rule management unit.

[0109] Step 207: Using the index threshold optimization method combined with comprehensive root cause evaluation and historical rule information, determine the threshold of each perception index, take the average value of the perception index threshold as the best optimization result, and use the best optimization result to adaptively correct the rule information.

[0110] In this embodiment of the disclosure, the threshold of each perceived indicator is determined by combining the comprehensive root cause evaluation and historical rule information using the indicator threshold optimization method. The average result of the perceived indicator threshold is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result. Here, the indicator is used to measure the quality of business in a certain aspect. It is necessary to select an appropriate threshold for each indicator to meet the customer evaluation requirements for business quality. This system divides the indicator threshold into benchmark threshold and challenge threshold.

[0111] Each indicator value can be divided into three ranges: poor quality, normal quality, and excellent quality. A Gaussian Mixture Model (GMM) can be used to cluster the indicators, and the baseline and challenge thresholds are determined based on the clustering results. Specifically, each indicator can be viewed as a mixture of Gaussian distributions, and GMM can be used to divide it into three Gaussian distribution clusters. Then, for the poor quality and excellent quality ranges, their means can be calculated, and these means can be used as the baseline and challenge thresholds, respectively.

[0112] When the optimization cycle arrives, the average value of each threshold learned at each granularity is taken as the best optimization result, and the rules are updated through the rule management unit.

[0113] Step 208: Filter the boundary indicators whose confidence levels meet the preset confidence levels. By analyzing the quality difference correlation between the perceived indicators and the boundary indicators, and combining the comprehensive root cause evaluation and historical rule information, determine the quality difference threshold of the boundary indicators. The quality difference threshold with the highest frequency is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result.

[0114] In this embodiment of the disclosure, firstly, delimitation indicators that meet the preset confidence level are selected. Specifically, in order to delimit the problem to different domains, appropriate pipeline-type delimitation indicators need to be selected for different business applications, different problem domains, and each different perception indicator. Delimitation indicators refer to indicators that can distinguish problem domains, generally referring to transport layer indicators, such as TCP downlink retransmission rate, TCP uplink retransmission rate, TCP uplink RTT average latency, TCP downlink RTT average latency, etc. In this system, we call them pipeline-type indicators.

[0115] Perception metrics generally lack delimitation capabilities and require correlation with pipeline-related metrics for delimitation. However, the degree of influence and correlation between pipeline-related metrics and perception metrics varies. Delimiting metrics based on fixed pipeline-related metrics cannot meet the requirements of different business scenarios. Some mechanisms of the pipeline transmission protocol itself can also avoid affecting perception metrics to a certain extent. For example, if the downlink retransmission rate of a certain business's pipeline TCP is very high, it indicates that packet loss is serious. However, the TCP protocol's fast retransmission mechanism can quickly detect packet loss and notify the sender to retransmit, thereby ensuring that perception is not affected. Therefore, it is necessary to determine this correlation based on big data analysis, as relying on experience is unreliable.

[0116] Boundary indicator optimization is achieved by finding the set of pipeline indicators that are most strongly correlated with the perceived indicators through multiple correlation. Based on the characteristics of business behavior, a static mapping relationship between the perceived indicators and the pipeline boundary indicators is first established. For each perceived indicator, one to three boundary indicators can be specified for a certain domain. Based on the mapping relationship and combined with the correlation coefficient analysis of big data, boundary indicators with high confidence are selected.

[0117] When calculating the multiple correlation coefficient, the number of boundary indicators should not exceed three. For example, if a perception indicator K0 can define pipeline-related indicators K1, K2, and K3 in a certain field, then the multiple correlation coefficients for (K0,K1), (K0,K2), (K0,K3), (K0,K1,K2), (K0,K1,K3), and (K0,K2,K3) should be calculated separately. If there is only one boundary indicator, the simple correlation coefficient can be calculated. For two indicators, taking (K0,K1,K2) as an example, the relationship between indicators K0 and K1,K2 is calculated as follows:

[0118]

[0119] In the formula, r12 is the correlation coefficient between K1 and K2, r10 is the correlation coefficient between K1 and K0, and r20 is the correlation coefficient between K2 and K0. r0.12 is the multiple correlation coefficient between K0 and K1 and K2. Finally, the set with the largest absolute value of the multiple correlation coefficients is selected to generate the bounding index.

[0120] By analyzing the quality difference correlation between perceived and boundary indicators, and combining comprehensive root cause evaluation and historical rule information, the quality difference threshold for boundary indicators is determined. The quality difference threshold with the highest frequency is taken as the optimal optimization result, and the rule information is adaptively corrected using the optimal optimization result. Specifically, by analyzing the quality difference correlation between perceived and boundary indicators, selecting an appropriate quality difference threshold for boundary indicators can meet the judgment criteria for quality differences in the corresponding domain. The quality difference threshold for boundary indicators is determined by regression fitting of the perceived indicator and each boundary indicator to a logarithmic curve, and then obtaining the intersection point of the baseline threshold of the perceived indicator and the fitted curve as the quality difference threshold for the boundary indicator, as follows: Figure 6 As shown.

[0121] The logarithmic equation used is: y = a*log(x) + b, where a and b are the coefficients of the logarithmic equation.

[0122] First, calculate the coefficients of a and b using the least squares method based on the sample data. Then, use the calculated y value, which is the threshold value of the boundary index, based on x = the baseline threshold of the perception index.

[0123] When the optimization cycle arrives, the set of boundary indicators with the highest frequency is selected as the best optimization result, and the rules are updated through the rule management unit.

[0124] Step 209: Calculate the negative impact of each perceived object using the negative impact calculation formula, filter the root cause objects whose negative impact is greater than the preset negative impact threshold and their corresponding perceived index scores, update the negative impact threshold according to the root cause objects and their corresponding perceived index scores, take the current negative impact threshold that has reached the optimization cycle as the best optimization result, and use the best optimization result to adaptively correct the rule information.

[0125] In this embodiment of the disclosure, the negative impact degree of each perceived object is calculated using a negative impact degree calculation formula. Root cause objects with negative impact degrees greater than a preset negative impact degree threshold and their corresponding perceived index scores are selected. The negative impact degree threshold is updated based on the root cause objects and their corresponding perceived index scores. The current negative impact degree threshold reaching the optimization cycle is taken as the optimal optimization result, and the rule information is adaptively corrected using the optimal optimization result. Specifically:

[0126] To identify valuable root causes, appropriate negative impact levels need to be selected for different business applications, problem domains, and each perceived metric. Impact level refers to the degree to which an individual object affects the overall metric. The formula for calculating the impact level for each individual object is as follows:

[0127]

[0128] In the formula, Smax represents the maximum KQI score. This system uses a five-point scale, i.e., Smax = 5; Sentiment is the overall KQI score; Others(x) is the KQI score calculated for all remaining objects after removing individual object x from the whole. When individual object x is removed from the whole, if the score of Others(x) increases compared to the Sentiment score, it indicates that individual object x has a negative impact on the whole. Therefore, the calculated D(x) is positive, representing the degree of negative impact, with a value range of [0, 100]. The KQI scoring function is shown in the Scorei calculation formula of the quality difference identification unit.

[0129] The negative impact threshold is used to filter out low-quality objects with economic value. During system initialization, an initial negative impact threshold, called the initial threshold α0, is set. The current threshold for the root cause is α. If the system threshold has not been adjusted, then α = α0. Threshold value estimation is measured through comprehensive root cause evaluation and the proportion of root causes. The value estimation formula is as follows:

[0130]

[0131] In the formula, Ei is the comprehensive evaluation score of each root cause divided by the total score of 5, normalized to [0,1], M is the number of root causes in the negative impact degree optimization threshold interval, and N is the total number of root causes in the current threshold interval (M≦N). k is a weighting coefficient that balances the root cause value and the number of root causes. Through experiments, the default value of the k coefficient is 0.6.

[0132] The negative impact threshold optimization process is as follows: First, calculate VE0 for the current granularity; then, filter out root causes with a negative impact greater than α+Sα (Sα is the step size, with a default value of 0.5%) and calculate VE1. If VE1 is greater than VE0, put the timestamp of the current granularity into queue Q; otherwise, clear queue Q. Finally, determine the length of queue Q. If it is greater than T (the system default value is 168 for hourly granularity and 14 for daily granularity), where T is the threshold for the number of connected granularities, then update the negative impact threshold to α+Sα and clear queue Q.

[0133] When the optimization cycle is reached, the latest negative impact threshold is taken as the best optimization result, and the rules are updated through the rule management unit.

[0134] Step 210: Calculate the negative deviation of each perceived object using the deviation calculation formula, filter the root cause objects whose negative deviation is greater than the preset negative impact threshold and their corresponding perceived index scores, update the negative deviation threshold according to the root cause objects and their corresponding perceived index scores, take the current negative deviation threshold that has reached the optimization cycle as the best optimization result, and use the best optimization result to adaptively correct the rule information.

[0135] In this embodiment of the disclosure, the negative deviation degree of each perceived object is calculated using the deviation degree calculation formula. Root cause objects with negative deviation degrees greater than a preset negative impact degree threshold and their corresponding perceived index scores are selected. The negative deviation degree threshold is updated based on the root cause objects and their corresponding perceived index scores. The current negative deviation degree threshold reaching the optimization cycle is taken as the optimal optimization result, and the rule information is adaptively corrected using the optimal optimization result. Specifically, in order to select valuable root causes, appropriate negative deviation degrees need to be selected for different business applications, different problem domains, and each different perceived index. Deviation degree is the degree of deviation of an individual index compared to the overall index. The deviation degree calculation formula for each individual object is as follows:

[0136]

[0137] In the formula, Smax represents the full score of KQI. This system adopts a five-point system, i.e., Smax = 5; Sentiment is the overall score of KQI index; Sindividuality(x) is the KQI score of individual object x; if the Sindividuality(x) score is less than the Sentiment score, it indicates that the individual object index has deteriorated and produced a negative deviation. Therefore, the calculated D'(x) is a positive value, representing the degree of negative deviation, and the value range is [0,100].

[0138] The formula for estimating the value of the negative deviation threshold is the same as the formula for estimating the value of the negative impact (VE).

[0139] The negative deviation threshold optimization process is as follows: First, calculate VE0' for this granularity; then, filter out root causes with a negative impact greater than β+Sβ (Sβ is the step size, with a default value of 0.5%), and calculate their VE1'. If VE1' is greater than VE0', then put the timestamp of this granularity into queue Q; otherwise, clear queue Q. Finally, determine the length of queue Q. If it is greater than T, where T is the threshold for the number of connected granularities, then update the negative deviation threshold to α+Sα and clear queue Q.

[0140] When the optimization cycle arrives, the latest negative deviation threshold is taken as the best optimization result, and the rules are updated through the rule management unit.

[0141] The rule management unit categorizes and stores the results of rule-based intelligent optimization. The primary storage medium is a general-purpose database, with each rule category stored in a separate database table (collectively referred to as the rule table). The rule table is backed up periodically and supports rule table recovery operations. The rule management unit provides simple query and update interfaces to meet the read and write requirements of other units.

[0142] The rule status acquisition unit is responsible for collecting rule status data, which mainly includes statistical data on indicators at the user identifier level and statistical data on indicators in various domains. First, the statistical data on indicators at the user identifier level loaded from the database is converted into indicator histogram probability distribution data and then sent to the rule intelligent optimization unit for indicator weight and threshold intelligent optimization. Second, perceptual indicators and pipeline indicators are calculated from the statistical data of each domain in the database and then sent to the rule intelligent optimization unit for boundary indicator and quality difference threshold intelligent optimization. Finally, root cause information is loaded from the database for negative impact degree and negative deviation degree intelligent optimization.

[0143] Key points of this application may include:

[0144] (1) Abstract the processing logic of the business quality poor identification and positioning process, extract parameterizable rules, including business quality indicator benchmark threshold, challenge threshold, weight, negative impact degree of poor quality object, negative deviation degree threshold, boundary indicator and poor quality threshold, etc. Control the business processing process through rules, optimize and adjust the rules in a timely manner according to the actual environment and feedback, improve the efficiency and quality of the business process, and continuously improve the business processing process.

[0145] (2) Based on the rule status data of the business quality poor identification process, the histogram probability distribution is transformed. The rules for quality poor identification are optimized by algorithms such as variance, correlation coefficient, and GMM. Rules such as indicator weight and indicator threshold are applied, and a scoring system is used to comprehensively evaluate business perception, identify poor quality business and the contribution of KQI to QoE quality poorness.

[0146] (3) Based on the rule status data of the business quality defect positioning process, the rule optimization of defect positioning is carried out by algorithms such as multiple correlation coefficient, least squares method, polynomial curve fitting, and incentive strategy. This avoids the problem that the expert database or knowledge base cannot perceive environmental changes and update in real time, as well as the bottleneck of human experience analysis conclusions.

[0147] (4) Optimal rule unified management supports rule update, access and persistence. Each unit will periodically read the quality difference identification rule from the rule management unit and generate a backup cache. After the backup cache is initialized, the primary and backup caches will be switched. The original primary cache becomes the backup cache and the backup cache becomes the primary cache. Since the primary and backup switching is completed by atomic operation of the flag bit, it does not affect the normal processing flow and does not require adding a mutual exclusion mechanism to the primary and backup cache read and write.

[0148] (5) Extract feedback after the implementation of root cause treatment, automatically compare and verify the effect, form a comprehensive evaluation, and apply it to rule-based intelligent optimization. Through the feedback mechanism, the rules can be continuously improved without human intervention to achieve intelligent closed loop.

[0149] This application mainly abstracts the logic of the process for identifying and locating poor business quality, extracts parameterizable rules, and combines rule status data and root cause evaluation data to intelligently optimize the rules, enabling them to adaptively and iteratively optimize. The optimal rules are then uniformly managed to form a rule library, which is then applied to the process of identifying and locating poor business quality.

[0150] Compared to existing technical solutions, this application can meet the business quality analysis requirements of continuously evolving systems and changing business characteristics. It intelligently optimizes rules by combining real-time data, root cause evaluation information, and historical rule information, continuously adapting to changes in the network environment, improving rule accuracy, and avoiding the problem of expert databases or knowledge bases failing to perceive environmental changes and update in real time, as well as the bottleneck of human experience-based analysis conclusions. The root cause evaluation feedback mechanism continuously improves the accuracy of quality defect identification and location results, and the intelligent closed loop of the entire system. Compared to existing technical solutions with fixed rules or knowledge bases, this application's technical solution can intelligently adjust rules using a feedback mechanism to improve system accuracy.

[0151] In summary, compared with existing technologies, the data processing method provided in this disclosure acquires XDR data messages, historical rule information, and rule information of the data processing process; determines comprehensive root cause evaluation data in the quality defect localization process based on the XDR data messages and rule information; and adaptively corrects the rule information based on the comprehensive root cause evaluation data and historical rule information. In this way, rule information of the data processing process is acquired, and combined with historical rule information and comprehensive root cause evaluation data, the rule information is intelligently optimized, enabling adaptive iterative optimization. This meets the business quality analysis requirements of continuously evolving systems and changing business characteristics, while improving the accuracy of quality defect identification and localization results and the intelligent closed loop of the entire system.

[0152] Based on the above Figure 1 and Figure 2 To provide a specific implementation of the method shown, this embodiment offers a data processing device, such as... Figure 7 As shown, the device includes: an acquisition module 31, a determination module 32, and an optimization module 33;

[0153] The acquisition module 31 is used to acquire XDR data packets, historical rule information, and rule information of the data processing process;

[0154] The determination module 32 is used to determine the comprehensive root cause evaluation data in the quality defect localization process based on the XDR data message and the rule information;

[0155] The optimization module 33 is used to adaptively correct the rule information based on the comprehensive root cause evaluation data and the historical rule information.

[0156] In specific application scenarios, the rule information includes a preset perception scoring threshold and a determination module 32, which can be used to perform concurrent processing on the XDR data packets to obtain XDR memory records. The XDR memory records are transmitted in the form of XDR messages through message queues.

[0157] Read the XDR messages in the message pair column, perform perception index clustering based on business dimensions on the XDR records in each XDR message until a preset time granularity is reached to obtain perception indicators; calculate the comprehensive score of the perception indicators based on the rule information, and filter out poor-quality services whose comprehensive scores are less than a preset perception score threshold; determine the comprehensive root cause evaluation data based on the XDR memory records of the poor-quality services.

[0158] In specific application scenarios, the rule information includes the baseline threshold and challenge threshold of the perception indicator. The determination module 32 can be used to calculate the perception indicator score of the perception indicator based on the baseline threshold and challenge threshold of the perception indicator; and to calculate the comprehensive score of the perception indicator according to the perception weight corresponding to the perception indicator score.

[0159] In specific application scenarios, the determination module 32 can be used to perform boundary location processing on the XDR memory record to obtain root cause objects and root cause data; based on the root cause data, it is determined whether the occurrence frequency of the root cause object exceeds a preset threshold. If it exceeds the preset threshold, a root cause record is generated; based on the evaluation scores and weights of different evaluation dimensions in the root cause record, comprehensive root cause evaluation data is determined.

[0160] In specific application scenarios, the rule information includes negative impact degree, negative deviation degree, and boundary indicators. The determination module 32 can be used to calculate the contribution value of each perceived indicator to the poor quality business; cluster the XDR memory records according to different dimension objects to generate statistical tables under preset dimensions and preset granularities; calculate the perceived indicator score of each dimension object from the statistical tables based on the contribution value; filter target poor quality objects under the same dimension object based on the negative impact degree and negative deviation degree of the perceived indicator scores; determine the root cause object according to the boundary indicators of the target poor quality object; and determine the root cause data based on the negative impact degree, the negative deviation degree, and the root cause object.

[0161] In specific application scenarios, the rule information includes a preset negative impact threshold, a preset negative deviation threshold, and a preset boundary index threshold. The determination module 32 can be used to filter the perception objects whose perception index scores have not reached the preset scores.

[0162] The perceived object whose negative impact exceeds a preset negative impact threshold is identified as the initial poor quality object, and the initial poor quality object whose negative deviation exceeds a preset negative deviation threshold is identified as the target poor quality object.

[0163] The target poor quality object that meets the preset conditions is determined as the root cause object, wherein the preset conditions include: the target poor quality object whose boundary index exceeds the preset boundary index threshold, and the boundary indexes of the same type of objects of the target poor quality object all exceed the preset boundary index threshold.

[0164] In specific application scenarios, the optimization module 33 can be used to determine the weight of each perception indicator by combining the comprehensive root cause evaluation and the historical rule information with the indicator weight optimization algorithm, determine the best optimization result among the perception indicator weights by using the k-means clustering method, and adaptively correct the rule information by using the best optimization result.

[0165] By combining the comprehensive root cause evaluation and the historical rule information with the index threshold optimization method, the threshold of each perception index is determined. The mean result of the perception index threshold is taken as the optimal optimization result, and the rule information is adaptively corrected using the optimal optimization result.

[0166] The confidence level of the bounding indicators that meet the preset confidence level is selected. By sensing the correlation between the quality difference of the indicators and the bounding indicators, and combining the comprehensive root cause evaluation and the historical rule information, the quality difference threshold of the bounding indicators is determined. The quality difference threshold with the highest frequency is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result.

[0167] The negative impact degree of each perceived object is calculated using the negative impact degree calculation formula. Root cause objects whose negative impact degree is greater than a preset negative impact degree threshold and their corresponding perceived index scores are screened. The negative impact degree threshold is updated according to the root cause objects and their corresponding perceived index scores. The current negative impact degree threshold that has reached the optimization cycle is taken as the best optimization result. The rule information is adaptively corrected using the best optimization result.

[0168] The negative deviation degree of each perceived object is calculated using the deviation degree calculation formula. Root cause objects and their corresponding perceived index scores that have a negative deviation degree greater than a preset negative impact degree threshold are selected. The negative deviation degree threshold is updated based on the root cause objects and their corresponding perceived index scores. The current negative deviation degree threshold that reaches the optimization cycle is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result.

[0169] In specific application scenarios, the device also includes: a generation module 34;

[0170] The generation module 34 is used to read rule information and generate a main cache and a backup cache, wherein the main cache is used to switch between the main cache and the backup cache.

[0171] It should be noted that other corresponding descriptions of the functional units involved in the data processing apparatus provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding descriptions of the Chinese methods will not be repeated here.

[0172] Based on the above, Figure 1 and Figure 2 Accordingly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 and Figure 2 The method shown.

[0173] Based on this understanding, the technical solution of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various implementation scenarios of this disclosure.

[0174] Based on the above, Figure 1 and Figure 2 The method shown, and Figure 7 To achieve the above objectives, this disclosure also provides an electronic device, configurable on a vehicle (e.g., an electric vehicle), in accordance with the illustrated virtual device embodiment. The device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to implement the above-described virtual device. Figure 1 and Figure 2 The method shown.

[0175] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0176] Those skilled in the art will understand that the physical device structure provided in this disclosure does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0177] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that this disclosure can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented through hardware. Compared with the prior art, the data processing method, apparatus, and electronic equipment provided by this disclosure acquire XDR data messages, historical rule information, and rule information of the data processing process; determine comprehensive root cause evaluation data in the quality defect localization process based on the XDR data messages and rule information; and adaptively correct the rule information based on the comprehensive root cause evaluation data and historical rule information. In this way, rule information of the data processing process is acquired, and combined with historical rule information and comprehensive root cause evaluation data, the rule information is intelligently optimized, enabling adaptive iterative optimization. This meets the business quality analysis requirements of continuous system evolution and changing business characteristics, while improving the accuracy of quality defect identification and localization results and the intelligent closed loop of the entire system.

[0179] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0180] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data processing method, characterized in that, The method includes: Obtain XDR data packets, historical rule information, and rule information for the data processing process; Based on the XDR data message and the rule information, the comprehensive root cause evaluation data in the quality defect localization process is determined; The rule information is adaptively corrected based on the comprehensive root cause evaluation data and the historical rule information. The step of adaptively correcting the rule information based on the comprehensive root cause evaluation data and the historical rule information includes: The weights of each perceived indicator are determined by combining the comprehensive root cause evaluation and the historical rule information with the indicator weight optimization algorithm. The best optimization result among the perceived indicator weights is determined by the k-means clustering method, and the rule information is adaptively corrected by the best optimization result. By combining the comprehensive root cause evaluation and the historical rule information with the index threshold optimization method, the threshold of each perception index is determined. The mean result of the perception index threshold is taken as the optimal optimization result, and the rule information is adaptively corrected using the optimal optimization result. The confidence level of the bounding indicators that meet the preset confidence level is selected. By sensing the correlation between the quality difference of the indicators and the bounding indicators, and combining the comprehensive root cause evaluation and the historical rule information, the quality difference threshold of the bounding indicators is determined. The quality difference threshold with the highest frequency is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result. The negative impact degree of each perceived object is calculated using the negative impact degree calculation formula. Root cause objects whose negative impact degree is greater than a preset negative impact degree threshold and their corresponding perceived index scores are screened. The negative impact degree threshold is updated according to the root cause objects and their corresponding perceived index scores. The current negative impact degree threshold that has reached the optimization cycle is taken as the best optimization result. The rule information is adaptively corrected using the best optimization result. The negative deviation degree of each perceived object is calculated using the deviation degree calculation formula. Root cause objects and their corresponding perceived index scores that have a negative deviation degree greater than a preset negative impact degree threshold are selected. The negative deviation degree threshold is updated based on the root cause objects and their corresponding perceived index scores. The current negative deviation degree threshold that reaches the optimization cycle is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result.

2. The method according to claim 1, characterized in that, The rule information includes a preset perception scoring threshold. The step of determining the comprehensive root cause evaluation data during the quality defect localization process based on the XDR data message and the rule information includes: The XDR data packets are processed concurrently to obtain XDR memory records, which are then transmitted in the form of XDR messages through message queues. Read the XDR messages in the message pair column, and perform perception index clustering based on business dimensions on the XDR records in each XDR message until a preset time granularity is reached to obtain the perception index. Calculate the comprehensive score of the perception index based on the rule information, and filter out poor-quality services whose comprehensive score is less than the preset perception score threshold; The comprehensive root cause evaluation data is determined based on the XDR memory records of the poor-quality service.

3. The method according to claim 2, characterized in that, The rule information includes a baseline threshold and a challenge threshold for the perception indicator. The calculation of the comprehensive score for the perception indicator based on the rule information includes: The perception index score is calculated based on the baseline threshold and the challenge threshold of the perception index. The comprehensive score of the perception index is calculated based on the perception weight corresponding to the perception index score.

4. The method according to claim 3, characterized in that, The step of determining the comprehensive root cause evaluation data based on the XDR memory records of the poor-quality service includes: The XDR memory records are delimited and located to obtain root cause objects and root cause data; Based on the root cause data, determine whether the frequency of occurrence of the root cause object exceeds a preset threshold. If it exceeds the preset threshold, generate a root cause record. Based on the evaluation scores and weights of different evaluation dimensions in the root cause records, comprehensive root cause evaluation data is determined.

5. The method according to claim 4, characterized in that, The rule information includes negative impact degree, negative deviation degree, and delimitation index. The delimitation and location processing of the XDR memory records to obtain root cause objects and root cause data includes: Calculate the contribution of each of the aforementioned perception metrics to the poor-quality service; The XDR memory records are clustered according to different dimension objects to generate statistical tables under preset dimensions and preset granularities. Based on the contribution value, the perception index score of each dimension object is calculated from the statistical table; Based on the negative impact degree and negative deviation degree of the perceived index score, target poor quality objects under the same dimension are screened, and the root cause object is determined according to the delimitation index of the target poor quality object; The root cause data is determined based on the negative impact degree, the negative deviation degree, and the root cause object.

6. The method according to claim 5, characterized in that, The rule information includes a preset negative impact threshold, a preset negative deviation threshold, and a preset boundary index threshold. The process of filtering target poor-quality objects within the same dimension based on the negative impact and negative deviation scores of the perceived index, and determining the root cause object based on the boundary index of the target poor-quality object, includes: Filter out perception objects whose perception index scores do not reach the preset score; The perceived object whose negative impact exceeds a preset negative impact threshold is identified as the initial poor quality object, and the initial poor quality object whose negative deviation exceeds a preset negative deviation threshold is identified as the target poor quality object. The target poor quality object that meets the preset conditions is determined as the root cause object, wherein the preset conditions include: the target poor quality object whose boundary index exceeds the preset boundary index threshold, and the boundary indexes of the same type of objects of the target poor quality object all exceed the preset boundary index threshold.

7. The method according to claim 1, characterized in that, The method further includes: Read the rule information and generate a primary cache and a backup cache, wherein the primary cache is used to switch between the primary cache and the backup cache.

8. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire XDR data packets, historical rule information, and rule information for the data processing process; The determination module is used to determine the comprehensive root cause evaluation data in the quality defect localization process based on the XDR data message and the rule information. The optimization module is used to adaptively correct the rule information based on the comprehensive root cause evaluation data and the historical rule information; The step of adaptively correcting the rule information based on the comprehensive root cause evaluation data and the historical rule information includes: The weights of each perceived indicator are determined by combining the comprehensive root cause evaluation and the historical rule information with the indicator weight optimization algorithm. The best optimization result among the perceived indicator weights is determined by the k-means clustering method, and the rule information is adaptively corrected by the best optimization result. By combining the comprehensive root cause evaluation and the historical rule information with the index threshold optimization method, the threshold of each perception index is determined. The mean result of the perception index threshold is taken as the optimal optimization result, and the rule information is adaptively corrected using the optimal optimization result. The confidence level of the bounding indicators that meet the preset confidence level is selected. By sensing the correlation between the quality difference of the indicators and the bounding indicators, and combining the comprehensive root cause evaluation and the historical rule information, the quality difference threshold of the bounding indicators is determined. The quality difference threshold with the highest frequency is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result. The negative impact degree of each perceived object is calculated using the negative impact degree calculation formula. Root cause objects whose negative impact degree is greater than a preset negative impact degree threshold and their corresponding perceived index scores are screened. The negative impact degree threshold is updated according to the root cause objects and their corresponding perceived index scores. The current negative impact degree threshold that has reached the optimization cycle is taken as the best optimization result. The rule information is adaptively corrected using the best optimization result. The negative deviation degree of each perceived object is calculated using the deviation degree calculation formula. Root cause objects and their corresponding perceived index scores that have a negative deviation degree greater than a preset negative impact degree threshold are selected. The negative deviation degree threshold is updated based on the root cause objects and their corresponding perceived index scores. The current negative deviation degree threshold that reaches the optimization cycle is taken as the best optimization result, and the rule information is adaptively corrected using the best optimization result.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

Images