Error set packet data management method and device, equipment and storage medium

Abstract

The invention relates to the field of logistics, and discloses a wrong set packet data management method, device and equipment and a storage medium, and the method is used for counting and analyzing wrong set packet data. The method comprises the following steps: acquiring wrong set packet data, and verifying the wrong set packet data to obtain verified wrong set packet data; sending the verified wrong set packet data to a message queue, and dividing the verified wrong set packet data into a plurality of segments of data to be analyzed according to a time sequence; counting the number of error set packets of different error categories of each segment of data to be analyzed and the sum of the number of the error set packets, and integrating the statistical results of the multiple segments of data to be analyzed to obtain counted error set packet data; on the basis of the statistical error set packet data, using a time sequence analysis algorithm to predict and obtain the error set packet prediction quantity of different error categories; and generating an error set packet analysis table in combination with the error set packet data after statistics and the error set packet prediction quantity.

Description

Technical Field

[0001] The present invention relates to the field of logistics technology, and in particular to a method, device, equipment and storage medium for managing staggered package data. Background Art

[0002] In the rapid development of the express delivery industry, accurate package collection and delivery are crucial. However, due to the continuous growth of business volume, the complexity of operating procedures and human factors, the problem of wrong package collection occurs from time to time. Specifically, wrong package collection may include the following situations:

[0003] Wrong destination: A package that was supposed to be sent to a certain destination is packed into a package sent to another destination due to incorrect classification.

[0004] Route Error: In the express delivery network, different routes correspond to different transportation paths and time limits. If a package is mistakenly classified into a collection package that does not match its actual transportation route, it may cause transportation delays.

[0005] Wrong size or type: Sometimes couriers will sort packages by size, weight or type to optimize shipping efficiency. If errors are made during this process, packages may be packed into inappropriate bundles.

[0006] Misassembled packages not only cause inconvenience to consumers (such as delays, loss, etc.), but also increase the operating costs of express delivery companies (such as re-sorting, compensation, etc.). Therefore, it is necessary to obtain misassembled package data, analyze and process them, and improve the situation of misassembled packages.

[0007] However, data collection on mismatched packages mostly relies on manual recording and regular report summaries. This method is not only time-consuming and labor-intensive, but also inefficient, and easily leads to data omissions or errors due to human factors, thus affecting subsequent analysis and decision-making. At the data processing level, traditional processing frameworks are unable to cope with the massive and rapidly growing express delivery data. They often lack sufficient flexibility and scalability, and are difficult to cope with high-concurrency, low-latency data processing requirements.

[0008] Therefore, the existing technology still needs to be improved and developed. Summary of the invention

[0009] The present invention provides a method, device, equipment and storage medium for managing mis-packet data, which are used for statistics and analysis of mis-packet data.

[0010] The first aspect of the present invention provides a method for managing error packet data, which includes: acquiring error packet data, verifying the error packet data, and obtaining verified error packet data; sending the verified error packet data to a message queue, and dividing the verified error packet data into multiple segments of data to be analyzed in chronological order; counting the number of error packets of different error categories and the total number of error packets in each segment of data to be analyzed, and integrating the statistical results of the multiple segments of data to be analyzed to obtain the statistical error packet data; based on the statistical error packet data, using a time series analysis algorithm to predict the predicted number of error packets of different error categories; and generating an error packet analysis table in combination with the statistical error packet data and the predicted number of error packets.

[0011] Optionally, in a first implementation method of the first aspect of the present invention, the step of obtaining error package data, verifying the error package data, obtaining verified error package data, and sending the verified error package data to a message queue includes: obtaining error package data from an error package database; performing numerical verification and format verification on the collected error package data to obtain verified error package data; and converting the verified error package data into JSON format.

[0012] Optionally, in a second implementation of the first aspect of the present invention, sending the verified error-collected data to a message queue, and dividing the verified error-collected data into multiple segments of data to be analyzed in chronological order, includes:

[0013] The verified error-collected package data is sent to a message queue, and each error-collected package data item in the verified error-collected package data is traversed to record the timestamp corresponding to each error-collected package data item; based on the timestamp corresponding to the error-collected package data item, all the error-collected package data items are sorted in ascending order; based on a sliding window of a preset length, the sorted error-collected package data items are segmented to obtain multiple segments of data to be analyzed.

[0014] Optionally, in a third implementation method of the first aspect of the present invention, the counting of the number of error packets of different error categories and the total number of error packets of each segment of data to be analyzed, and integrating the statistical results of the multiple segments of data to be analyzed to obtain the error packet data after statistics, includes: classifying the error packets of each segment of data to be analyzed to obtain a set of error packet data of multiple error categories corresponding to each segment of data to be analyzed; counting the number of error packets in each set of error packet data corresponding to each segment of data to be analyzed to obtain the number of error packets of different error categories of each segment of data to be analyzed, and counting the number of error packets in all sets of error packet data corresponding to each segment of data to be analyzed to obtain the total number of error packets corresponding to each segment of data to be analyzed; integrating the statistical results of multiple segments of data to be analyzed to obtain the error packet data after statistics.

[0015] Optionally, in a fourth implementation method of the first aspect of the present invention, the method uses a time series analysis algorithm to predict the predicted number of error packets of different error categories based on the statistical error packet data, including: extracting the number of error packets of different error categories of each segment of data to be analyzed one by one based on the statistical error packet data; putting the number of error packets of the same error type into the same set to form multiple time series data corresponding to the error categories; and using a time series analysis algorithm to predict the predicted number of error packets corresponding to the error category for each time series data.

[0016] Optionally, in a fifth implementation of the first aspect of the present invention, the use of a time series analysis algorithm to predict each time series data to obtain a predicted number of error packets corresponding to the error category includes: collecting historical error packet data, preprocessing the historical error packet data, and dividing the preprocessed historical error packet data into a training set and a test set; using the training set to train a pre-constructed time series analysis model, and using the test set to evaluate and tune the pre-constructed time series analysis model to obtain a trained time series analysis model; inputting each time series data into the time series analysis model one by one to obtain the predicted number of error packets corresponding to the error category.

[0017] Optionally, in a sixth implementation method of the first aspect of the present invention, the generation of an error packet analysis table by combining the statistical error packet data and the predicted number of error packets includes: calling a preset error packet analysis table template, and filling the statistical error packet data and the predicted number of error packets into the default parameters of the error packet analysis table template; extracting the number of error packets of different error categories of multiple segments of data to be analyzed from the statistical error packet data, and drawing an error packet trend chart based on the extracted data; inserting the error packet trend chart as a chart into the filled error packet analysis table template to generate the error packet analysis table.

[0018] The second aspect of the present invention provides a device for managing error packet data, including: a verification module, used to obtain error packet data, verify the error packet data, and obtain the verified error packet data; a division module, used to send the verified error packet data to a message queue, and divide the verified error packet data into multiple segments of data to be analyzed in chronological order; a statistical module, used to count the number of error packets of different error categories and the total number of error packets in each segment of data to be analyzed, and integrate the statistical results of the multiple segments of data to be analyzed to obtain the statistical error packet data; a prediction module, used to predict the predicted number of error packets of different error categories based on the statistical error packet data using a time series analysis algorithm; a generation module, used to generate an error packet analysis table in combination with the statistical error packet data and the predicted number of error packets.

[0019] Optionally, in a first implementation method of the second aspect of the present invention, the verification module includes: an acquisition unit, used to obtain error package data from an error package database; a verification unit, used to perform numerical verification and format verification on the collected error package data to obtain verified error package data; and a conversion unit, used to convert the verified error package data into JSON format.

[0020] Optionally, in a second implementation of the second aspect of the present invention, the partitioning module includes: a recording unit, used to send the verified error package data to a message queue, and traverse each error package data item in the verified error package data, and record the timestamp corresponding to each error package data item; a sorting unit, used to sort all error package data items in ascending order based on the timestamps corresponding to the error package data items; and a partitioning unit, used to segment the sorted error package data items based on a sliding window of a preset length to obtain multiple segments of data to be analyzed.

[0021] Optionally, in a third implementation of the second aspect of the present invention, the statistical module includes: a first classification unit, used to classify the error packets of each segment of data to be analyzed, and obtain error packet data sets of multiple error categories corresponding to each segment of data to be analyzed; a statistical unit, used to count the number of error packets in each error packet data set corresponding to each segment of data to be analyzed, so as to obtain the number of error packets of different error categories of each segment of data to be analyzed, and to count the number of error packets in all error packet data sets corresponding to each segment of data to be analyzed, so as to obtain the total number of error packets corresponding to each segment of data to be analyzed; an integration unit, used to integrate the statistical results of multiple segments of data to be analyzed, and obtain the statistical error packet data.

[0022] Optionally, in a fourth implementation method of the second aspect of the present invention, the prediction module includes: an extraction unit, used to extract the number of error packets of different error categories for each segment of data to be analyzed one by one based on the statistical error packet data; a second classification unit, used to put the number of error packets of the same error type into the same set to form a plurality of time series data corresponding to the error categories; a prediction unit, used to use a time series analysis algorithm to predict each time series data to obtain the predicted number of error packets corresponding to the error category.

[0023] Optionally, in a fifth implementation of the second aspect of the present invention, the generation module includes: a calling unit, used to call a preset error packet analysis table template, and fill the statistical error packet data and the predicted number of error packets into the default parameters of the error packet analysis table template; a drawing unit, used to extract the number of error packets of different error categories of multiple segments of data to be analyzed from the statistical error packet data, and draw an error packet trend chart based on the extracted data; a generation unit, used to insert the error packet trend chart as a chart into the filled error packet analysis table template to generate an error packet analysis table.

[0024] The third aspect of the present invention provides a misaligned packet data management device, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the computer-readable instructions in the memory so that the misaligned packet data management device executes each step of the misaligned packet data management method as described above.

[0025] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, which, when executed on a computer, enable the computer to execute the various steps of the above-mentioned method for managing mismatched packet data.

[0026] In the technical solution provided by the present invention, the error packet data after verification is divided into multiple sections of data to be analyzed in chronological order, the number of error packets of different error categories and the total number of error packets of each section of data to be analyzed are counted, and the statistical results of the multiple sections of data to be analyzed are integrated to obtain the error packet data after statistics, and based on the error packet data after statistics, a time series analysis algorithm is used to predict the predicted number of error packets of different error categories, and finally a error packet analysis table is generated by combining the error packet data after statistics and the predicted number of error packets. By statistically analyzing and predicting the error packet data using the time series analysis algorithm, not only can the current error status of the error packet be understood and the error trends and patterns in different time periods be identified, but also future error trends can be predicted. In addition, the generated error packet analysis table presents data in a visual manner, so that decision makers can quickly understand the problem and formulate corresponding solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A first flow chart of the mismatched packet data management method provided by an embodiment of the present invention;

[0028] Figure 2 A second flow chart of the mismatched packet data management method provided by an embodiment of the present invention;

[0029] Figure 3 A third flow chart of the mismatched packet data management method provided by an embodiment of the present invention;

[0030] Figure 4 A fourth flow chart of the mismatched packet data management method provided by an embodiment of the present invention;

[0031] Figure 5 A fifth flow chart of the mismatched packet data management method provided by the embodiment of the present invention;

[0032] Figure 6 A sixth flow chart of the mismatched packet data management method provided by an embodiment of the present invention;

[0033] Figure 7 A schematic diagram of the structure of a mismatched packet data management device provided by an embodiment of the present invention;

[0034] Figure 8 A schematic diagram of the structure of a mismatched packet data management device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The embodiment of the present invention provides a method, device, equipment and storage medium for managing error packet data, and the method is used to perform statistics and analysis on the error packet data. The method includes: obtaining the error packet data, verifying the error packet data, and obtaining the verified error packet data; sending the verified error packet data to a message queue, and dividing the verified error packet data into multiple segments of data to be analyzed in chronological order; counting the number of error packets of different error categories and the total number of error packets in each segment of data to be analyzed, and integrating the statistical results of multiple segments of data to be analyzed to obtain the statistical error packet data; based on the statistical error packet data, using a time series analysis algorithm to predict the predicted number of error packets of different error categories; combining the statistical error packet data and the predicted number of error packets to generate an error packet analysis table.

[0036] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 A first embodiment of a method for managing mismatched packet data in an embodiment of the present invention includes:

[0038] S101, obtaining wrong package data, verifying the wrong package data, and obtaining verified wrong package data.

[0039] It is understandable that the execution subject of the present invention may be a staggered packet data management device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0040] In this embodiment, when the courier collects and signs for the parcel, he inputs the relevant information of the wrong package through the application, and the application uploads the data to the wrong package database on the server side in real time for storage. Therefore, when obtaining the wrong package data, the wrong package data can be obtained by querying the wrong package database.

[0041] In this embodiment, a verification algorithm (such as CRC verification, Hamming verification, etc.) is used to verify the wrong packet data to ensure the integrity and accuracy of the data.

[0042] S102: sending the verified error-collected data to a message queue, and dividing the verified error-collected data into multiple segments of data to be analyzed in chronological order.

[0043] In this embodiment, it can be implemented through the client library of the message queue, such as the Producer API of Kafka, and the verified error package data is sent to the message queue.

[0044] In this embodiment, the consumer API of the message queue is used to receive and process data. The consumer can divide the data into multiple segments to be analyzed according to the timestamp. For example, the data can be divided into multiple segments to be analyzed according to a preset time interval (such as every day, every hour, etc.).

[0045] S103, counting the number of wrong packets of different error categories and the total number of wrong packets for each segment of data to be analyzed, and integrating the statistical results of multiple segments of data to be analyzed to obtain statistical wrong packet data.

[0046] In this embodiment, the error packets in each segment of data to be analyzed are classified to identify different error categories (such as destination error, line error, and type error, etc.), and then the number of error packets in each error category and the total number of error packets in each segment of data to be analyzed are counted.

[0047] S104. Based on the statistically correct error packet data, a time series analysis algorithm is used to predict the predicted number of error packets of different error categories.

[0048] In this embodiment, a time series analysis algorithm (such as ARIMA, LSTM, etc.) is used to analyze the statistically incorrect packet data to predict the number of incorrect packets of different error categories in the future.

[0049] S105. Generate a mismatched packet analysis table by combining the statistical mismatched packet data and the predicted number of mismatched packets.

[0050] In this embodiment, the statistically incorrect packet data and the predicted number of incorrect packets are integrated, and a preset incorrect packet analysis table template is used to generate an incorrect packet analysis table including key information such as error category, number of incorrect packets, predicted number, etc., and presented in a visual way (such as line graph, pie chart, etc.).

[0051] The present embodiment provides a method for managing error packet data, which divides the error packet data after verification into multiple segments of data to be analyzed in chronological order, counts the number of error packets of different error categories and the total number of error packets of each segment of data to be analyzed, and integrates the statistical results of multiple segments of data to be analyzed to obtain the error packet data after statistics, and based on the error packet data after statistics, uses a time series analysis algorithm to predict the predicted number of error packets of different error categories, and finally generates an error packet analysis table by combining the error packet data after statistics and the predicted number of error packets. By statistically analyzing and predicting the error packet data using the time series analysis algorithm, not only can the current error status of the error packet be understood and the error trends and patterns in different time periods be identified, but also future error trends can be predicted. In addition, the generated error packet analysis table presents data in a visual manner, so that decision makers can quickly understand the problem and formulate corresponding solutions.

[0052] See also Figure 2 , a second embodiment of the method for managing mismatched packet data in an embodiment of the present invention includes:

[0053] S201. Obtaining wrong package data from a wrong package database.

[0054] In this embodiment, when the courier collects and signs for the parcel, he inputs the relevant information of the mis-grouped package through the application program, and the application program uploads the data in real time to the mis-grouped package database on the server side for storage.

[0055] In this embodiment, a database connection tool or library (such as JDBC, SQLAlchemy, pymysql, etc.) is used to connect to the error collection package database, execute SQL query statements, and retrieve the required error collection package data from the database. The error collection package data includes information such as the error collection package ID, quantity, abnormal quantity, and collection location.

[0056] S202, performing value verification and format verification on the collected error-collected package data to obtain verified error-collected package data.

[0057] In this embodiment, the error collection package data includes multiple data items. When performing numerical verification on the collected error collection package data, each error collection package data item is traversed to check whether its "quantity" field is a non-negative integer and whether it is within a preset reasonable range. In this embodiment, the "collection package location" field of each error collection package data item is format-checked. A regular expression is used to ensure that the location string conforms to the expected format (such as containing a valid address, zip code, city name, etc.).

[0058] If any data inconsistency or format error is found during the verification process, the error information is recorded and the data is corrected. Finally, all the data items that have passed the verification are filtered out from the original data to form the error package data after verification.

[0059] S203, converting the verified error package data into JSON format.

[0060] In this embodiment, a JSON library (such as Python's json module, Java's org.json library, or JavaScript's JSON.stringify() method) is used to serialize the verified error packet data set into a string in JSON format.

[0061] In this embodiment, by performing numerical verification on the number of incorrect package collections and the number of abnormalities, the rationality and accuracy of the data can be ensured, and subsequent processing problems caused by data errors can be avoided. At the same time, the format verification of the package collection location ensures the standardization of the address or location, which is helpful for subsequent data processing and analysis.

[0062] See also Figure 3 A third embodiment of a method for managing mismatched packet data in an embodiment of the present invention includes:

[0063] S301: Send the verified error-collected package data to a message queue, traverse each error-collected package data item in the verified error-collected package data, and record a timestamp corresponding to each error-collected package data item.

[0064] Use the client library or API of the message queue to connect to the message queue service, use the verified error package data, that is, the serialized JSON string as the content of the message, call the message queue's send method, and send the message to the specified queue.

[0065] In this embodiment, the message queue refers to the Kafka message queue. Kafkaa has high throughput, scalability, and fault tolerance, and is very suitable for processing large amounts of data.

[0066] In this embodiment, Apache Flink can be used to read the error-checked packet data from the message queue. Apache Flink is an open source stream processing framework for processing unbounded and bounded data streams. It is known for its high throughput, low latency, and powerful state management capabilities, and is suitable for real-time data analysis and event-driven applications.

[0067] In this embodiment, when using Apache Flink to read the verified error-collected data from the message queue, it is necessary to create a Flink Kafka consumer and specify the Kafka topic to be read.

[0068] In this embodiment, when traversing each wrong package data item in the verified wrong package data and recording the timestamp corresponding to each wrong package data item, Flink's MapFunction or a similar mechanism can be used to parse each wrong package data item in the verified wrong package data and extract the timestamp field in each data item.

[0069] S302: Sort all the misaligned package data items in ascending order based on the timestamps corresponding to the misaligned package data items.

[0070] In this embodiment, a sorting algorithm (such as quick sort, merge sort, heap sort, etc.) is used to sort all the wrong set packet data items.

[0071] S303 , segmenting the sorted mismatched packet data items based on a sliding window of a preset length to obtain multiple segments of data to be analyzed.

[0072] In this embodiment, a sliding window can be defined using an API such as timeWindow or countWindow, and a sliding window of a preset length can be set. The sliding window allows the window to slide on the data stream, moving forward a certain time or amount each time, thereby performing segmentation.

[0073] In this embodiment, each error package data item in the error package data after traversal verification is recorded, and the timestamp corresponding to each error package data item is recorded. Based on the timestamp corresponding to the error package data item, all the error package data items are sorted in ascending order, and then the sorted error package data items are segmented based on a sliding window of a preset length, which is very flexible.

[0074] See also Figure 4 A fourth embodiment of a method for managing mismatched packet data in an embodiment of the present invention includes:

[0075] S401 , classifying the error packets of each segment of data to be analyzed, and obtaining a data set of error packets of multiple error categories corresponding to each segment of data to be analyzed.

[0076] In the present embodiment, each section of data to be analyzed includes a plurality of error packets. In each data section, classification statistics are performed according to the error category. That is, first, the error category of each error packet is identified, for example, the error type of each error packet is determined according to a predefined error code, error message or error feature. Then, the error packets belonging to the same error category are placed in the same error packet data set.

[0077] In this embodiment, the error categories include destination error, line error, and type error.

[0078] S402. Count the number of error packets in each error packet data set corresponding to each segment of data to be analyzed to obtain the number of error packets of different error categories for each segment of data to be analyzed, and count the number of error packets in all error packet data sets corresponding to each segment of data to be analyzed to obtain the total number of error packets corresponding to each segment of data to be analyzed.

[0079] In this embodiment, when each segment of data to be analyzed corresponds to multiple error packet data sets, the number of error packets in each error packet data set is counted to obtain the number of error packets of different error categories for each segment of data to be analyzed, and the number of error packets of different error categories is added together to obtain the total number of error packets corresponding to each segment of data to be analyzed.

[0080] In this embodiment, for each error packet in each data segment, a counter can be maintained for each error category, and whenever a error packet belonging to this category is encountered, the value of the counter is increased by 1. Finally, the number of each error category is added to obtain the total number of error packets.

[0081] S403, integrating the statistical results of multiple segments of data to be analyzed to obtain statistically correct error-collected data.

[0082] In this embodiment, a wrong set package database table is created, and the statistical result of each segment of data to be analyzed is added as an entry into the wrong set package database table.

[0083] In this embodiment, the integrated data may be indexed or sorted according to the timestamp.

[0084] In this embodiment, by classifying the mismatched packets according to the error categories, the distribution of various types of errors in each segment of data can be clearly understood, and the number of mismatched packets in each error category can be accurately counted, and the total number of mismatched packets in each segment of data can be calculated. This precise quantity statistics provides a solid foundation for data analysis, making subsequent analysis and decision-making more accurate and reliable.

[0085] See also Figure 5 A fifth embodiment of a method for managing mismatched packet data in an embodiment of the present invention includes:

[0086] S501 , based on the statistical error packet data, extract the number of error packets of different error categories of each segment of data to be analyzed one by one.

[0087] In this embodiment, illustratively, it is assumed that the verified error-collected data is divided into five sections of data to be analyzed in chronological order, namely, first data to be analyzed, second data to be analyzed, third data to be analyzed, fourth data to be analyzed, and fifth data to be analyzed, and the number of error-collected data of different error categories in each section of data to be analyzed is as follows:

[0088]

[0089] S502: Put the number of error packets of the same error type into the same set to form a plurality of time series data corresponding to the error categories.

[0090] The set corresponding to the destination error is {100,100,120,105,100}, the set corresponding to the line error is {80.80.100.75,80}, and the set corresponding to the type error is {50,50,55,55,50}.

[0091] In this embodiment, each time series data may be subjected to a stationarity check. If the check result shows that the data is not stationary, the time series data is subjected to differential processing, and the time series data after the differential processing is used for quantity prediction.

[0092] In this embodiment, the ADF (Augmented Dickey-Fuller) test can be used to check the stationarity of the extracted time series data. Analyze the test results to determine whether the data is stationary. If the p-value is less than the significance level (such as 0.05), the null hypothesis is rejected and the data is considered to be stationary. If the ADF test result shows that the data is not stationary, the data needs to be differentially processed until the data becomes stable, and the differential data is tested again with an ADF test to ensure data stationarity.

[0093] S503: For each time series data, use a time series analysis algorithm to predict the predicted number of error packets corresponding to the error category.

[0094] In this embodiment, a time series analysis algorithm is used to predict each time series data to obtain a predicted number of error packets corresponding to the error category, including: collecting historical error packet data, preprocessing the historical error packet data, and dividing the preprocessed historical error packet data into a training set and a test set; using the training set to train a pre-built time series analysis model, and using the test set to evaluate and tune the pre-built time series analysis model to obtain a trained time series analysis model; inputting each time series data into the time series analysis model one by one to obtain the predicted number of error packets corresponding to the error category.

[0095] In this embodiment, the pre-built time series analysis model may be an ARIMA, SARIMA, exponential smoothing, neural network or other time series analysis model.

[0096] In this embodiment, before dividing the historical wrong set package data into a training set and a test set, the historical wrong set package data is preprocessed, including processing abnormal values ​​and missing values.

[0097] In this embodiment, the collected historical mismatched packet data are arranged in chronological order, each time point corresponds to the number of mismatched packets, and the first 80% of the data are taken as a training set, and the last 20% of the data are taken as a test set.

[0098] In this embodiment, the sample data in the training set and the test set of the historical error set package may also be divided according to error categories, and then the pre-built time series analysis model may be trained and evaluated using the data of each error category one by one.

[0099] In this embodiment, the training process generally includes steps such as initializing model parameters, setting model hyperparameters, iteratively optimizing the model to reduce prediction errors, etc. After the model training is completed, the model needs to be evaluated using the test set data. The evaluation process generally includes calculating the prediction error of the model (such as mean square error, mean absolute error, etc.), drawing a comparison chart of the prediction results and the actual values, etc. Based on the evaluation results, the model can be tuned to improve its prediction performance.

[0100] In this embodiment, the number of error packets of the same error type is put into the same set to form multiple time series data corresponding to the error categories. A time series analysis algorithm is used to predict each time series data to obtain the predicted number of error packets corresponding to the error category. Independent prediction can be made for each error category, which helps to more accurately understand the changing trend of each error category and possible future situations.

[0101] See also Figure 6 A sixth embodiment of a method for managing mismatched packet data in an embodiment of the present invention includes:

[0102] S601. Call a preset mismatched packet analysis table template, and fill the statistical mismatched packet data and the predicted number of mismatched packets into the default parameters of the mismatched packet analysis table template.

[0103] In this embodiment, a mismatched packet analysis table template is pre-set.

[0104] S602: extract the number of wrong packet sets of different error categories of multiple segments of data to be analyzed from the statistical wrong packet set data, and draw a wrong packet set trend chart based on the extracted data.

[0105] In this embodiment, the number of error packets of different error categories of multiple segments of data to be analyzed is extracted. For the number of error packets of the same error category, the number of error packets of each segment of data to be analyzed is taken as a unit, and then a broken line is drawn in a line graph according to multiple values, and different error categories are represented by different broken lines.

[0106] S603: insert the wrong package trend chart as a chart into the filled wrong package analysis table template to generate the wrong package analysis table.

[0107] In this embodiment, the error package trend chart is inserted in the blank area of ​​the error package analysis table template. The insert function of the template or the export / copy function of the data visualization tool can be used to insert the trend chart as an image or chart object into the specified position of the template.

[0108] In this embodiment, by calling a preset wrong package analysis table template, combining the statistical wrong package data and the predicted number of wrong packages to generate a wrong package analysis table, automatically calling the template and filling in the data reduces the errors and time consumption of manual input, and improves the overall work efficiency. Moreover, the number of wrong packages of different error categories in multiple segments of data to be analyzed is extracted from the statistical wrong package data, and a wrong package trend chart is drawn according to the extracted data. The wrong package trend chart is inserted as a chart into the filled wrong package analysis table template, so that the changes of wrong packages of different error categories over time can be intuitively observed, making the data more intuitive and easy to understand.

[0109] The above describes the method for managing the mismatched packet data in the embodiment of the present invention. The following describes the device in the embodiment of the present invention. Figure 7 , the implementation method of the staggered packet data management device in the embodiment of the present invention includes:

[0110] The verification module 701 is used to obtain the wrong package data, verify the wrong package data, and obtain the verified wrong package data;

[0111] A division module 702 is used to send the verified error-collected data to a message queue, and divide the verified error-collected data into multiple segments of data to be analyzed in chronological order;

[0112] A statistical module 703 is used to count the number of wrong packets of different error categories and the total number of wrong packets for each segment of data to be analyzed, and integrate the statistical results of the multiple segments of data to be analyzed to obtain the wrong packet data after statistics;

[0113] A prediction module 704 is used to predict the number of wrong packets of different error categories based on the statistical wrong packet data using a time series analysis algorithm;

[0114] The generating module 705 is used to generate a misaligned packet analysis table by combining the statistical misaligned packet data and the predicted number of misaligned packets.

[0115] In this embodiment, the verification module 701 includes: an acquisition unit 7011, which is used to obtain the error package data from the error package database; a verification unit 7012, which is used to perform numerical verification and format verification on the collected error package data to obtain the verified error package data; a conversion unit 7013, which is used to convert the verified error package data into JSON format and send it to the message queue.

[0116] In this embodiment, the division module 702 includes: a recording unit 7021, which is used to send the verified error-collected package data to a message queue, and traverse each error-collected package data item in the verified error-collected package data to record the timestamp corresponding to each error-collected package data item; a sorting unit 7022, which is used to sort all the error-collected package data items in ascending order based on the timestamps corresponding to the error-collected package data items; and a division unit 7023, which is used to segment the sorted error-collected package data items based on a sliding window of a preset length to obtain multiple segments of data to be analyzed.

[0117] In this embodiment, the statistical module 703 includes: a first classification unit 7031, which is used to classify the error packets of each segment of data to be analyzed, and obtain error packet data sets of multiple error categories corresponding to each segment of data to be analyzed; a statistical unit 7032, which is used to count the number of error packets in each error packet data set corresponding to each segment of data to be analyzed, so as to obtain the number of error packets of different error categories of each segment of data to be analyzed, and to count the number of error packets in all error packet data sets corresponding to each segment of data to be analyzed, so as to obtain the total number of error packets corresponding to each segment of data to be analyzed; an integration unit 7033, which is used to integrate the statistical results of multiple segments of data to be analyzed, and obtain the statistical error packet data.

[0118] In this embodiment, the prediction module 704 includes: an extraction unit 7041, which is used to extract the number of error packets of different error categories for each segment of data to be analyzed one by one based on the statistical error packet data; a second classification unit 7042, which is used to put the number of error packets of the same error type into the same set to form a plurality of time series data corresponding to the error categories; a prediction unit 7043, which is used to use a time series analysis algorithm to predict each time series data to obtain the predicted number of error packets corresponding to the error category.

[0119] In this embodiment, the generation module 705 includes: a calling unit 7051, which is used to call a preset error packet analysis table template, and fill the statistical error packet data and the predicted number of error packets into the default parameters of the error packet analysis table template; a drawing unit 7052, which is used to extract the number of error packets of different error categories of multiple segments of data to be analyzed from the statistical error packet data, and draw an error packet trend chart based on the extracted data; a generation unit 7053, which is used to insert the error packet trend chart as a chart into the filled error packet analysis table template to generate an error packet analysis table.

[0120] In this embodiment, the error packet data after verification is divided into multiple segments of data to be analyzed in chronological order, the number of error packets of different error categories and the total number of error packets of each segment of data to be analyzed are counted, and the statistical results of the multiple segments of data to be analyzed are integrated to obtain the error packet data after statistics, and based on the error packet data after statistics, a time series analysis algorithm is used to predict the predicted number of error packets of different error categories, and finally a error packet analysis table is generated by combining the error packet data after statistics and the predicted number of error packets. By statistically analyzing and predicting the error packet data using the statistical analysis and time series analysis algorithm, not only can the current error status of the error packet be understood and the error trends and patterns in different time periods be identified, but also future error trends can be predicted. In addition, the generated error packet analysis table presents data in a visual manner, so that decision makers can quickly understand the problem and formulate corresponding solutions.

[0121] above Figure 7 The staggered packet data management device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The staggered packet data management device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0122] Figure 81 is a schematic diagram of the structure of a mismatched packet data management device provided by an embodiment of the present invention. The device 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 810 (for example, one or more processors) and a memory 820, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 833 or data 832. Among them, the memory 820 and the storage medium 830 can be short-term storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the device 800. Furthermore, the processor 810 may be configured to communicate with the storage medium 830 to execute a series of instruction operations in the storage medium on the device 800.

[0123] The device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input and output interfaces 860, and / or one or more operating systems 831, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc.

[0124] An embodiment of the present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the error packet data management method.

[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0127] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for managing mismatched packet data, characterized in that: The mismatched packet data management method comprises: Obtaining wrong package data, verifying the wrong package data, and obtaining verified wrong package data; Sending the verified error-collected data to a message queue, and dividing the verified error-collected data into multiple segments of data to be analyzed in chronological order; Counting the number of wrong packets of different error categories and the total number of wrong packets for each segment of data to be analyzed, and integrating the statistical results of the multiple segments of data to be analyzed to obtain the wrong packet data after statistics; Based on the statistical error packet data, a time series analysis algorithm is used to predict the number of error packets of different error categories; A misaligned packet analysis table is generated by combining the statistical misaligned packet data and the predicted number of misaligned packets.

2. The method for managing mismatched packet data according to claim 1, characterized in that: The step of obtaining the wrong package data and verifying the wrong package data to obtain the verified wrong package data includes: Obtaining the wrong set package data from the wrong set package database; Performing value verification and format verification on the collected wrong package data to obtain the verified wrong package data; The verified error packet data is converted into JSON format.

3. The method for managing mismatched packet data according to claim 1, characterized in that: The step of sending the verified error-collected data to a message queue and dividing the verified error-collected data into multiple segments of data to be analyzed in chronological order includes: Send the verified error-collected package data to a message queue, and traverse each error-collected package data item in the verified error-collected package data, and record a timestamp corresponding to each error-collected package data item; Based on the timestamps corresponding to the mismatched packet data items, all mismatched packet data items are sorted in ascending order; The sorted mismatched packet data items are segmented based on a sliding window of a preset length to obtain multiple segments of data to be analyzed.

4. The method for managing mismatched packet data according to claim 1, characterized in that: The counting of the number of error packets of different error categories and the total number of error packets of each segment of data to be analyzed, and integrating the statistical results of the multiple segments of data to be analyzed to obtain the statistical error packet data, includes: Classifying the error packets of each segment of data to be analyzed to obtain a plurality of error-classified error packet data sets corresponding to each segment of data to be analyzed; Counting the number of wrong packets in each wrong packet data set corresponding to each segment of data to be analyzed, so as to obtain the number of wrong packets of different error categories of each segment of data to be analyzed, and counting the number of wrong packets in all wrong packet data sets corresponding to each segment of data to be analyzed, so as to obtain the total number of wrong packets corresponding to each segment of data to be analyzed; Integrate the statistical results of multiple segments of data to be analyzed to obtain statistical error-free data.

5. The method for managing mismatched packet data according to claim 1, characterized in that: The method of using a time series analysis algorithm to predict the number of wrong packet sets of different error categories based on the statistical wrong packet set data includes: Based on the statistical error-collected packet data, the number of error-collected packets of different error categories of each segment of data to be analyzed is extracted one by one; The number of error packets of the same error type is put into the same set according to the timestamp, forming multiple time series data corresponding to the error categories; For each time series data, the time series analysis algorithm is used to predict the predicted number of error sets corresponding to the error category.

6. The method for managing mismatched packet data according to claim 5, characterized in that: The method of using a time series analysis algorithm to predict each time series data to obtain a predicted number of error sets corresponding to the error category includes: Collect historical error packet data, pre-process the historical error packet data, and divide the pre-processed historical error packet data into a training set and a test set; Use the training set to train the pre-built time series analysis model, and use the test set to evaluate and tune the pre-built time series analysis model to obtain a trained time series analysis model; Each time series data is input into the time series analysis model one by one to obtain the predicted number of error sets corresponding to the error category.

7. The method for managing mismatched packet data according to claim 1, characterized in that: The generating of the error packet analysis table by combining the statistical error packet data and the predicted number of error packets includes: Call the preset wrong packet analysis table template, and fill the wrong packet data after statistics and the predicted number of wrong packets into the default parameters of the wrong packet analysis table template; Extract the number of wrong packets of different error categories in multiple segments of data to be analyzed from the wrong packet data after statistics, and draw a wrong packet trend chart based on the extracted data; Insert the wrong package trend chart as a chart into the filled wrong package analysis table template to generate the wrong package analysis table.

8. A device for managing mismatched packet data, characterized in that: include: A verification module is used to obtain the wrong package data, verify the wrong package data, and obtain the verified wrong package data; A division module, used for sending the verified error-collected data to a message queue, and dividing the verified error-collected data into multiple segments of data to be analyzed in chronological order; A statistical module, used to count the number of wrong packets of different error categories and the total number of wrong packets for each segment of data to be analyzed, and integrate the statistical results of the multiple segments of data to be analyzed to obtain the wrong packet data after statistics; A prediction module, used to predict the predicted number of wrong packets of different error categories based on the statistical wrong packet data using a time series analysis algorithm; A generating module is used to generate a misaligned packet analysis table by combining the statistical misaligned packet data and the predicted number of misaligned packets.

9. A mismatched packet data management device, characterized in that: comprising a memory and at least one processor, wherein the memory has computer-readable instructions stored therein; The at least one processor calls the computer-readable instructions in the memory to execute the steps of the error packet data management method according to any one of claims 1-7.

10. A computer-readable storage medium having computer-readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the steps of the method for managing mismatched packet data as claimed in any one of claims 1 to 7 are implemented.

Images