Methods, apparatus and readable storage media for determining abnormal traffic data

Abstract

This application discloses a method, apparatus, and readable storage medium for determining abnormal traffic data, relating to the field of communication technology, and used to improve the accuracy of determining abnormal traffic data. The method includes: acquiring traffic data of a target area within a historical time period; determining predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model; determining abnormal unit times in the target area, where the difference between the traffic data and the predicted traffic data within the abnormal unit time period is greater than a first threshold, and the abnormal unit time period is located within the target time period; and determining the traffic data of the target area as abnormal traffic data when the number of abnormal unit times exceeds a second threshold and is consecutive.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method, apparatus and readable storage medium for determining abnormal traffic data. Background Technology

[0002] With the rapid development of information technology, traffic data is growing exponentially. In order to detect anomalies in traffic data in a timely manner, it is necessary to monitor traffic data in real time and compare the traffic data obtained from the monitoring with preset traffic thresholds to determine whether the current traffic data is abnormal.

[0003] However, traffic data changes dynamically in real time, and relying solely on fixed, preset traffic thresholds to determine whether traffic data is abnormal has low accuracy. Therefore, improving the accuracy of identifying abnormal traffic data is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, and readable storage medium for determining abnormal traffic data, which can improve the accuracy of determining abnormal traffic data.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, a method for determining abnormal traffic data is provided, comprising: acquiring traffic data of a target area within a historical time period; determining the predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model; determining abnormal unit time periods of the target area, wherein the difference between the traffic data and the predicted traffic data within the abnormal unit time period is greater than a first threshold, and the abnormal unit time period is located within the target time period; and determining the traffic data of the target area as abnormal traffic data when the number of abnormal unit times exceeds a second threshold and is continuous.

[0007] Based on the technical solution provided in this application, after acquiring traffic data of a target area within a historical time period, the determining device can determine the predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model. Thus, since the traffic prediction model is pre-trained, the predicted traffic data of the target area within the target time period can be accurately determined. Furthermore, the determining device identifies abnormal unit times in the target area. If the difference between the traffic data and the predicted traffic data within an abnormal unit time period is greater than a first threshold, the abnormal unit time period is located within the target time period. If the number of abnormal unit times exceeds a second threshold and is continuous, the traffic data of the target area is determined to be abnormal traffic data. Therefore, since the traffic data within the target time period is dynamically determined based on traffic data within the historical time period, and the difference between the traffic data and the predicted traffic data within an abnormal unit time period is greater than the first threshold, rather than solely relying on a fixed preset traffic threshold to determine whether the traffic data is abnormal, the determination of whether the traffic data is abnormal can be more accurate. In addition, since traffic anomalies often exhibit continuity, judging the number and continuity of abnormal unit times can further improve the accuracy of determining abnormal traffic data.

[0008] Optionally, the traffic data of the target area within a historical time period can be obtained, including: obtaining the original traffic data of the target area within a historical time period; cleaning the original traffic data to obtain the traffic data of the target area within a historical time period; the data cleaning includes supplementing invalid data in the original traffic data, where invalid data is data with null values.

[0009] Optionally, the target time period includes multiple unit times. Determining abnormal unit times for the target area includes: obtaining traffic data for each unit time in the target area within the target time period; if the difference between the traffic data in the first unit time and the predicted traffic data in the first unit time is greater than a first threshold, then the first unit time is determined to be an abnormal unit time, and the first unit time is any one of the multiple unit times.

[0010] Optionally, the method for determining abnormal traffic data further includes: acquiring multiple sets of sample data, including traffic data of the sample area within a first time period and traffic data within a second time period; the second time period is after the first time period; dividing the multiple sets of sample data into a training set and a test set according to a preset ratio; inputting the traffic data of the sample area in the first time period and the traffic data in the second time period of the training set into the traffic prediction model for training, adjusting the model parameters in the traffic prediction model until the output result is the same as the traffic data in the second time period, thus obtaining the trained traffic prediction model; inputting the traffic data of the sample area in the first time period and the traffic data in the second time period of the test set into the trained traffic prediction model, and determining the trained traffic prediction model as the trained traffic prediction model if the accuracy of the output result is greater than a third threshold.

[0011] Secondly, an apparatus for determining abnormal traffic data is provided. The apparatus includes: an acquisition unit and a determination unit. The acquisition unit is used to acquire traffic data of a target area within a historical time period. The determination unit is used to determine the predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model. The determination unit is also used to determine abnormal unit times in the target area, wherein the difference between the traffic data and the predicted traffic data within the abnormal unit time period is greater than a first threshold, and the abnormal unit time period is located within the target time period. The determination unit is also used to determine the traffic data of the target area as abnormal traffic data when the number of abnormal unit times is greater than a second threshold and is continuous.

[0012] Optionally, the acquisition unit is specifically used for: acquiring raw traffic data of the target area within a historical time period; cleaning the raw traffic data to acquire traffic data of the target area within a historical time period; data cleaning includes supplementing invalid data in the raw traffic data, where invalid data is data with null values.

[0013] Optionally, the determination unit is specifically used to: obtain the traffic data of the target area in each unit time within the target time period; if the difference between the traffic data in the first unit time and the predicted traffic data in the first unit time is greater than a first threshold, then the first unit time is determined to be an abnormal unit time, and the first unit time is any one of multiple unit times.

[0014] Optionally, the device further includes a processing unit; an acquisition unit, further configured to acquire multiple sets of sample data, including traffic data of a sample region within a first time period and traffic data within a second time period; the second time period is located after the first time period; a processing unit, configured to divide the multiple sets of sample data into a training set and a test set according to a preset ratio; a processing unit, further configured to input the traffic data of the sample region of the training set within the first time period and the traffic data within the second time period into the traffic prediction model for training, adjusting the model parameters in the traffic prediction model until the output result is the same as the traffic data within the second time period, thereby obtaining the trained traffic prediction model; and a determination unit, further configured to input the traffic data of the sample region of the test set within the first time period and the traffic data within the second time period into the trained traffic prediction model, and determine the trained traffic prediction model as a trained traffic prediction model if the accuracy of the output result is greater than a third threshold.

[0015] Thirdly, an apparatus for determining abnormal traffic data is provided. This apparatus can perform the functions of the apparatus for determining abnormal traffic data in the above-mentioned aspects or possible designs. The functions can be implemented by hardware. For example, in one possible design, the apparatus for determining abnormal traffic data may include a processor and a communication interface. The processor can be used to support the apparatus for determining abnormal traffic data in performing the functions involved in the first aspect or any possible design of the first aspect.

[0016] In another possible design, the apparatus for determining abnormal traffic data may further include a memory for storing necessary computer execution instructions and data. When the apparatus for determining abnormal traffic data is running, the processor executes the computer execution instructions stored in the memory to cause the apparatus for determining abnormal traffic data to perform the first aspect or any of the possible methods for determining abnormal traffic data described above.

[0017] Fourthly, a computer-readable storage medium is provided, which may be a readable non-volatile storage medium storing computer instructions or programs that, when executed on a computer, enable the computer to perform the method for determining abnormal traffic data described in the first aspect or any of the possible methods described in the first aspect.

[0018] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, enables the computer to perform the method for determining abnormal traffic data according to the first aspect or any possible design of the above aspects.

[0019] A sixth aspect provides an electronic device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions, which, when executed by the one or more processors, cause the electronic device to perform a method for determining abnormal traffic data as described in the first aspect or any possible design of the first aspect.

[0020] In a seventh aspect, a chip system is provided, comprising a processor and a communication interface, which can be used to implement the functions performed by the device for determining abnormal traffic data in the first aspect or any possible design of the first aspect. In one possible design, the chip system further includes a memory for storing program instructions and / or data. The chip system may be composed of chips or may include chips and other discrete devices, without limitation. Attached Figure Description

[0021] Figure 1 A schematic diagram of a system for determining abnormal traffic data provided in an embodiment of this application;

[0022] Figure 2 A schematic diagram of another system for determining abnormal traffic data provided in the embodiments of this application;

[0023] Figure 3 A schematic diagram of the structure of a device for determining abnormal traffic data provided in an embodiment of this application;

[0024] Figure 4 A flowchart illustrating a method for determining abnormal traffic data provided in an embodiment of this application;

[0025] Figure 5 A flowchart illustrating another method for determining abnormal traffic data provided in an embodiment of this application;

[0026] Figure 6 A flowchart illustrating another method for determining abnormal traffic data provided in an embodiment of this application;

[0027] Figure 7 A flowchart illustrating another method for determining abnormal traffic data provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of another device for determining abnormal traffic data provided in an embodiment of this application. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0031] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0032] With the rapid development of information technology, traffic data is growing exponentially. In order to detect anomalies in traffic data in a timely manner, it is necessary to monitor traffic data in real time and compare the traffic data obtained from the monitoring with preset traffic thresholds to determine whether the current traffic data is abnormal.

[0033] However, traffic data changes dynamically in real time, and relying solely on fixed, preset traffic thresholds to determine whether traffic data is abnormal has low accuracy. Therefore, improving the accuracy of identifying abnormal traffic data is a pressing technical problem that needs to be solved.

[0034] In addition, common anomalies include sudden spikes or drops in traffic data, and values ​​that do not conform to normal conditions at the current moment. It is precisely this uncertainty that leads to a high rate of missed detections and false alarms in existing traffic data detection systems. For example, some traffic data may show an overall smooth decline on a particular day, which the system fails to identify as an anomaly; other traffic data may experience short-term fluctuations that are normal, but the system frequently issues anomaly alarms.

[0035] Furthermore, due to the uncertainty of real-time traffic data, missed detections and false alarms cannot be completely eliminated. However, depending on the characteristics of the system, operations and maintenance personnel often need to configure it according to their own business needs. For example, some traffic monitoring systems monitor critical core devices, allowing for a higher false alarm rate but not accepting missed detections. On the other hand, some routine monitoring systems, or those whose data fluctuates frequently, require operations and maintenance personnel to avoid excessive alarm disturbances. Often, continuous adjustments need to be made based on data conditions after the system is actually online, resulting in poor flexibility.

[0036] In view of this, embodiments of this application provide a method for determining abnormal traffic data, including: acquiring traffic data of a target area within a historical time period; determining predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model; determining abnormal unit time periods of the target area, wherein the difference between the traffic data and the predicted traffic data within the abnormal unit time period is greater than a first threshold, and the abnormal unit time period is located within the target time period; and determining the traffic data of the target area as abnormal traffic data when the number of abnormal unit times exceeds a second threshold and is continuous.

[0037] The methods provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0038] It should be noted that the network system described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network systems and the emergence of other network systems, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0039] Figure 1 The diagram shown is a schematic representation of a system for determining abnormal traffic data provided in an embodiment of this application. Figure 1 As shown, the abnormal traffic data determination system may include a data storage device 11 and an abnormal traffic data determination device 12. The data storage device 11 is connected to the abnormal traffic data determination device 12 (hereinafter referred to as the determination device 12). For example, the target terminal 11 and the determination device 12 can be connected wirelessly or wiredly, and this embodiment of the invention does not limit the connection.

[0040] In the embodiments of this application, the data storage device 11 may be an operator's database, a data processing center, a server, etc. The embodiments of this application do not limit the specific technology, quantity, or form of the data storage device 11.

[0041] The determining device 12 involved in the embodiments of this application can be a server, computer, etc., or it can be called a terminal, mobile station (MS), mobile terminal (MT), etc. It is a device that provides voice and / or data connectivity to users. For example, the determining device 12 can be a handheld device with wireless connection function, in-vehicle device, etc. Specifically, it can be a smartphone, pocket personal computer (PPC), handheld computer, personal digital assistant (PDA), laptop computer, tablet computer, wearable device, or in-vehicle device, etc. The embodiments of this application do not limit the specific technology, quantity, or form of the determining device 12.

[0042] The data storage device 11 stores traffic data of the target area within a historical time period and sends this traffic data to the determining device 12. The determining device 12, upon receiving the traffic data of the target area within a historical time period from the data storage device 11, determines the predicted traffic data of the target area within a target time period and, based on the predicted traffic data, determines whether the traffic data of the target area is abnormal.

[0043] In different application scenarios, the data storage device 11 and the determining device 12 can be independent devices or integrated into the same device. This embodiment of the invention does not impose specific limitations on this.

[0044] It should be noted that, Figure 1 This is just an example framework diagram. Figure 1 The names of the various devices included are unrestricted, and except for Figure 1 In addition to the functional nodes shown, other nodes may also be included, but this application embodiment does not limit this.

[0045] Figure 2 This diagram illustrates yet another system for determining abnormal traffic data provided in an embodiment of this application. Figure 2 As shown, the system for determining abnormal traffic data may include a data access module, a data management module, a data preprocessing module, a data prediction module, a rule engine module, a dictionary module, and a pipeline configuration module.

[0046] The following section describes the various modules of the system.

[0047] 1. Data Access Module: This module connects to a data storage device (also known as an application system) via a preset method. It periodically retrieves traffic data from the data storage device over historical time periods and supports preliminary cleaning of the traffic data using Structured Query Language (SQL). The final result is then written to a MySQL database. Preset methods include File Transfer Protocol (FTP), HIVE, and Kafka.

[0048] 2. Data Management Module: This module allows for the management of incoming traffic data. Management operations may include creating, categorizing, viewing, and periodically cleaning up traffic data.

[0049] 3. Data Preprocessing Module: This module provides Python-based code writing capabilities. By using custom scripts, it processes incoming traffic data into the format required by subsequent data prediction models and removes outlier data to enhance the system's robustness.

[0050] 4. Data Prediction Module: This module can be used to output predicted traffic data for a target time period based on the input traffic data for a historical time period.

[0051] 5. Rule Engine Module: This module can be used to determine whether traffic data in a target area is abnormal.

[0052] 6. Dictionary Module: This module encapsulates data into a dictionary format for subsequent data transmission back via Kafka. By configuring key-value pairs, additional information can be appended to the original data for application systems to identify and process it accordingly.

[0053] 7. Pipeline Configuration Module: This module can be used to encapsulate the above modules into components, providing pipeline templates for configuration into production-oriented pipelines.

[0054] It should be noted that, Figure 2 This is just an example framework diagram. Figure 2 The names of the modules included are unrestricted, and except for Figure 2 In addition to the functional modules shown, other modules may also be included, but this application embodiment does not limit this.

[0055] In practical implementation, Figure 1 Each device in the process can be adopted Figure 3 The shown composition structure, or including Figure 3 The components shown. Figure 3This is a schematic diagram illustrating the composition of a determining device 200 provided in an embodiment of this application. The determining device 200 can be a server, or it can be a chip or system-on-a-chip within the server. Figure 3 As shown, the determining device 200 includes a processor 201, a communication interface 202, and a communication line 203.

[0056] Furthermore, the determining device 200 may also include a memory 204. The processor 201, memory 204, and communication interface 202 can be connected via a communication line 203.

[0057] The processor 201 can be a CPU, a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 201 can also be other devices with processing capabilities, such as circuits, devices, or software modules, without limitation.

[0058] Communication interface 202 is used to communicate with other devices or other communication networks. Communication interface 202 can be a module, circuit, communication interface, or any device capable of enabling communication.

[0059] Communication line 203 is used to transmit information between the components included in determining device 200.

[0060] Memory 204 is used to store instructions. These instructions can be computer programs.

[0061] The memory 204 can be a read-only memory (ROM) or other type of static storage device that can store static information and / or instructions; it can also be a random access memory (RAM) or other type of dynamic storage device that can store information and / or instructions; it can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, etc., without limitation.

[0062] It should be noted that the memory 204 can exist independently of the processor 201 or can be integrated with the processor 201. The memory 204 can be used to store instructions, program code, or some data, etc. The memory 204 can be located inside or outside the determining device 200, without limitation. The processor 201 is used to execute the instructions stored in the memory 204 to implement the abnormal traffic data determination method provided in the following embodiments of this application.

[0063] In one example, processor 201 may include one or more CPUs, for example, Figure 3 CPU0 and CPU1 in the CPU.

[0064] As an optional implementation, the determining device 200 includes multiple processors, for example, besides Figure 3 In addition to processor 201, it may also include processor 205.

[0065] It should be pointed out that, Figure 3 The composition shown does not constitute a basis for this. Figure 1 The limitations of each device in the process, except Figure 3 In addition to the components shown, Figure 1 The various devices in the can include ratio Figure 3 More or fewer components, or combinations of certain components, or different arrangements of components.

[0066] In this embodiment of the application, the chip system may be composed of chips or may include chips and other discrete devices.

[0067] Furthermore, the actions, terms, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are merely examples, and other names may be used in specific implementations without limitation.

[0068] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0069] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0070] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0071] The following is combined with Figure 1 The system for determining abnormal traffic data shown herein describes the method for determining abnormal traffic data provided in the embodiments of this application.

[0072] Figure 4 This application provides a method for determining abnormal traffic data, which can be applied to a server or a determining device. Figure 1 The determining device 12 can also be a component within the determining device 12, such as a chip. This application embodiment uses an application to the determining device 12 as an example for illustration. Figure 4 As shown, the method includes the following S401-S404:

[0073] S401, The device determines the flow data of the target area within a historical time period.

[0074] The target area can be set as needed. For example, it can be a city area or the coverage area of ​​one or more base stations. The historical time period can also be set as needed. For example, it can be the week before the current time, the month before the current time, or the year before the current time, etc.

[0075] Traffic data can refer to data traffic generated by base stations or user registrations. Traffic data can include the numerical value of the traffic and its corresponding timestamp, and can also include the base station identifier that generated the traffic data, error logs, etc., without limitation.

[0076] In one possible implementation, the determining device can be communicatively connected to a data storage device. The determining device can send a first subscription message to the data storage device, requesting traffic data for a historical time period. Upon receiving the first subscription message, the data storage device can send a first reply message, which includes the traffic data for the historical time period. After receiving the first reply message, the determining device can retrieve the traffic data for the historical time period based on it.

[0077] S402. The determining device determines the predicted flow data of the target area within the target time period based on the flow data within the historical time period and the trained flow prediction model.

[0078] The traffic forecasting model can be configured as needed. For example, it can be an autoregressive integrated moving average (ARIMA) model (also known as an integrated moving average autoregressive model).

[0079] The target time period can be determined based on the duration of the historical time period. For example, if the historical time period is one year prior to the current moment, the target time period could be one month following the current moment. Another example is that if the historical time period is one month prior to the current moment, the target time period could be one week following the current moment. Yet another example is that if the historical time period is one week prior to the current moment, the target time period could be one day following the current moment.

[0080] As one possible implementation, after the determining device acquires the traffic data within a historical time period, it can input the traffic data within the historical time period into a trained traffic prediction model. The traffic prediction model can then process the traffic data within the historical time period according to its built-in algorithm to obtain the predicted traffic data for the target area within the target time period.

[0081] It should be noted that the trained traffic prediction model can be pre-configured by operations and maintenance personnel in a specific device.

[0082] S403, Determine the abnormal unit time of the target area.

[0083] Specifically, an abnormal unit of time is defined as one in which the difference between the traffic data and the predicted traffic data exceeds a first threshold, and the abnormal unit of time falls within the target time period. The unit of time can be set as needed, for example, 10 seconds, 1 minute, 5 minutes, etc.

[0084] As one possible implementation, the data storage device can also acquire traffic data within the target area from the network management system at regular intervals and send the traffic data within the target area to the determining device at regular intervals. The determining device can acquire the traffic data within the target area from the data storage device at regular intervals. After acquiring the traffic data of the target area within the current unit time, the predicted traffic data for the current unit time can be determined from the target time period, and the difference between the predicted traffic data and the actual traffic data of the target area within the current unit time can be determined. If the difference is greater than a first threshold, the current unit time is determined to be an abnormal unit time.

[0085] It should be noted that the first threshold can be set as needed. For example, it can be 10G or 100G, etc.

[0086] S404. If the number of abnormal units of time exceeds the second threshold and is continuous, the flow data of the target area shall be determined as abnormal flow data.

[0087] The second threshold can be set as needed. For example, it can be 3, 5, or 10, etc. Abnormal unit time continuity means there is no interval between unit times. For example, if abnormal unit times include unit time 1 (10:00-10:05) and unit time 2 (10:05-10:10), then unit time 1 and unit time 2 can be determined to be continuous.

[0088] As one possible implementation, the device is equipped with a comparator that compares the number of abnormal events per unit time with a second threshold. The device can then determine whether the number of abnormal events per unit time exceeds the second threshold. Furthermore, if the number of abnormal events per unit time exceeds the second threshold continuously, the device determines the traffic data in the target area as abnormal traffic data.

[0089] For example, if the abnormal unit time includes a first unit time (10:00-10:05) and a second unit time (10:05-10:10), and the second threshold is 2, the determining device can determine that the traffic data of the target area is abnormal traffic data.

[0090] Furthermore, if the traffic data in the target area is determined to be abnormal, the determining device can generate an alarm message. If the number of abnormal traffic events per unit time is less than or equal to a second threshold, the determining device can compress and record the data traffic per abnormal unit time.

[0091] Based on the technical solution provided in this application, after acquiring traffic data of a target area within a historical time period, the determining device can determine the predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model. Thus, since the traffic prediction model is pre-trained, the predicted traffic data of the target area within the target time period can be accurately determined. Furthermore, the determining device identifies abnormal unit times in the target area. If the difference between the traffic data and the predicted traffic data within an abnormal unit time period is greater than a first threshold, the abnormal unit time period is located within the target time period. If the number of abnormal unit times exceeds a second threshold and is continuous, the traffic data of the target area is determined to be abnormal traffic data. Therefore, since the traffic data within the target time period is dynamically determined based on traffic data within the historical time period, and the difference between the traffic data and the predicted traffic data within an abnormal unit time period is greater than the first threshold, rather than solely relying on a fixed preset traffic threshold to determine whether the traffic data is abnormal, the determination of whether the traffic data is abnormal can be more accurate. In addition, since traffic anomalies often exhibit continuity, judging the number and continuity of abnormal unit times can further improve the accuracy of determining abnormal traffic data.

[0092] One possible implementation, such as Figure 5 As shown, in order to obtain traffic data of the target area within a historical time period, S401 in the deployment method of this application may further include the following S501-S502.

[0093] S501, The device determines the raw flow data of the target area within a historical time period.

[0094] This involves obtaining the raw traffic data of the target area within a historical time period.

[0095] It should be noted that the specific implementation method of this step can be referred to the description of S401 above, and will not be repeated here.

[0096] S502, The device cleans the raw flow data to obtain the flow data of the target area within a historical time period.

[0097] Data cleaning includes supplementing invalid data in the original traffic data. Invalid data refers to data with null values.

[0098] As one possible implementation, after acquiring the raw flow data, the determining device can identify the missing unit time in the raw flow data and fill the missing unit time with the average value of the flow rate before and after the missing unit time.

[0099] For example, the flow rate values ​​per unit time before and after the missing unit time are x respectively. i-1 xi+1 In this case, the value of the flow rate per unit time is (x i-1 +x i+1 ) / 2.

[0100] It should be noted that the flow rate value for a missing unit of time is a null value.

[0101] Thus, after acquiring the raw traffic data of the target area within a historical time period, the determining device can perform data cleaning on the raw traffic data to supplement invalid data in the raw traffic data, thereby obtaining the traffic data of the target area within a historical time period more accurately.

[0102] One possible implementation, such as Figure 6 As shown, in order to determine the abnormal unit time, S403 in the determination method of this application may further include the following S601-S602.

[0103] S601, The device determines the flow data of the target area for each unit of time within the target time period.

[0104] As one possible implementation, the determining device can communicate with a network management system. The determining device can send a second subscription message to the network management system, requesting real-time traffic data. After receiving the second subscription message from the determining device, the network management system can send a second response message, which includes traffic data for each unit of time. After receiving the second response message from the network management system, the determining device can obtain traffic data for each unit of time within a target time period based on the second response message.

[0105] S602. If the difference between the flow data in the first unit time and the predicted flow data in the first unit time is greater than the first threshold, the determining device determines the first unit time as an abnormal unit time.

[0106] The first unit of time can be any one of multiple unit times.

[0107] As one possible implementation, the determining device includes a comparator that compares the difference between the flow data and the predicted flow data within a first unit time period with a first threshold. The comparator then sends an indication message to the determining device, indicating the relationship between the difference between the flow data and the predicted flow data within the first unit time period and the first threshold. Accordingly, the determining device can determine, based on the indication message, whether the difference between the flow data and the predicted flow data within the first unit time period is greater than the first threshold.

[0108] One possible implementation, such as Figure 7As shown, in order to obtain a well-trained traffic prediction model, the determination method of this application may further include the following steps S701-S704.

[0109] S701, The device acquires multiple sets of sample data.

[0110] The sample data includes traffic data for the sample area during the first time period and traffic data during the second time period. The second time period follows the first time period.

[0111] It should be noted that the method for obtaining this step can be the same as that for S401 above, and will not be repeated here.

[0112] S702. The device determines that multiple sets of sample data are divided into training set and test set according to a preset ratio.

[0113] The preset ratio can be set as needed. For example, it can be 8:1, 6:1, or 5:1, etc.

[0114] S703, The determining device inputs the traffic data of the sample area of ​​the training set in the first time period and the traffic data in the second time period into the traffic prediction model for training, and adjusts the model parameters in the traffic prediction model until the output result is the same as the traffic data in the second time period, thus obtaining the trained traffic prediction model.

[0115] As one possible implementation, the flow determination device is equipped with initial model parameters and a preset adjustment step size. The flow determination device can determine the trained flow prediction model based on the initial model parameters, the preset adjustment step size, and the corresponding output results.

[0116] It should be noted that the preset adjustment step size can be set as needed. For example, the preset adjustment step size can be 1, 2, 5, etc.

[0117] For example, with initial model parameters of x and a preset adjustment step size of y, the detection device can first use the initial model parameters x to output a first result. If the first result differs from the traffic data in the corresponding second time period, the initial model parameters are adjusted to x+y, and a second result is output. If the second result is the same as the traffic data in the corresponding second time period, the preset model with adjusted parameters of x+y is determined to be the trained traffic prediction model.

[0118] S704. The determining device inputs the traffic data of the sample area of ​​the test set in the first time period and the traffic data in the second time period into the trained traffic prediction model, and determines the trained traffic prediction model as the trained traffic prediction model if the accuracy of the output result is greater than the third threshold.

[0119] The third threshold can be set as needed. For example, it can be 80%, 90%, or 95%, etc.

[0120] As one possible implementation, the determining device includes a comparator that compares the accuracy of the output result with a third threshold. The comparator then sends an indication message to the determining device, indicating the relationship between the accuracy of the output result and the third threshold. Accordingly, the determining device can use the indication message to determine whether the accuracy of the output result is greater than the third threshold.

[0121] It should be noted that, in order to prevent excessive iterations from affecting computational efficiency, the determining device can also be set with a range of model parameters or an iteration threshold.

[0122] The various solutions in the above embodiments of this application can be combined without contradiction.

[0123] This application embodiment can divide the determining device into functional modules or functional units according to the above method examples. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0124] When dividing each function into modules according to its corresponding function. Figure 8 A schematic diagram of a determining device is shown. This determining device can be a server or a chip applied in a server. This determining device can be used to perform the functions related to the server involved in the above embodiments. Figure 8 The determining device shown may include: an acquisition unit 801 and a determining unit 802; the acquisition unit 801 is used to acquire traffic data of the target area within a historical time period; the determining unit 802 is used to determine the predicted traffic data of the target area within a target time period based on the traffic data within the historical time period and a trained traffic prediction model; the determining unit 802 is also used to determine abnormal unit times of the target area, wherein the difference between the traffic data and the predicted traffic data within the abnormal unit time period is greater than a first threshold, and the abnormal unit time is located within the target time period; the determining unit 802 is also used to determine the traffic data of the target area as abnormal traffic data when the number of abnormal unit times is greater than a second threshold and is continuous.

[0125] In one possible design, the acquisition unit 801 is specifically used to: acquire the original traffic data of the target area within a historical time period; perform data cleaning on the original traffic data to acquire the traffic data of the target area within a historical time period; the data cleaning includes supplementing invalid data in the original traffic data, where invalid data is data with null values.

[0126] In one possible design, the determining unit 802 is specifically used to: acquire the traffic data of the target area for each unit time within the target time period; if the difference between the traffic data in the first unit time and the predicted traffic data in the first unit time is greater than a first threshold, then the first unit time is determined to be an abnormal unit time, and the first unit time is any one of multiple unit times.

[0127] In one possible design, the device further includes a processing unit 803; an acquisition unit 801, further configured to acquire multiple sets of sample data, including traffic data of a sample region within a first time period and traffic data within a second time period; the second time period is located after the first time period; the processing unit 803, configured to divide the multiple sets of sample data into a training set and a test set according to a preset ratio; the processing unit 803, further configured to input the traffic data of the sample region of the training set within the first time period and the traffic data within the second time period into the traffic prediction model for training, adjusting the model parameters in the traffic prediction model until the output result is the same as the traffic data within the second time period, thus obtaining the trained traffic prediction model; and a determination unit 802, further configured to input the traffic data of the sample region of the test set within the first time period and the traffic data within the second time period into the trained traffic prediction model, and if the accuracy of the output result is greater than a third threshold, determine the trained traffic prediction model as a trained traffic prediction model.

[0128] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. This program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be an internal storage unit of the determining device (including a data sending end and / or a data receiving end) in any of the foregoing embodiments, such as a hard disk or memory of the determining device. The computer-readable storage medium can also be an external storage device of the terminal device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device. Further, the computer-readable storage medium can include both the internal storage unit of the determining device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the determining device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0129] It should be noted that the terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0130] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0136] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining abnormal traffic data, characterized in that, The method includes: Obtain traffic data for the target area within a historical time period; Based on the traffic data within the historical time period and the trained traffic prediction model, the predicted traffic data for the target area within the target time period is determined. An abnormal unit time in the target area is determined, wherein the difference between the traffic data and the predicted traffic data within the abnormal unit time is greater than a first threshold, and the abnormal unit time is located within the target time period. If the number of abnormal units of time exceeds the second threshold and is continuous, the traffic data of the target area is determined to be abnormal traffic data; If the number of abnormal units of time is less than or equal to the second threshold, the data traffic of the abnormal units of time will be compressed and recorded.

2. The method according to claim 1, characterized in that, The acquisition of traffic data for the target area within a historical time period includes: Obtain raw traffic data for the target area within a historical time period; The raw traffic data is cleaned to obtain traffic data for the target area within the historical time period; the data cleaning includes supplementing invalid data in the raw traffic data, wherein invalid data is data with null values.

3. The method according to claim 1, characterized in that, The target time period includes multiple unit times, and determining the abnormal unit times of the target region includes: Obtain the traffic data of the target area for each unit of time within the target time period; If the difference between the traffic data in the first unit time and the predicted traffic data in the first unit time is greater than the first threshold, then the first unit time is determined to be an abnormal unit time, and the first unit time is any one of the plurality of unit times.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Multiple sets of sample data are acquired, including traffic data of the sample area in a first time period and traffic data in a second time period; the second time period is located after the first time period. According to a preset ratio, the multiple sets of sample data are divided into a training set and a test set; The traffic data of the sample region in the training set during the first time period and the traffic data during the second time period are input into the traffic prediction model for training. The model parameters in the traffic prediction model are adjusted until the output result is the same as the traffic data during the second time period, and the trained traffic prediction model is obtained. The traffic data of the sample region of the test set during the first time period and the traffic data during the second time period are input into the trained traffic prediction model. If the accuracy of the output result is greater than a third threshold, the trained traffic prediction model is determined to be the trained traffic prediction model.

5. A device for determining abnormal traffic flow data, characterized in that, The device includes: an acquisition unit and a determination unit; The acquisition unit is used to acquire traffic data of the target area within a historical time period; The determining unit is used to determine the predicted traffic data of the target area within the target time period based on the traffic data within the historical time period and the trained traffic prediction model. The determining unit is further configured to determine the abnormal unit time of the target area, wherein the difference between the traffic data and the predicted traffic data within the abnormal unit time is greater than a first threshold, and the abnormal unit time is located within the target time period; The determining unit is further configured to determine the traffic data of the target area as abnormal traffic data when the number of abnormal units of time is greater than the second threshold and is continuous. If the number of abnormal units of time is less than or equal to the second threshold, the data traffic of the abnormal units of time will be compressed and recorded.

6. The apparatus according to claim 5, characterized in that, The acquisition unit is specifically used for: Obtain raw traffic data for the target area within a historical time period; The raw traffic data is cleaned to obtain traffic data for the target area within the historical time period; the data cleaning includes supplementing invalid data in the raw traffic data, wherein invalid data is data with null values.

7. The apparatus according to claim 5, characterized in that, The determining unit is specifically used for: Obtain the traffic data of the target area for each unit of time within the target time period; If the difference between the traffic data in the first unit time and the predicted traffic data in the first unit time is greater than the first threshold, then the first unit time is determined to be an abnormal unit time, and the first unit time is any one of the plurality of unit times.

8. The apparatus according to any one of claims 5-7, characterized in that, The device also includes a processing unit; The acquisition unit is further configured to acquire multiple sets of sample data, including traffic data of the sample area in a first time period and traffic data in a second time period; the second time period is located after the first time period. The processing unit is used to divide the multiple sets of sample data into a training set and a test set according to a preset ratio; The processing unit is further configured to input the traffic data of the sample region of the training set in the first time period and the traffic data in the second time period into the traffic prediction model for training, adjust the model parameters in the traffic prediction model until the output result is the same as the traffic data in the second time period, and obtain the trained traffic prediction model. The determining unit is further configured to input the traffic data of the sample region of the test set in the first time period and the traffic data in the second time period into the trained traffic prediction model, and determine the trained traffic prediction model as the trained traffic prediction model if the accuracy of the output result is greater than a third threshold.

9. A computer-readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed, implement the method as described in any one of claims 1-4.

10. A device for determining abnormal traffic flow data, characterized in that, include: The processor, memory, and communication interface; wherein the communication interface is used for communication between the device for determining the abnormal traffic data and other devices or networks; The memory is used to store one or more programs, the one or more programs including computer-executable instructions. When the device for determining abnormal traffic data is running, the processor executes the computer-executable instructions stored in the memory to cause the device for determining abnormal traffic data to perform the method of any one of claims 1-4.

Images