Data detection method and apparatus, and storage medium

Abstract

The application provides a data detection method and device and a storage medium, relates to the technical field of data detection, and can solve the problem of low efficiency of detecting data abnormities in the prior art. The method comprises the following steps: acquiring a plurality of historical quantity values and a target quantity value; the historical quantity value is a quantity value of flow data in each time period in a plurality of historical time periods; the target quantity value is a quantity value of actual flow data on a target date; obtaining a target prediction interval through a target time sequence algorithm and the historical quantity value; the target prediction interval is a prediction interval of the quantity value of the flow data; and comparing the target quantity value with the target prediction interval to obtain a data detection result. The application can improve the efficiency of data detection.

Description

Technical Field

[0001] This application relates to the field of data detection technology, and in particular to a data detection method, apparatus and storage medium. Background Technology

[0002] Data quality is crucial in the use of data; only with high-quality data can algorithms arrive at answers closer to the correct ones. One criterion for assessing data quality is completeness, which determines whether the data is comprehensive.

[0003] As holiday schedules change, so does the amount of data. Current technologies for monitoring the amount of data are inefficient at detecting anomalies. Summary of the Invention

[0004] This application provides a data detection method, apparatus, and storage medium, which solves the problem of low efficiency in the detection of data anomalies in the prior art and can improve the efficiency of data detection.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides a data detection method, which includes: acquiring multiple historical quantity values ​​and a target quantity value; the historical quantity value is the quantity value of traffic data in each time period within multiple historical time periods; the target quantity value is the quantity value of actual traffic data on a target date; obtaining a target prediction interval through a target time series algorithm and the historical quantity values; the target prediction interval is the prediction interval of the quantity value of traffic data; determining whether the target prediction interval includes the target quantity value, and obtaining a data detection result.

[0007] The above solution offers at least the following advantages: Based on the above technical solution, the data detection method provided in this application first involves the data detection device acquiring multiple historical quantity values ​​and a target quantity value. The data detection device can then calculate the target prediction interval using a target time series algorithm and the multiple historical quantity values. Since the historical quantity values ​​represent the quantity of traffic data in each time period within multiple historical time periods, and the target quantity value represents the quantity of actual traffic data on the target date, compared to the inefficiency of existing technologies in detecting anomalies in data, the data detection device can quickly obtain data detection results by determining whether the target prediction interval includes the target quantity value, thus greatly improving the efficiency of data detection.

[0008] In conjunction with the first aspect above, in one possible implementation, the method further includes: obtaining historical statements, target statements, connection identifiers of historical statements, and connection identifiers of target statements; the historical statements are used to represent multiple historical time periods of multiple historical quantity values; the target statements are used to represent the target date of the target quantity value; the historical quantity value is determined from the target database based on the historical statements and the connection identifiers of the historical statements; and the target quantity value is determined from the target database based on the target statements and the connection identifiers of the target statements.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: determining the historical quantity value of the target quantity within the target period; the target period includes the historical time period of the target quantity; based on the target time prediction algorithm, obtaining a first prediction interval by fitting the historical quantity value of the target quantity within each target period; and calculating the target prediction interval by linear regression of multiple first prediction intervals.

[0010] In conjunction with the first aspect mentioned above, in one possible implementation, the method further includes: determining that the data detection result is normal when the target quantity value is within the target prediction range; and determining that the data detection result is abnormal when the target quantity value is not within the target prediction range.

[0011] Secondly, this application provides a data detection device, which includes: a communication unit and a processing unit; the communication unit is used to acquire multiple historical quantity values ​​and a target quantity value; the historical quantity values ​​are the quantity values ​​of traffic data in each time period within multiple historical time periods; the target quantity value is the quantity value of actual traffic data on a target date; the processing unit is used to obtain a target prediction interval through a target time series algorithm and the historical quantity values; the target prediction interval is the prediction interval of the quantity value of traffic data; the processing unit is also used to determine whether the target prediction interval includes the target quantity value, and obtain a data detection result.

[0012] In conjunction with the second aspect above, in one possible implementation, the communication unit is further configured to acquire historical statements, target statements, connection identifiers of historical statements, and connection identifiers of target statements; the historical statements are used to represent multiple historical time periods of multiple historical quantity values; the target statements are used to represent the target date of the target quantity value; the processing unit is further configured to determine historical quantity values ​​from the target database based on the historical statements and connection identifiers of historical statements; the processing unit is further configured to determine target quantity values ​​from the target database based on the target statements and connection identifiers of target statements.

[0013] In conjunction with the second aspect above, in one possible implementation, the processing unit is further configured to: determine the historical quantity value of the target quantity within the target period; the target period includes the historical time period of the target quantity; based on the target time prediction algorithm, obtain a first prediction interval by fitting the historical quantity value of the target quantity within each target period; and calculate the target prediction interval by linear regression of multiple first prediction intervals.

[0014] In conjunction with the second aspect above, in one possible implementation, the processing unit is further configured to: determine that the data detection result is normal when the target quantity value is within the target prediction range; and determine that the data detection result is abnormal when the target quantity value is not within the target prediction range.

[0015] Thirdly, this application provides a data detection apparatus, which includes: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the data detection method as described in the first aspect and any possible implementation thereof.

[0016] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform the data detection method as described in the first aspect and any possible implementation thereof.

[0017] Fifthly, this application provides a computer program product containing instructions that, when run on a data detection device, cause the data detection device to perform the data detection method as described in the first aspect and any possible implementation thereof.

[0018] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the data detection method as described in the first aspect and any possible implementation thereof.

[0019] Specifically, the chip provided in this application also includes a memory for storing computer programs or instructions.

[0020] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the device, or it may be packaged separately from the processor of the device; this application does not impose any limitation on this.

[0021] In a seventh aspect, this application provides a data detection system, comprising: a data detection device and a data server, wherein the data detection device is used to perform the data detection method as described in the first aspect and any possible implementation thereof.

[0022] The descriptions of aspects two through seven in this application can be referenced to the detailed description of aspect one; and the beneficial effects of the descriptions of aspects two through seven can be referenced to the analysis of the beneficial effects of aspect one, which will not be repeated here.

[0023] In this application, the names of the aforementioned data detection devices do not limit the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those in this application, they fall within the scope of the claims of this application and their equivalents.

[0024] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the architecture of a data detection system provided in an embodiment of this application;

[0026] Figure 2 A flowchart illustrating a data detection method provided in an embodiment of this application;

[0027] Figure 3 A flowchart illustrating a data detection method provided in an embodiment of this application;

[0028] Figure 4 A flowchart illustrating another data detection method provided in this application embodiment;

[0029] Figure 5 This is a schematic diagram of the structure of a data detection device provided in an embodiment of this application;

[0030] Figure 6 This is a schematic diagram of another data detection device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0033] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0034] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0035] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0036] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0037] Data quality is crucial in the use of data; only with high-quality data can algorithms arrive at answers closer to the correct ones. One criterion for assessing data quality is completeness, which determines whether the data is comprehensive.

[0038] As holiday schedules change, so does the amount of data. Current technologies for monitoring the amount of data are inefficient at detecting anomalies.

[0039] Based on the above technical solution, the data detection method provided in this application first involves a data detection device acquiring multiple historical quantity values ​​and a target quantity value. The data detection device can then calculate a target prediction interval using a target time series algorithm and the multiple historical quantity values. Since the historical quantity values ​​represent the quantity of traffic data in each time period within multiple historical time periods, and the target quantity value represents the quantity of actual traffic data on the target date, compared to the inefficiency of existing technologies in detecting anomalies in data, the data detection device can quickly obtain data detection results by determining whether the target prediction interval includes the target quantity value, thus greatly improving the efficiency of data detection.

[0040] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0041] Figure 1This is an architecture diagram of a data detection system 10 provided in an embodiment of this application. Figure 1 As shown, the data detection system 10 includes a data detection device 101 and a data server 102.

[0042] The data detection device 101 and the data server 102 can be one or more, for ease of understanding. Figure 1 Only one is shown in the image.

[0043] The data detection device 101 and the data server 102 are connected via a communication link. This communication link can be a wired communication link or a wireless communication link, and this application does not limit it in this way.

[0044] The aforementioned data detection device 101 and data server 102 include:

[0045] The processor can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program in this application.

[0046] A transceiver can be any type of transceiver used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0047] Memory can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory can exist independently and be connected to the processor via communication lines. Memory can also be integrated with the processor.

[0048] One possible implementation is that the data detection device 101 obtains the target prediction interval by training multiple historical quantity values ​​provided by the data server 102. Then, the data detection device 101 obtains the data detection result by comparing the target quantity values ​​provided by the data server 102 with the target prediction interval.

[0049] For example, such as Figure 2 As shown, the data detection device 101 is applied to the Airflow scheduling system. The data detection device 101 obtains historical quantity values ​​from the Airflow scheduling system's metadata database using historical statements provided by the user and connection identifiers of those statements specified by the Airflow scheduling system's operators. The data detection device 101 also obtains target quantity values ​​from the Airflow scheduling system's metadata database using target statements provided by the user and connection identifiers of those statements specified by the Airflow scheduling system's operators. The data detection device 101 acquires a time-series algorithm determined by the user, inputs the algorithm into the Airflow scheduling system's operators to complete the configuration, and executes the algorithm to obtain the target prediction interval. The data detection device 101 then determines whether the target quantity value is within the target prediction interval. If the target quantity value is not within the target prediction interval, the data detection device 101 sends data quality information to the user through the Airflow scheduling system.

[0050] In one possible implementation, the data server 102 is used to provide the data detection device 101 with multiple historical quantity values ​​and target quantity values.

[0051] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.

[0052] Figure 3 This is a flowchart illustrating a data detection method provided in an embodiment of this application. Figure 3 As shown, the method includes the following steps:

[0053] S301, The data detection device acquires multiple historical quantity values ​​and target quantity values.

[0054] Among them, the historical quantity value is the quantity of traffic data in each time period within multiple historical time periods, and the target quantity value is the quantity of actual traffic data on the target date.

[0055] One possible implementation is that the data detection device obtains the quantity value of traffic data for each historical time period within multiple historical time periods from the target database of the scheduling system, as well as the quantity value of actual traffic data for the target date.

[0056] For example, taking any day before the target date as each historical time period, the data detection device obtains the quantity value of traffic data for each day before the target date, as well as the quantity value of actual traffic data on the target date, from the metadata database of the Airflow scheduling system.

[0057] For example, a list of historical count values ​​in the form of [(2022-08-25,12),(2022-08-26,13)] indicates that the historical count of traffic data on August 25, 2022 is 12, and the historical count of traffic data on August 26, 2022 is 13. A target count value in the form of [(34)] indicates that the target count of actual traffic data on the target date is 34.

[0058] One possible implementation is that the above S301 can be implemented by the following S3011-S3013.

[0059] S3011, The data detection device acquires historical statements, target statements, connection identifiers of historical statements, and connection identifiers of target statements.

[0060] Among them, the historical statement is used to represent multiple historical time periods of multiple historical quantity values, and the target statement is used to represent the target date of the target quantity value.

[0061] For example, the data detection device acquires historical statements and target statements provided by the user. The data detection device determines the connection identifier of the historical statement and the connection identifier of the target statement specified by the operator of the Airflow scheduling system.

[0062] For example, taking the historical query statement as `train_query_sql`, the target query statement as `target_query_sql`, and the connection identifier as the type and connection method of the target database, the data detection device acquires the `train_query_sql` and `target_query_sql` statements provided by the user. The data detection device determines the type and connection method of the target database corresponding to the `train_query_sql` statement specified by the Airflow scheduling system's operator, as well as the type and connection method of the target database corresponding to the `target_query_sql` statement.

[0063] For example, the specific algorithm for the train_query_sql statement is shown below.

[0064] train_query_sql="""

[0065] SELECT substar(log_time,1,10)As ds,

[0066] Count(user_id)As y

[0067] From db.table

[0068] Where log_time<'2020-03-1800:00:00'

[0069] GROUP BY substr(log_time,1,10)

[0070] """

[0071] S3012. The data detection device determines the historical quantity value from the target database based on the historical statements and the connection identifiers of the historical statements.

[0072] For example, taking the historical query statement as `train_query_sql`, the connection identifier of the historical query statement as the type and connection method of the target database corresponding to the `train_query_sql` statement, and the historical quantity value as the quantity of traffic data for each day before the target date, the data detection device retrieves the target database corresponding to the `train_query_sql` statement from the metadata database of the Airflow scheduling system based on the type of the target database. Furthermore, the data detection device retrieves the quantity of traffic data for each day before the target date from the target database based on the connection method of the target database.

[0073] S3013. The data detection device determines the target quantity value from the target database based on the target statement and the connection identifier of the target statement.

[0074] For example, taking the target statement as `target_query_sql`, the connection identifier of the target statement as the type and connection method of the target database corresponding to the `target_query_sql` statement, and the target quantity as the actual traffic data quantity for the target date, the data detection device retrieves the target database corresponding to the `target_query_sql` statement from the metadata database of the Airflow scheduling system based on the type of the target database. Furthermore, the data detection device retrieves the actual traffic data quantity for the target date from the target database based on the connection method of the target database.

[0075] S302, the data detection device obtains the target prediction interval through the target time series algorithm and historical data.

[0076] The target prediction interval is the prediction interval for the quantity value of the actual traffic data.

[0077] One possible implementation is that the data detection device uses a target time series algorithm to calculate multiple historical quantity values, thereby obtaining a predicted range for the quantity values ​​of the traffic data.

[0078] For example, consider the quantity values ​​of traffic data for each day prior to the target date, based on multiple historical values. The data detection device uses a target time series algorithm to calculate the quantity values ​​of traffic data for each day prior to the target date, thereby obtaining a predicted range for the quantity values ​​of the traffic data.

[0079] S303, The data detection device determines whether the target prediction interval includes the target quantity value and obtains the data detection result.

[0080] For example, the data detection device determines whether the target quantity value is within the target prediction range.

[0081] One possible implementation is that, if the target data is within the target prediction range, the data detection device determines that the data detection result is normal.

[0082] For example, if the target quantity value is within the target prediction range, the data detection device determines the data detection result to be normal. The data detection device sends the normal data detection result to the user via email or SMS.

[0083] Another possible implementation is that if the target data is not within the target prediction range, the data detection device determines that the data detection result is abnormal.

[0084] For example, if the target quantity value is outside the target prediction range, the data detection device determines the data detection result to be abnormal. The data detection device sends the abnormal data detection result to the user via email or SMS.

[0085] Based on the above technical solution, the data detection method provided in this application first involves a data detection device acquiring multiple historical quantity values ​​and a target quantity value. The data detection device can then calculate a target prediction interval using a target time series algorithm and the multiple historical quantity values. Since the historical quantity values ​​represent the quantity of traffic data in each time period within multiple historical time periods, and the target quantity value represents the quantity of actual traffic data on the target date, this method significantly improves data detection efficiency compared to the inefficiency of existing technologies in detecting anomalies. By determining whether the target prediction interval includes the target quantity value, the data detection device can quickly obtain the data detection result, greatly improving the efficiency of data detection.

[0086] As one possible embodiment of this application, combined with Figure 3 ,like Figure 4 As shown, the above S302 can also be implemented by the following S401-S403.

[0087] S401, The data detection device determines the historical quantity value of the target quantity within the target period.

[0088] The target period includes the historical time period of the target quantity.

[0089] For example, taking a target period of 7 days as an example, the data detection device determines the historical quantity values ​​for the 7 days prior to the target date.

[0090] S402, The data detection device obtains the first prediction interval by fitting the historical quantity value of the target within each target period based on the target time prediction algorithm.

[0091] For example, the data detection device obtains a first prediction interval by fitting historical quantity values ​​within each target period before the target date based on a target time prediction algorithm.

[0092] For example, the specific algorithm implementation of the target time prediction algorithm is shown below.

[0093] from fbprophet import prophet

[0094] m = prophet(

[0095] #growth='logistic', #Sets the trend term tree; the default is linear.

[0096] n_changepoints = 25. # Set the number of changepoints.

[0097] changepoint_range = 0.9, # Set the range of changepoint distribution

[0098] changepoint_prior_scale = 0.3, # Sets the distribution of the changepoint growth rate; a larger value provides greater flexibility.

[0099] weekly_seasonality = True, # Set the periodicity

[0100] holidays_prior_scale = 10.0 # Sets the impact of holidays; the default is 10. )

[0102] test_timeseries2=timeseriesDQOperator(task_id='test_timeseries2',

[0103] conn_id = 'impala_kcard'

[0104] train_query_sql=train_query_sql,

[0105] target_query_sql=target_query_sql,

[0106] prophet_model = m,

[0107] dag=dag)

[0108] For example, the specific algorithm implementation for setting the periodicity of the target time prediction algorithm is shown below.

[0109] m = add_seasonality(

[0110] name = 'weekly',

[0111] period = 7,

[0112] fourier_order = 3,

[0113] prior_scale = 0.1 )

[0115] S403, The data detection device obtains the target prediction interval by linear regression calculation of multiple first prediction intervals.

[0116] Understandably, since the multiple first prediction intervals are prediction intervals derived from the quantity values ​​of traffic data over multiple target periods prior to the target date, the data detection device can improve the accuracy of the obtained target prediction interval by performing linear regression calculations on the multiple first prediction intervals.

[0117] Based on the above technical solution, the data detection device determines the historical quantity values ​​of the target quantity within a target period and obtains a first prediction interval by fitting the historical quantity values ​​of the target quantity within each target period based on a target time prediction algorithm. The data detection device accurately calculates the target prediction interval by performing linear regression calculations on multiple first prediction intervals.

[0118] This application embodiment can divide the data detection device into functional modules or functional units according to the above method examples. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0119] like Figure 5 The diagram shown is a structural schematic of a data detection device 50 provided in an embodiment of this application. The device includes a communication unit 501 and a processing unit 502.

[0120] The communication unit 501 is used to acquire multiple historical quantity values ​​and a target quantity value; the historical quantity value is the quantity value of traffic data in each time period within multiple historical time periods; the target quantity value is the quantity value of actual traffic data on the target date; the processing unit 502 is used to obtain a target prediction interval through a target time series algorithm and historical quantity values; the target prediction interval is the prediction interval of the quantity value of traffic data; the processing unit 502 is also used to determine whether the target prediction interval includes the target quantity value and obtain the data detection result.

[0121] The communication unit 501 is further configured to acquire historical statements, target statements, connection identifiers of historical statements, and connection identifiers of target statements; the historical statements are used to represent multiple historical time periods of multiple historical quantity values; the target statements are used to represent the target date of the target quantity value; the processing unit 502 is further configured to determine historical quantity values ​​from the target database based on the historical statements and connection identifiers of historical statements; the processing unit 502 is further configured to determine target quantity values ​​from the target database based on the target statements and connection identifiers of target statements.

[0122] The processing unit 502 is further configured to determine the historical quantity value of the target quantity within the target period; the target period includes the historical time period of the target quantity; based on the target time prediction algorithm, a first prediction interval is obtained by fitting the historical quantity value of the target quantity within each target period; and the target prediction interval is obtained by calculating multiple first prediction intervals through linear regression.

[0123] The processing unit 502 is also used to determine that the data detection result is normal when the target quantity value is within the target prediction range, and to determine that the data detection result is abnormal when the target quantity value is not within the target prediction range.

[0124] In one possible implementation, the data detection device 50 may further include a storage unit 503. Figure 5 (shown in dashed box) The storage unit 503 stores a program or instruction. When the processing unit 502 executes the program or instruction, the data detection device 50 can perform the data detection method described in the above method embodiment.

[0125] When implemented in hardware, the communication unit 501 in this embodiment can be integrated onto the communication interface, and the processing unit 502 can be integrated onto the processor. Specific implementation methods are as follows: Figure 6 As shown.

[0126] Figure 6 A schematic diagram of another possible structure of the data detection device involved in the above embodiments is shown. The data detection device includes a processor 602 and a communication interface 601. The processor 602 is used to control and manage the operation of the data detection device, for example, executing the steps performed by the processing unit 502 described above, and / or performing other processes of the technology described herein. The communication interface 601 is used to support communication between the data detection device and other network entities, for example, executing the steps performed by the communication unit 501 described above. The data detection device may also include a memory 603 and a bus 604, the memory 603 being used to store the program code and data of the data detection device.

[0127] The memory 603 may be a memory in a data detection device, and the memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk or solid-state drive; the memory may also include a combination of the above types of memory.

[0128] The processor 602 described above can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0129] Bus 604 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 604 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0130] Figure 6 The data detection device in the middle can also be a chip. The chip includes one or more processors 602 and a communication interface 601.

[0131] In some embodiments, the chip further includes a memory 603, which may include read-only memory and random access memory, and provides operation instructions and data to the processor 602. A portion of the memory 603 may also include non-volatile random access memory (NVRAM).

[0132] In some implementations, memory 603 stores elements such as execution modules or data structures, or subsets thereof, or extended sets thereof.

[0133] In this embodiment of the application, the corresponding operation is executed by calling the operation instructions stored in the memory 603 (the operation instructions may be stored in the operating system).

[0134] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0135] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the data detection method described in the above method embodiments.

[0136] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the data detection method in the method flow shown in the above method embodiments.

[0137] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0138] Since the data detection device, computer-readable storage medium, and computer program product in the embodiments of this application can be applied to the above method, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0140] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0142] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data detection method, characterized in that, The method includes: The process involves obtaining historical statements, target statements, connection identifiers of the historical statements, and connection identifiers of the target statements. The historical statements represent multiple historical time periods for multiple historical quantity values. The target statements represent the target date for the target quantity value. Based on the historical statements and their connection identifiers, historical quantity values ​​are determined from the target database. Based on the target statements and their connection identifiers, the target quantity value is determined from the target database. The historical quantity value represents the quantity of traffic data for each time period within the multiple historical time periods. The target quantity value represents the quantity of actual traffic data for the target date. The quantity value represents the number of traffic entries used. Obtaining a target prediction interval using a target time series algorithm and the historical quantity values ​​includes: determining the historical quantity values ​​of the target quantity within a target period; the target period includes the historical time period of the target quantity; obtaining a first prediction interval by fitting the historical quantity values ​​of the target quantity within each target period based on the target time series prediction algorithm; calculating the target prediction interval by linear regression of multiple first prediction intervals; the target prediction interval is the prediction interval of the quantity values ​​of the actual traffic data. Determine whether the target prediction interval includes the target quantity value to obtain the data detection result.

2. The method according to claim 1, characterized in that, The step of determining whether the target prediction interval includes the target quantity value to obtain the data detection result includes: If the target quantity value falls within the target prediction range, the data detection result is determined to be normal. If the target quantity value is not within the target prediction range, the data detection result is determined to be abnormal.

3. A data detection device, characterized in that, The device includes: a communication unit and a processing unit; The communication unit is configured to acquire historical statements, target statements, connection identifiers of the historical statements, and connection identifiers of the target statements; the historical statements represent multiple historical time periods for multiple historical quantity values; the target statements represent the target date for the target quantity value; determine historical quantity values ​​from the target database based on the historical statements and their connection identifiers; determine the target quantity value from the target database based on the target statements and their connection identifiers; the historical quantity value is the quantity of traffic data for each time period within the multiple historical time periods; the target quantity value is the quantity of actual traffic data for the target date; and the quantity value is the number of traffic usage entries. The processing unit is configured to obtain a target prediction interval using a target time series algorithm and the historical quantity values, including: determining the historical quantity values ​​of the target quantity within a target period; the target period including the historical time period of the target quantity; obtaining a first prediction interval by fitting the historical quantity values ​​of the target quantity within each target period based on the target time series prediction algorithm; calculating the target prediction interval by linear regression of multiple first prediction intervals; the target prediction interval being the prediction interval for the quantity values ​​of the traffic data. The processing unit is also used to determine whether the target prediction interval includes the target quantity value, and to obtain the data detection result.

4. The apparatus according to claim 3, characterized in that, The processing unit is also used for: If the target quantity value falls within the target prediction range, the data detection result is determined to be normal. If the target quantity value is not within the target prediction range, the data detection result is determined to be abnormal.

5. A data detection device, characterized in that, include: A processor and a communication interface; the communication interface is coupled to the processor, the processor being used to run computer programs or instructions to implement the data detection method as described in any one of claims 1-2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the data detection method as described in any one of claims 1-2.

Images