A time data cleaning method and system
By establishing multiple data sets and exchanging and cleaning according to specific conditions, the analysis problem of complex time data is solved, and the rapid and accurate extraction of event time is achieved. It is suitable for data with many characteristics, large quantities, repeated, and disorderly conditions.
Patent Information
- Application Number
- CN202411262141.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2044-09-10
AI Technical Summary
Existing data analysis methods are difficult to effectively process time data with many features, large quantities, repeated, and disordered features, especially when analyzing the start and end times of events.
By establishing multiple data sets and cleaning the data according to specific conditions and thresholds, including exchange and clearing of the first data set, the second data set, the third data set and the fourth data set, pointing to different data sets with pointers, combining time range query and data supplementation, the start time and end time of the event are determined.
It realizes rapid and accurate analysis of complex time data, and can accurately extract the start and end time of events from a large number of repeated and disordered data, which is suitable for data processing across the New Year.
Smart Images
Figure CN119201920B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to data cleaning technology, in particular to a time data cleaning method and system. Background Art
[0002] In the era of digitalization and intelligence, a large amount of data is generated every day. There are many types of data. How to obtain the required data from a large amount of data and analyze the obtained data to draw relevant conclusions is an urgent problem that needs to be solved in data analysis. Analyzing the data to get the start time and end time of an event is a particularly core problem. The current analysis methods are only applicable to simple data, and are not very suitable for data with many features, large volume, repetition, and disorder. Therefore, the above problems need to be solved urgently. Summary of the invention
[0003] Purpose of the invention: The purpose of the present invention is to provide a time data cleaning method and system, aiming to solve the problem of how to analyze the start time and end time of an event from a large number of repeated, disordered and messy data.
[0004] Technical solution: In the time data cleaning method of the present invention, the event data to be cleaned includes the secondary event name and time, and the start time and end time of the time are obtained through data cleaning;
[0005] The data to be cleaned is stored in the original data set to establish a first data set, a second data set, a third data set, and a fourth data set; the length of the first data set and the second data set is the number of secondary events, and the third data set and the fourth data set are used to record the start time and the end time in the first data set and the second data set respectively;
[0006] The time data cleaning method comprises the following steps:
[0007] (1) Obtain a piece of data A from the original data set. If the first condition is met, A is put into the first data set, the third data set is updated and step (4) is executed; if the first condition is not met, step (2) is executed;
[0008] The first condition is that there is no data in the first data set or the time interval between the time in A and the start time in the third data set exceeds a threshold;
[0009] (2) If the second condition is met, the data in the third data set is stored in the result data set, the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged. At this time, the second data set and the fourth data set are cleared; if the third condition is met, A is placed in the first data set, the third data set is updated and step (4) is executed; if the third condition is not met, A is placed in the second data set, the fourth data set is updated and step (4) is executed; if the second condition is not met, step (3) is executed;
[0010] The second condition is that the length of the data in the first data set is equal to the number of secondary events;
[0011] The third condition is that there is no data in the first data set or the minimum time interval between the time in A and the time in the third data set exceeds a threshold;
[0012] The result dataset is used to store the start time and end time of the event;
[0013] (3) If the fourth condition is satisfied, determine whether the fifth condition is satisfied. If the fifth condition is satisfied, put A into the second data set, update the fourth data set and execute step (4). If the fifth condition is not satisfied, indicating that the data in the first data set is incomplete, query and supplement data B according to the time range of the event, store B into the result data set, clear the first data set and the third data set, exchange the data in the first data set and the second data set, and exchange the data in the third data set and the fourth data set. At this time, the second data set and the fourth data set are cleared. If the third condition is satisfied, put A into the first data set, update the third data set and execute step (4). If the third condition is not satisfied, put A into the second data set, update the fourth data set and execute step (4).
[0014] The fourth condition is that the length of the data in the first data set is less than the number of secondary events;
[0015] The fifth condition is that there is no data in the second data set or the minimum time interval between the time in A and the time in the fourth data set exceeds a threshold;
[0016] (4) If the original data set is not empty, execute step (1) until the original data set is empty and output the result data set.
[0017] Furthermore, the step (3) of querying and supplementing data B according to the time range of the event includes:
[0018] Determine whether the data in the third data set is within the first range. If so, search the database for data B within the first range and add it to the first data set and update the third data set, and store the data in the third data set in the result data set; if not, directly store the data in the third data set in the result data set.
[0019] Furthermore, in the first data set or the second data set, when the secondary event name of the stored data already exists in the data set, the third data set or the fourth data set is updated according to the existing data, and then the existing data is overwritten with the newly stored data, and the third data set or the fourth data set is updated.
[0020] Furthermore, the method for updating the third data set and the method for updating the fourth data set is: if there is no data in the third data set or the fourth data set, the time in A is directly used as the start time and end time of the third data set or the fourth data set, and the start time is equal to the end time; if there is data in the third data set or the fourth data set, the time in A is compared with the time in the third data set or the fourth data set, and the minimum value is taken as the start time of the third data set or the fourth data set, and the maximum value is taken as the end time of the third data set or the fourth data set.
[0021] Furthermore, step (4) further includes: if the first data set is not empty, storing the data in the third data set into the result data set; if the second data set is not empty, storing the data in the fourth data set into the result data set.
[0022] Furthermore, the pointer dataHashMapOne points to the first data set, the pointer dataHashMapTwo points to the second data set, the pointer oneData points to the third data set, and the pointer twoData points to the fourth data set; in step (1) and step (3), the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged. At this time, the second data set and the fourth data set are cleared including:
[0023] Clear the data sets pointed to by dataHashMapOne and oneData;
[0024] Re-point dataHashMapOne to the data set pointed to by dataHashMapTwo, and re-point oneData to the data set pointed to by twoData;
[0025] dataHashMapTwo points to the previous data set of dataHashMapOne, and twoData points to the previous data set pointed to by oneData;
[0026] Complete the clearing and exchange of data.
[0027] Furthermore, the original data set is an ordered data set, and the data to be cleaned is arranged according to the two dimensions of secondary event number and time to obtain an ordered original data set.
[0028] In the time data cleaning system of the present invention, the data to be cleaned of the event includes the secondary event name and time, and the system is used to obtain the start time and end time of the time through data cleaning, and the system includes:
[0029] A data set establishment unit is used to store the data to be cleaned into an original data set, and establish a first data set, a second data set, a third data set, and a fourth data set; the length of the first data set and the second data set is the number of secondary events, and the third data set and the fourth data set are used to record the start time and the end time in the first data set and the second data set respectively;
[0030] A first data cleaning unit is used to obtain a piece of data A from the original data set, and if the first condition is met, A is put into the first data set, the third data set is updated and the fourth data cleaning unit is executed; if the first condition is not met, the second data cleaning unit is executed;
[0031] The first condition is that there is no data in the first data set or the time interval between the time in A and the start time in the third data set exceeds a threshold;
[0032] The second data cleaning unit is used for storing the data in the third data set into the result data set when the second condition is met, clearing the first data set and the third data set, exchanging the data in the first data set and the second data set, and exchanging the data in the third data set and the fourth data set, at which time the second data set and the fourth data set are cleared; when the third condition is met, putting A into the first data set, updating the third data set and executing the fourth data cleaning unit; if the third condition is not met, putting A into the second data set, updating the fourth data set and executing the fourth data cleaning unit; if the second condition is not met, executing the third data cleaning unit;
[0033] The second condition is that the length of the data in the first data set is equal to the number of secondary events;
[0034] The third condition is that there is no data in the first data set or the minimum time interval between the time in A and the time in the third data set exceeds a threshold;
[0035] The result dataset is used to store the start time and end time of the event;
[0036] A third data cleaning unit is used to determine whether the fifth condition is met when the fourth condition is met. If the fifth condition is met, A is put into the second data set, the fourth data set is updated, and the fourth data cleaning unit is executed; if the fifth condition is not met, it means that the data in the first data set is incomplete, then the data B is queried and supplemented according to the time range of the event, B is stored in the result data set, the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged, and the second data set and the fourth data set are cleared at this time; if the third condition is met, A is put into the first data set, the third data set is updated, and the fourth data cleaning unit is executed; if the third condition is not met, A is put into the second data set, the fourth data set is updated, and the fourth data cleaning unit is executed;
[0037] The fourth condition is that the length of the data in the first data set is less than the number of secondary events;
[0038] The fifth condition is that there is no data in the second data set or the minimum time interval between the time in A and the time in the fourth data set exceeds a threshold;
[0039] The fourth data cleaning unit is used to execute the first data cleaning unit when the original data set is not empty, until the original data set is an empty set, and output a result data set.
[0040] The electronic device of the present invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the time data cleaning method is implemented when the computer program is loaded into the processor.
[0041] The computer-readable storage medium of the present invention stores a computer program, and the computer program implements the time data cleaning method when executed by a processor.
[0042] Beneficial effects: Compared with the prior art, the advantages of the present invention are: (1) the present invention can accurately and quickly process data that is multi-featured, large in volume, repetitive, and disordered; (2) the present invention can accurately and quickly process data that spans multiple years; (3) the present invention can accurately and quickly analyze a large amount of repetitive, disordered data to derive the start time and end time of each event within a period of time. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A simple schematic diagram of the method of the present invention;
[0044] Figure 2 The following is a detailed flow chart of the method of the present invention. DETAILED DESCRIPTION
[0045] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0046] This embodiment is described by taking train inspection as an example. It should be pointed out that the train inspection here is only one usage scenario of the present method, and the present method can also be used in other scenarios.
[0047] In this embodiment, the event is a train inspection. For example, a train number is JC006. The train has multiple carriages, one of which is named JC006-TC. Each carriage has 4 axles. The 4 axles of carriage JC006-TC are named JC006-TC-1, JC006-TC-2, JC006-TC-3, and JC006-TC-4. When inspecting a train, the name of each axle of the train carriage (such as JC006-TC-1) and the time of inspection (such as 2024-07-16 09:30:30) are mainly recorded. In the case of many trains, each axle of each carriage of each train is inspected, and information such as the axle name, inspection time, and inspection results are recorded. When the train number and year are entered, the start time and end time of each inspection of the train within a year can be found. For inquiries at the beginning of the year, you can look back 10 days. In this embodiment, the data to be cleaned is the inspection data of all axles of the train.
[0048] In order to obtain the start and end time of each inspection of the train within a period of time, Figure 1 The figure shows a simple schematic diagram of the method of the present invention. All the inspected data are filtered out by train number (such as JC006) and inspection year (such as 2024), and arranged in order by inspection time and carriage axle name. An arranged data set is obtained, which is set as originDataList. There is a large amount of repeated data in the data set.
[0049] Furthermore, get the number of all axes of a train, set as axisNum, whose size is the number of carriages * 4.
[0050] Furthermore, two data sets are set up and pointed to by pointers named dataHashMapOne and dataHashMapTwo. Both data sets have a fixed length, which is equal to axisNum. The inspection information of the train axles is stored in key-value format. The key is the name of the axis and the value is the inspection time and other information of the axis.
[0051] Furthermore, two data sets are set up, and pointers named oneData and twoData are used to point to them, which correspond to dataHashMapOne and dataHashMapTwo respectively. OneData records the start time and end time of the axis check time in dataHashMapOne, and twoData records the start time and end time of the axis check time in dataHashMapTwo.
[0052] The data sets specified by dataHashMapOne and dataHashMapTwo will be cleared and swapped during the calculation process. The data sets specified by oneData and twoData will be cleared and swapped during the calculation process. And the resulting data (including train number, start time of inspection, and end time of inspection) will be stored in dataList.
[0053] In addition, since the axle of the train may be inspected multiple times within a period of time, the data sets represented by dataHashMapOne and dataHashMapTwo can be overwritten when the axle names are the same. For example, axle JC006-TC-4 was inspected twice at 09:30:30 and 09:31:30 on 2024-07-16. The data at 09:30:30 is first stored in dataHashMapOne, and the data set pointed to by oneData is analyzed and updated. The data at 09:31:30 is then used to overwrite the data at 09:30:30 in dataHashMapOne, and analysis is performed again. At this time, the amount of data in dataHashMapOne remains unchanged, but the start and end times corresponding to oneData may change.
[0054] like Figure 2 Shown is a detailed process flow chart of the method of the present invention, which includes the following steps.
[0055] S201. Get a piece of data A from originDataList, where the data A includes information such as axis name, inspection time, and inspection result.
[0056] S202. Determine whether the data set pointed to by dataHashMapOne has no data, or whether the time in A minus the start time in oneData is less than or equal to 10 days. If the data set pointed to by dataHashMapOne has no data, it means that it has just started. If the time in A differs from the start time in oneData by 10 days, it means that they are the same batch of data. If one of the conditions is met, go to S203. If both conditions are not met, go to S206.
[0057] S203. Put data A into the data set pointed to by dataHashMapOne.
[0058] S204. The time in A is compared with the time in oneData, and the start time and end time in oneData are updated. The basis for the update is: if oneData has no data, the time in A is directly given to the start time and end time in oneData, and the start time and end time are the same; if oneData has data, the time in A is compared with the end time of oneData, and the larger value is assigned to the end time in oneData, and the time in A is compared with the start time of oneData, and the smaller value is assigned to the start time in oneData.
[0059] S205. Determine whether there is any data in originDataList. If so, the loop is not ended and go to S201; otherwise, go to S218.
[0060] S206. This indicates that there is data in dataHashMapOne, and the data in A is greater than 10 days compared with the start time of oneData, which means that A is another batch of inspection data. If the data volume of dataHashMapOne = axisNum at this time, enter S207, otherwise enter S214.
[0061] S207. This indicates that the train has completed an inspection, and the data in oneData is put into dataList, indicating that a piece of available data has been generated. dataList stores the train number of this inspection and the start time (start time in oneData) and end time (end time in oneData) of the inspection.
[0062] S208. Clear the data sets pointed to by dataHashMapOne and oneData.
[0063] S209. Re-point dataHashMapOne to the data set pointed to by dataHashMapTwo, and re-point oneData to the data set pointed to by twoData.
[0064] S210.dataHashMapTwo points to the previous data set of dataHashMapOne, and twoData points to the data set pointed to by the previous oneData.
[0065] Steps S208 to S210 complete the clearing and swapping of data. At this time, dataHashMapTwo and twoData represent the cleared data areas, and they no longer have data. The data sets pointed to by dataHashMapOne and oneData will be used to store new data. At this time, data A has not been stored yet, and data A needs to be processed.
[0066] S211. Determine whether the data set pointed to by dataHashMapOne has no data, or the time in A minus the start time in oneData <= 10 days. If the data set pointed to by dataHashMapOne has no data, it indicates the start of a new batch of inspection data. Or if the time difference between the time in A and the start time in oneData is 10 days, it indicates the same batch of data. If either condition is met, go to S203; otherwise, go to S212.
[0067] S212. Put data A into the data set pointed to by dataHashMapTwo.
[0068] S213. Compare the time in A with the time in twoData, and update the start time and end time in twoData. The basis for the update is as follows: If twoData has no data, directly assign the time in A to the start time and end time in twoData, and at this time the start time and end time are the same; if there is data in twoData, compare the time in A with the end time in twoData, and assign the larger value to the end time in twoData, compare the time in A with the start time in twoData, and assign the smaller value to the start time in twoData; after this step, the processing of data A is completed, go to S205 for the judgment of the next loop.
[0069] S214. If the data volume of dataHashMapOne < axisNum, go to S215; otherwise, go to S205.
[0070] S215. If there is no data in dataHashMapTwo, or the time in A minus the start time in twoData <= 10 days, it indicates the same batch of data, go to S212; otherwise, it indicates that the data pointed to by dataHashMapOne is incomplete and a complete inspection of the train has not been completed. At this time, it is necessary to judge whether it is the data at the beginning of the year in oneData for processing, and go to S216.
[0071] S216. Determine whether oneData is the beginning of the year data, the basis for the determination is: check whether the start time in oneData is between January 1 and January 10, if yes, it is the beginning of the year data, otherwise it is not the beginning of the year data. If oneData is the beginning of the year data, go to S217, otherwise go to S207.
[0072] S217. oneData is the data at the beginning of the year. The data between January 1 and January 10 of the year represented by oneData is queried from the database, and then all these data are put into dataHashMapOne, and the start time and end time in oneData are updated. The update rules are the same as step S204, and then go to step S207 to complete the data acquisition. If oneData is not the data at the beginning of the year, go directly to step S207.
[0073] The data in S218.originDataList has been processed, and the start time and settlement time of each train inspection are recorded in dataList. However, there may be data in the data sets pointed to by dataHashMapOne and dataHashMapTwo, and the existing data needs to be put into dataList. Therefore, this step determines whether dataHashMapOne is empty. If it is not empty, go to S219, otherwise go to S220.
[0074] S219. Put the data in oneData into dataList, indicating a piece of available data. The whole process ends here.
[0075] S220. Determine whether dataHashMapTwo is empty. If it is not empty, go to S221, otherwise the whole process ends here.
[0076] S221. Put the data in twoData into dataList, indicating a piece of available data. The whole process ends here.
[0077] Through the above steps, dataList stores the start time and end time of each train inspection within a period of time.
[0078] In the time data cleaning system of the present invention, the data to be cleaned of the event includes the secondary event name and time, and the system is used to obtain the start time and end time of the time through data cleaning, and the system includes:
[0079] A data set establishment unit is used to store the data to be cleaned into an original data set, and establish a first data set, a second data set, a third data set, and a fourth data set; the length of the first data set and the second data set is the number of secondary events, and the third data set and the fourth data set are used to record the start time and the end time in the first data set and the second data set respectively;
[0080] A first data cleaning unit is used to obtain a piece of data A from the original data set, and if the first condition is met, A is put into the first data set, the third data set is updated and the fourth data cleaning unit is executed; if the first condition is not met, the second data cleaning unit is executed;
[0081] The first condition is that there is no data in the first data set or the time interval between the time in A and the start time in the third data set exceeds a threshold;
[0082] The second data cleaning unit is used for storing the data in the third data set into the result data set when the second condition is met, clearing the first data set and the third data set, exchanging the data in the first data set and the second data set, and exchanging the data in the third data set and the fourth data set, at which time the second data set and the fourth data set are cleared; when the third condition is met, putting A into the first data set, updating the third data set and executing the fourth data cleaning unit; if the third condition is not met, putting A into the second data set, updating the fourth data set and executing the fourth data cleaning unit; if the second condition is not met, executing the third data cleaning unit;
[0083] The second condition is that the length of the data in the first data set is equal to the number of secondary events;
[0084] The third condition is that there is no data in the first data set or the minimum time interval between the time in A and the time in the third data set exceeds a threshold;
[0085] The result dataset is used to store the start time and end time of the event;
[0086] A third data cleaning unit is used to determine whether the fifth condition is met when the fourth condition is met. If the fifth condition is met, A is put into the second data set, the fourth data set is updated, and the fourth data cleaning unit is executed; if the fifth condition is not met, it means that the data in the first data set is incomplete, then the data B is queried and supplemented according to the time range of the event, B is stored in the result data set, the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged, and the second data set and the fourth data set are cleared at this time; if the third condition is met, A is put into the first data set, the third data set is updated, and the fourth data cleaning unit is executed; if the third condition is not met, A is put into the second data set, the fourth data set is updated, and the fourth data cleaning unit is executed;
[0087] The fourth condition is that the length of the data in the first data set is less than the number of secondary events;
[0088] The fifth condition is that there is no data in the second data set or the minimum time interval between the time in A and the time in the fourth data set exceeds a threshold;
[0089] The fourth data cleaning unit is used to execute the first data cleaning unit when the original data set is not empty, until the original data set is an empty set, and output a result data set.
[0090] The electronic device of the present invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the time data cleaning method is implemented when the computer program is loaded into the processor.
[0091] The computer-readable storage medium of the present invention stores a computer program, and the computer program implements the time data cleaning method when executed by a processor.
[0092] The computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer.
[0093] The processor is used to execute the computer program stored in the memory to implement each step of the method involved in the above embodiment.
Claims
1. A time data cleaning method, characterized in that: The data to be cleaned of the event includes the secondary event name and time, and the start time and end time of the time are obtained through data cleaning; The data to be cleaned is stored in the original data set to establish a first data set, a second data set, a third data set, and a fourth data set; the length of the first data set and the second data set is the number of secondary events, and the third data set and the fourth data set are used to record the start time and the end time in the first data set and the second data set respectively; The time data cleaning method comprises the following steps: (1) Obtain a piece of data A from the original data set. If the first condition is met, A is put into the first data set, the third data set is updated and step (4) is executed; if the first condition is not met, step (2) is executed; The first condition is that there is no data in the first data set or the time interval between the time in A and the start time in the third data set exceeds a threshold; (2) If the second condition is met, the data in the third data set is stored in the result data set, the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged, at which point the second data set and the fourth data set are cleared; If the third condition is met, A is put into the first data set, the third data set is updated and step (4) is executed; if the third condition is not met, A is put into the second data set, the fourth data set is updated and step (4) is executed; if the second condition is not met, step (3) is executed; The second condition is that the length of the data in the first data set is equal to the number of secondary events; The third condition is that there is no data in the first data set or the minimum time interval between the time in A and the time in the third data set exceeds a threshold; The result dataset is used to store the start time and end time of the event; (3) If the fourth condition is satisfied, determine whether the fifth condition is satisfied. If the fifth condition is satisfied, put A into the second data set, update the fourth data set and execute step (4). If the fifth condition is not satisfied, indicating that the data in the first data set is incomplete, query and supplement data B according to the time range of the event, store B in the result data set, clear the first data set and the third data set, exchange the data in the first data set and the second data set, and exchange the data in the third data set and the fourth data set. At this time, the second data set and the fourth data set are cleared. If the third condition is met, A is put into the first data set, the third data set is updated and step (4) is executed; if the third condition is not met, A is put into the second data set, the fourth data set is updated and step (4) is executed; The fourth condition is that the length of the data in the first data set is less than the number of secondary events; The fifth condition is that there is no data in the second data set or the minimum time interval between the time in A and the time in the fourth data set exceeds a threshold; (4) If the original data set is not empty, execute step (1) until the original data set is empty and output the result data set.
2. The time data cleaning method according to claim 1, characterized in that: In step (3), querying and supplementing data B according to the time range of the event includes: Determine whether the data in the third data set is within the first range. If so, search the database for data B within the first range and add it to the first data set and update the third data set, and store the data in the third data set in the result data set; if not, directly store the data in the third data set in the result data set.
3. The time data cleaning method according to claim 1, characterized in that: In the first data set or the second data set, when the secondary event name of the stored data already exists in the data set, the third data set or the fourth data set is updated according to the existing data, and then the existing data is overwritten with the new stored data, and the third data set or the fourth data set is updated.
4. The time data cleaning method according to claim 1, characterized in that: The method for updating the third data set and the method for updating the fourth data set is: if there is no data in the third data set or the fourth data set, the time in A is directly used as the start time and end time of the third data set or the fourth data set, and the start time is equal to the end time; if there is data in the third data set or the fourth data set, the time in A is compared with the time in the third data set or the fourth data set, and the minimum value is taken as the start time of the third data set or the fourth data set, and the maximum value is taken as the end time of the third data set or the fourth data set.
5. The time data cleaning method according to claim 1, characterized in that: Step (4) also includes: if the first data set is not empty, storing the data in the third data set into the result data set; if the second data set is not empty, storing the data in the fourth data set into the result data set.
6. The time data cleaning method according to claim 1, characterized in that: The pointer dataHashMapOne points to the first data set, the pointer dataHashMapTwo points to the second data set, the pointer oneData points to the third data set, and the pointer twoData points to the fourth data set; in step (1) and step (3), the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged. At this time, the second data set and the fourth data set are cleared including: Clear the data sets pointed to by dataHashMapOne and oneData; Re-point dataHashMapOne to the data set pointed to by dataHashMapTwo, and re-point oneData to the data set pointed to by twoData; dataHashMapTwo points to the previous data set of dataHashMapOne, and twoData points to the previous data set pointed to by oneData; Complete the clearing and exchange of data.
7. The time data cleaning method according to claim 1, characterized in that: The original data set is an ordered data set, and the data to be cleaned are arranged according to the two dimensions of secondary event number and time to obtain an ordered original data set.
8. A time data cleaning system, characterized in that: The data to be cleaned of the event includes the secondary event name and time. The system is used to obtain the start time and end time of the time through data cleaning. The system includes: A data set establishment unit is used to store the data to be cleaned into an original data set, and establish a first data set, a second data set, a third data set, and a fourth data set; the length of the first data set and the second data set is the number of secondary events, and the third data set and the fourth data set are used to record the start time and the end time in the first data set and the second data set respectively; A first data cleaning unit is used to obtain a piece of data A from the original data set, and if the first condition is met, A is put into the first data set, the third data set is updated and the fourth data cleaning unit is executed; if the first condition is not met, the second data cleaning unit is executed; The first condition is that there is no data in the first data set or the time interval between the time in A and the start time in the third data set exceeds a threshold; The second data cleaning unit is used for storing the data in the third data set into the result data set when the second condition is met, clearing the first data set and the third data set, exchanging the data in the first data set and the second data set, and exchanging the data in the third data set and the fourth data set, at which time the second data set and the fourth data set are cleared; when the third condition is met, putting A into the first data set, updating the third data set and executing the fourth data cleaning unit; if the third condition is not met, putting A into the second data set, updating the fourth data set and executing the fourth data cleaning unit; if the second condition is not met, executing the third data cleaning unit; The second condition is that the length of the data in the first data set is equal to the number of secondary events; The third condition is that there is no data in the first data set or the minimum time interval between the time in A and the time in the third data set exceeds a threshold; The result dataset is used to store the start time and end time of the event; A third data cleaning unit is used to determine whether the fifth condition is met when the fourth condition is met. If the fifth condition is met, A is put into the second data set, the fourth data set is updated, and the fourth data cleaning unit is executed; if the fifth condition is not met, it means that the data in the first data set is incomplete, then the data B is queried and supplemented according to the time range of the event, B is stored in the result data set, the first data set and the third data set are cleared, the data in the first data set and the second data set are exchanged, and the data in the third data set and the fourth data set are exchanged, and the second data set and the fourth data set are cleared at this time; if the third condition is met, A is put into the first data set, the third data set is updated, and the fourth data cleaning unit is executed; if the third condition is not met, A is put into the second data set, the fourth data set is updated, and the fourth data cleaning unit is executed; The fourth condition is that the length of the data in the first data set is less than the number of secondary events; The fifth condition is that there is no data in the second data set or the minimum time interval between the time in A and the time in the fourth data set exceeds a threshold; The fourth data cleaning unit is used to execute the first data cleaning unit when the original data set is not empty, until the original data set is an empty set, and output a result data set.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the time data cleaning method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the time data cleaning method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Time slice-based method and device for event statistics
CN102340416A
Mass spatio-temporal data cleaning method and mass spatio-temporal data cleaning device
CN106570160A