Data processing method, system and storage medium based on file cyclic renaming
By using file loop rename method in vehicle-mounted equipment to exchange and decouple concurrent processing data, the problems of low data processing efficiency and poor memory queue integrity are solved, efficient data transmission and storage are achieved, processing flow is simplified and cost is reduced.
Patent Information
- Application Number
- CN202210727361.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-25
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-06-25
AI Technical Summary
When existing vehicle-mounted equipment is data processing and remote communication storage, there are problems such as high traffic consumption, low data processing efficiency, and poor memory queue data integrity and controllability.
Using a data processing method based on file cyclic rename, the decoupled concurrent processing data is exchanged by performing cyclic renames of multiple files, which is used for data reception, reading, storing and reporting, which simplifies the processing flow and improves data transmission and storage efficiency. .
It improves data processing efficiency, enhances the integrity and controllability of memory queues, simplifies processing and error handling processes, and saves on-board equipment space and reduces costs.
Smart Images

Figure CN115061983B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of communication and data storage, and in particular to a data processing method, device, system, equipment and storage medium based on file cyclic renaming. Background Art
[0002] The current methods for concurrent processing such as reading, writing, and reporting of collected data by on-board equipment usually include: processing with a small device database, processing with the combined use of device memory and file storage, and the device calling a remote cloud interface to store and process the data in a cloud database.
[0003] However, database processing and remote communication storage have the following main defects: large traffic consumption and relatively slow data processing efficiency. The main reason for the above defects is that the storage and query efficiency of the database is relatively slow, and direct reporting and processing through communication requires frequent sending and receiving of messages with the cloud, which has relatively slow processing efficiency. At the same time, the integrity and controllability of the memory queue data are poor. Summary of the invention
[0004] Therefore, in order to overcome the above-mentioned defects, an embodiment of the present invention provides a data processing method, apparatus, system, equipment and storage medium based on file cyclic renaming, which decouples concurrent data processing through file cyclic renaming exchange, and is used for data receiving, reading, storing, backing up and reporting to a remote server. It has the advantages of high processing efficiency and improved integrity and controllability of memory queue data.
[0005] To this end, a data processing method based on file cyclic renaming according to an embodiment of the present invention comprises the following steps:
[0006] Read a data record from the data queue;
[0007] Circularly renaming the first file, the second file, the third file and the fourth file to write the data record into the first file;
[0008] Determine whether the fourth file exists;
[0009] When the fourth file exists, continue to read a data record from the data queue;
[0010] When the fourth file does not exist, the first file is renamed as the fourth file and then a data record is continuously read from the data queue.
[0011] Preferably, the step of cyclically renaming the first file, the second file, the third file and the fourth file to write the data record into the first file comprises:
[0012] Determine whether the first file exists;
[0013] When the first file exists, writing the data record into the first file;
[0014] When the first file does not exist, determining whether the second file exists;
[0015] When the second file exists, renaming the second file to the first file and then writing the data record into the first file;
[0016] When the second file does not exist, determining whether the third file exists;
[0017] When the third file exists, renaming the third file to the first file and then writing the data record into the first file;
[0018] When the third file does not exist, the data record is written into the first file after the first file is newly created.
[0019] Preferably, the method further comprises the following steps:
[0020] Determine whether the fourth file exists;
[0021] When the fourth file exists, continuously reading and outputting data records from the fourth file until all data records are read and outputted;
[0022] Circularly renaming the second file, the third file and the fourth file to rename the fourth file to the second file or the third file, and then continue to determine whether the fourth file exists;
[0023] When the fourth file does not exist, the process sleeps for a while, and then continues to determine whether the fourth file exists.
[0024] Preferably, when the fourth file exists, the step of continuously reading and outputting data records from the fourth file until all data records are read and outputted includes:
[0025] When the fourth file exists, data records are continuously read and output one by one from the fourth file until all data records are read and outputted.
[0026] Preferably, the step of cyclically renaming the second file, the third file and the fourth file to rename the fourth file to the second file or the third file includes:
[0027] Determine whether the second file exists;
[0028] When the second file exists, determining whether the third file exists;
[0029] When the third file exists, rename the fourth file to the second file;
[0030] When the third file does not exist, renaming the fourth file to the third file;
[0031] When the second file does not exist, the fourth file is renamed as the second file.
[0032] A data processing device based on cyclic renaming according to an embodiment of the present invention includes:
[0033] A data record reading unit, used for reading a data record from a data queue;
[0034] a first cyclic renaming unit, used for cyclically renaming the first file, the second file, the third file and the fourth file, so as to write the data record into the first file;
[0035] A first judging unit, used for judging whether a fourth file exists;
[0036] A repeating unit, used for continuing to read a data record from the data queue when the fourth file exists;
[0037] The renaming repetition unit is used to, when the fourth file does not exist, rename the first file to the fourth file and then continue to read a data record from the data queue.
[0038] Preferably, the first cyclic renaming unit comprises:
[0039] A second judging unit, used to judge whether the first file exists;
[0040] A writing unit, used for writing the data record into the first file when the first file exists;
[0041] A third judgment unit, used for judging whether the second file exists when the first file does not exist;
[0042] A first renaming and writing unit, used for, when the second file exists, renaming the second file to the first file and then writing the data record into the first file;
[0043] A fourth judgment unit, used for judging whether the third file exists when the second file does not exist;
[0044] A second renaming and writing unit is used for, when the third file exists, renaming the third file to the first file and then writing the data record into the first file;
[0045] A new writing unit is used to write the data record into the first file after creating the first file when the third file does not exist.
[0046] Preferably, it also includes:
[0047] A fifth determination unit, used for determining whether a fourth file exists;
[0048] a data record reading unit, used for continuously reading and outputting data records from the fourth file when the fourth file exists until all data records are read and outputted;
[0049] A second cyclic renaming unit, used for cyclically renaming the second file, the third file and the fourth file, so as to rename the fourth file to the second file or the third file, and then continue to determine whether the fourth file exists;
[0050] The dormancy waiting unit is used for dormancy for a period of time when the fourth file does not exist, and then continues to determine whether the fourth file exists.
[0051] Preferably, the data record reading unit comprises:
[0052] The continuous single-line reading unit is used to continuously read and output data records one by one from the fourth file when the fourth file exists until all data records are read and output.
[0053] Preferably, the second cyclic renaming unit comprises:
[0054] A sixth determination unit, used to determine whether the second file exists;
[0055] A seventh judgment unit, used for judging whether the third file exists when the second file exists;
[0056] A first renaming unit, used for renaming the fourth file to the second file when the third file exists;
[0057] a second renaming unit, configured to rename the fourth file to the third file when the third file does not exist;
[0058] The third renaming unit is used to rename the fourth file into the second file when the second file does not exist.
[0059] An in-vehicle device according to an embodiment of the present invention includes:
[0060] one or more processors; and
[0061] A storage device is used to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned data processing method based on cyclic renaming.
[0062] A data processing system based on file cyclic renaming according to an embodiment of the present invention includes:
[0063] The above-mentioned vehicle-mounted equipment; and
[0064] The storage device is used to receive the data records output by the vehicle-mounted device for storage or backup, and / or output the data queue to the vehicle-mounted device.
[0065] Preferably, it also includes:
[0066] The remote server is used to receive data records reported by the vehicle-mounted device and / or output data queues to the vehicle-mounted device.
[0067] A computer-readable storage medium according to an embodiment of the present invention stores instructions, and when the instructions are executed by a processor, the data processing method based on cyclic renaming is implemented.
[0068] The data processing method, device, system, equipment and storage medium based on file cyclic renaming according to the embodiments of the present invention have the following advantages:
[0069] 1. Decoupling concurrent data processing through file loop renaming and exchange reduces technical complexity, is simple and convenient, and only requires simple text processing, thereby improving transmission and storage efficiency and helping to maintain data integrity, consistency and reusability.
[0070] 2. By storing, backing up and reporting to the remote server, it saves space on the vehicle equipment and reduces costs.
[0071] 3. The data storage method using file cyclic renaming is highly readable and does not require complex and tedious data queries.
[0072] 4. Data processing based on file cyclic renaming simplifies the processing flow and also simplifies the troubleshooting process. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] In order to more clearly illustrate the technical solutions in the specific implementation modes of the present invention, the drawings required for use in the description of the specific implementation modes will be briefly introduced below. Obviously, the drawings described below are some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0074] Figure 1 is a flowchart of a specific example of a data processing method based on cyclic renaming in Embodiment 1 of the present invention;
[0075] Figure 2 is a flowchart of another specific example of the data processing method based on cyclic renaming in Embodiment 1 of the present invention;
[0076] Figure 3 is a flowchart of another specific example of the data processing method based on cyclic renaming in Embodiment 1 of the present invention;
[0077] Figure 4 is a flowchart of another specific example of the data processing method based on cyclic renaming in Embodiment 1 of the present invention;
[0078] Figure 5 is a principle block diagram of a specific example of a data processing device based on cyclic renaming in Embodiment 2 of the present invention;
[0079] Figure 6 is a principle block diagram of another specific example of a data processing device based on cyclic renaming in Embodiment 2 of the present invention;
[0080] Figure 7 This is a principle block diagram of a specific example of a data processing system based on file cyclic renaming in Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0081] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0082] In the description of the present invention, it should be noted that the terms used herein are only used for the purpose of describing specific embodiments, and are not intended to limit the present invention. Unless the context clearly indicates, the singular forms "one", "an" and "the" as used herein are intended to also include plural forms. When using terms such as "include" and / or "comprise", it is intended to illustrate the existence of the feature, integer, step, operation, element and / or component, without excluding the existence or increase of one or more other features, integers, steps, operations, elements, components, and / or other combinations. The term "and / or" includes any and all combinations of one or more related listed items. The terms "first", "second", "third", etc. are only used for description purposes and cannot be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood by specific circumstances.
[0083] Although the exemplary embodiment is described as using multiple units to perform the exemplary process, it is understood that the exemplary process can also be performed by one or more modules. In addition, some of the figures in this specification are flowcharts for illustrating methods. It should be understood that each block in these flowcharts and the combination of blocks in these flowcharts can be implemented by computer program instructions. These computer program instructions can be loaded onto a computer or other programmable device to form a machine so that the instructions executed on the computer or other programmable device form a structure for implementing the functions specified in the flowchart blocks. These computer program instructions can also be stored in a computer readable memory, which can instruct a computer or other programmable device to work in a specific manner so that the instructions stored in the computer readable memory form a product containing an instruction structure for implementing the functions specified in the flowchart blocks. The computer program instructions can also be loaded onto a computer or other programmable device so that a series of operating steps are performed on the computer or other programmable device to form a process implemented by a computer, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flowchart blocks.
[0084] Accordingly, the blocks in each flow chart support a combination of structures for performing the specified functions and a combination of steps for performing the specified functions. It should also be understood that each block in the flow chart and the combination of blocks in the flow chart can be implemented by a computer system based on dedicated hardware that performs the specified functions or steps, or a combination of dedicated hardware and computer instructions.
[0085] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0086] Example 1
[0087] This embodiment provides a data processing method based on file cyclic renaming, which can be applied to vehicle-mounted equipment, for example, and is specifically divided into a writing thread and a reading thread, wherein, for example Figure 1 As shown, the specific steps for the vehicle-mounted device write thread to read the data queue of the storage device (such as memory, etc.) are:
[0088] S100, reading a data record from the data queue;
[0089] S200, renaming the data_write file, the data_bak1 file, the data_bak2 file and the data_read file in a loop, so as to write the data record into the data_write file;
[0090] S300, determine whether the data_read file exists; when the data_read file exists, return to S100 to continue reading a data record from the data queue;
[0091] S400. When the data_read file does not exist, rename the data_write file to the data_read file and then return to S100 to continue reading a data record from the data queue.
[0092] Preferably, if Figure 2 As shown, the steps of S200 include:
[0093] S201, determine whether the data_write file exists;
[0094] S202, when the data_write file exists, writing the data record into the data_write file;
[0095] S203, when the data_write file does not exist, determine whether the data_bak1 file exists;
[0096] S204, when the data_bak1 file exists, rename the data_bak1 file to data_write file and then go to S202;
[0097] S205, when the data_bak1 file does not exist, determine whether the data_bak2 file exists;
[0098] S206, when the data_bak2 file exists, rename the data_bak2 file to data_write file and then go to S202;
[0099] S207. When the data_bak2 file does not exist, create a new data_write file and go to S202.
[0100] The above-mentioned write thread can be briefly described as: first determine whether the data_write file exists. If not, determine whether the data_bak1 file exists. If so, rename the data_bak1 file to data_write. Otherwise, if the data_bak1 file does not exist, determine whether the data_bak2 file exists. If so, rename the data_bak2 file to data_write. If both the data_bak1 file and the data_bak2 file do not exist, directly add a data_write file, and write the data read from the memory queue directly to the data_write file. After writing is completed, determine whether the data_read file exists. If not, rename the data_write file to data_read. If so, continue to read the memory data and write it to the data_write file.
[0101] like Figure 3 As shown, the specific steps for the vehicle device reading thread to read the data_read file data are:
[0102] S500, determine whether the data_read file exists;
[0103] S600. When the data_read file exists, continuously read and output data records from the data_read file until all data records are read and output, for example, the data records are reported to the cloud; preferably, a single read and output method is adopted when reading and outputting data records.
[0104] S700, renaming the data_bak1 file, the data_bak2 file and the data_read file cyclically, so as to rename the data_read file into the data_bak1 file or the data_bak2 file and then proceed to S500;
[0105] S800. When the data_read file does not exist, the process goes to S500 after sleeping for a period of time. Preferably, the period of time is 5-15 ms, for example, 10 ms.
[0106] Preferably, the step of S700 includes:
[0107] S701, determine whether the data_bak1 file exists;
[0108] S702, when the data_bak1 file exists, determine whether the data_bak2 file exists;
[0109] S703, when the data_bak2 file exists, rename the data_read file to data_bak1 file;
[0110] S704, when the data_bak2 file does not exist, rename the data_read file to the data_bak2 file;
[0111] S705. When the data_bak1 file does not exist, rename the data_read file to the data_bak1 file.
[0112] The above reading thread can be briefly described as: first read the data_read file data, and then report it to the cloud after reading. After all records are read and reported, determine whether the data_bak1 file and the data_bak2 file exist. If both exist, rename them to data_bak1. If the data_bak1 file does not exist, rename it to data_bak1. If the data_bak2 file does not exist, rename it to data_bak2.
[0113] The above-mentioned data processing method based on file cyclic renaming decouples concurrent data processing through file cyclic renaming exchange, and is used for data reception, reading, storage, backup and reporting to a remote server, which reduces technical complexity, is simple and convenient, and only requires simple text processing, thereby improving transmission and storage efficiency, and is conducive to maintaining data integrity, consistency and reusability. By storing, backing up and reporting to a remote server, vehicle-mounted equipment space is saved and costs are reduced. The data storage method using file cyclic renaming is highly readable and does not require complex and cumbersome data queries. Data processing based on file cyclic renaming simplifies the processing flow and also simplifies the troubleshooting process.
[0114] Example 2
[0115] This embodiment provides a data processing device based on cyclic renaming, corresponding to the data processing method based on cyclic renaming in embodiment 1, such as Figure 5 As shown, including:
[0116] The data record reading unit 100 is used to read a data record from the data queue;
[0117] A first cyclic renaming unit 200 is used to cyclically rename the data_write file, the data_bak1 file, the data_bak2 file and the data_read file to write the data record into the data_write file;
[0118] The first judgment unit 300 is used to judge whether the data_read file exists; when the data_read file exists, return to the data record reading unit 100 to continue reading a data record from the data queue;
[0119] The renaming repeating unit 400 is used for, when the data_read file does not exist, renaming the data_write file to the data_read file and then returning to the data record reading unit 100 to continue reading a data record from the data queue.
[0120] Preferably, the first cyclic renaming unit comprises:
[0121] The second judgment unit is used to judge whether the data_write file exists;
[0122] A writing unit, used for writing the data record into the data_write file when the data_write file exists;
[0123] The third judgment unit is used to judge whether the data_bak1 file exists when the data_write file does not exist;
[0124] The first renaming and writing unit is used to rename the data_bak1 file to data_write file and then transfer it to the writing unit when the data_bak1 file exists;
[0125] A fourth judgment unit, used for judging whether the data_bak2 file exists when the data_bak1 file does not exist;
[0126] The second renaming and writing unit is used to rename the data_bak2 file to data_write file and then transfer it to the writing unit when the data_bak2 file exists;
[0127] Create a new write unit, which is used to create a new data_write file and transfer it to the write unit when the data_bak2 file does not exist.
[0128] Preferably, if Figure 6 As shown, the data processing device based on cyclic renaming also includes:
[0129] A fifth determination unit 500 is used to determine whether the data_read file exists;
[0130] The data record reading unit 600 is used to continuously read and output data records from the data_read file when the data_read file exists until all data records are read and output;
[0131] The second cyclic renaming unit 700 is used to cyclically rename the data_bak1 file, the data_bak2 file and the data_read file, so as to rename the data_read file into the data_bak1 file or the data_bak2 file and then transfer to the fifth judgment unit 500;
[0132] The sleep waiting unit 800 is used to switch to the fifth determination unit 500 after sleeping for a period of time when the data_read file does not exist.
[0133] Preferably, the second cyclic renaming unit comprises:
[0134] The sixth judgment unit is used to judge whether the data_bak1 file exists;
[0135] A seventh judgment unit, used for judging whether the data_bak2 file exists when the data_bak1 file exists;
[0136] The first renaming unit is used to rename the data_read file to data_bak1 file when the data_bak2 file exists;
[0137] The second renaming unit is used to rename the data_read file to data_bak2 file when the data_bak2 file does not exist;
[0138] The third renaming unit is used to rename the data_read file to the data_bak1 file when the data_bak1 file does not exist.
[0139] The above-mentioned data processing device based on file cyclic renaming decouples concurrent data processing through file cyclic renaming exchange, and is used for data reception, reading, storage, backup and reporting to a remote server, which reduces technical complexity, is simple and convenient, and only requires simple text processing, thereby improving transmission and storage efficiency, and is conducive to maintaining data integrity, consistency and reusability. By storing, backing up and reporting to a remote server, vehicle-mounted equipment space is saved and costs are reduced. The data storage method using file cyclic renaming is highly readable and does not require complex and cumbersome data queries. Data processing based on file cyclic renaming simplifies the processing flow and also simplifies the troubleshooting process.
[0140] Example 3
[0141] This embodiment provides a vehicle-mounted device, including:
[0142] one or more processors; and
[0143] A storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method based on cyclic renaming of embodiment 1.
[0144] Example 4
[0145] This embodiment provides a data processing system based on file cyclic renaming, such as Figure 7 As shown, including:
[0146] The vehicle-mounted device 10 of the third embodiment; and
[0147] The storage device 20 is used to receive the data records output by the vehicle-mounted device 10 for storage or backup, and / or output a data queue to the vehicle-mounted device 10 .
[0148] Preferably, it also includes:
[0149] The remote server 30 is used to receive data records reported by the vehicle-mounted device 10, and / or output data queues to the vehicle-mounted device 10. Thus, by storing, backing up and reporting to the remote server, the vehicle-mounted device space is saved and the cost is reduced.
[0150] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.
Claims
1. A data processing method based on file cyclic renaming, characterized in that: The data processing method based on file renaming includes a writing thread, and the writing thread includes the following steps: Read a data record from the data queue; Determine whether the first file exists; When the first file exists, writing the data record into the first file; When the first file does not exist, determining whether the second file exists; When the second file exists, renaming the second file to the first file and then writing the data record into the first file; When the second file does not exist, determining whether the third file exists; When the third file exists, renaming the third file to the first file and then writing the data record into the first file; When the third file does not exist, creating a new first file and writing the data record into the first file; Determine whether the fourth file exists; When the fourth file exists, continue to read a data record from the data queue; When the fourth file does not exist, renaming the first file to the fourth file to continue reading a data record from the data queue; The data processing method based on file renaming also includes a reading thread, and the reading thread includes the following steps: Determine whether the fourth file exists; When the fourth file exists, continuously reading and outputting data records from the fourth file until all data records are read and outputted; Circularly renaming the second file, the third file and the fourth file to rename the fourth file to the second file or the third file, and then continue to determine whether the fourth file exists; When the fourth file does not exist, sleep for a period of time, and then continue to determine whether the fourth file exists; The step of cyclically renaming the second file, the third file, and the fourth file to rename the fourth file to the second file or the third file comprises: Determine whether the second file exists; When the second file exists, determining whether the third file exists; When the third file exists, rename the fourth file to the second file; When the third file does not exist, renaming the fourth file to the third file; When the second file does not exist, the fourth file is renamed as the second file.
2. A vehicle-mounted device, characterized in that: include: one or more processors; as well as A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method based on file cyclic renaming as claimed in claim 1.
3. A data processing system based on file cyclic renaming, characterized in that: include: The vehicle-mounted device as claimed in claim 2; as well as The storage device is used to receive the data records output by the vehicle-mounted device for storage or backup, and / or output the data queue to the vehicle-mounted device.
4. The system according to claim 3, characterized in that Also includes: The remote server is used to receive data records reported by the vehicle-mounted device and / or output data queues to the vehicle-mounted device.
5. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the data processing method based on file cyclic renaming as claimed in claim 1 is implemented.
