A device data sequential reading method, device and medium

Abstract

This application discloses a method, device, and medium for sequentially reading device data. The method includes: a data reading control module receiving the ID of a data processing module; locking the data reading control module itself; reading a first piece of data to be processed from a dynamic queue management module according to the ID of the data processing module; determining whether the first piece of data was successfully read; if not, reading a second piece of data to be processed from the device data pool; caching the second piece of data in the dynamic queue management module, and determining whether the second piece of data was successfully cached; if not, determining that other data processing modules are no longer processing data of the device corresponding to the second piece of data, returning the second piece of data to the data processing module, and unlocking the data reading control module itself. This method achieves sequential processing of data from the same device while ensuring high concurrency efficiency and high flexibility, guaranteeing both the sequentiality of data processing and the performance of background service processing.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a method, device, and medium for sequentially reading device data. Background Technology

[0002] In many IoT applications, numerous front-end sensing devices transmit data to back-end servers. These servers need to process the data in real time and respond accordingly. In some scenarios, the back-end server must ensure that data from the same device is processed sequentially to avoid errors. For example, if thousands of vehicles report location data, the back-end service needs to calculate the vehicle's distance traveled based on its real-time location. For the same vehicle, its location data must be processed sequentially; otherwise, the calculated distance will be incorrect. Furthermore, since thousands of vehicles are uploading data simultaneously, the back-end service needs to run multiple data processing modules in parallel to process the data promptly. If two data processing modules are simultaneously processing two data points from the same vehicle, the unpredictable processing speeds of these modules may cause the module that reads the data first to finish processing it before the module that reads it later has completed processing, leading to calculation errors.

[0003] Currently, to ensure that data from the same device is processed sequentially, a fixed data processing module is usually bound to the front-end device for processing. Data from each device is processed only on one data processing module, and the data processing by each module is sequential, so the order of the device data is not disrupted.

[0004] However, if more front-end devices are added, the back-end service will need to add corresponding data processing modules. In this case, the binding relationship between the devices and the data processing modules needs to be adjusted. It cannot adapt automatically, and the adjustment is very troublesome, resulting in poor flexibility. Furthermore, when the number of front-end devices is large, too many data processing modules will also lead to high service performance pressure, ultimately resulting in low efficiency of sequential reading of device data. Summary of the Invention

[0005] This application provides a method, device, and medium for sequentially reading device data, which solves the problem of low efficiency in sequentially reading device data.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] On one hand, this application provides a method for sequentially reading device data. The method includes: a data reading control module receiving a call request from a data processing module and the ID of the data processing module; each data processing module corresponds to a different ID; locking the data reading control module itself; and reading a first piece of data to be processed from a dynamic queue management module according to the ID of the data processing module; the dynamic queue management module includes a correspondence between the data processing module currently processing data and the corresponding device, and a queue for temporarily caching the data by reading data from a device data pool; wherein different devices correspond to different queues; determining whether the first piece of data has been successfully read, and if so, then... If the first data is returned to the data processing module, then a second piece of data to be processed is read from the device data pool; the second data is data from any device; the second data is cached in the dynamic queue management module, and it is determined whether the second data is cached successfully; if not, it is determined that other data processing modules are not processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked; if yes, it is determined that other data processing modules are processing the data of the device corresponding to the second data, and a second piece of data to be processed is read again from the device data pool until the second data caching fails.

[0008] In one example, before reading a first piece of data to be processed from the dynamic queue management module according to the ID of the data processing module, the method further includes: determining the associated items of each data processing module; the associated items include the ID of the data processing module, the data key value of the device, the queue, and the time when the queue data was last read; constructing a dynamic list according to the associated items of each data processing module; constructing a timeout according to the maximum time required for each data processing module to process a piece of device data; constructing a data retrieval interface and a cached data interface, wherein the data retrieval interface is used to read device data from the dynamic queue management module, and the cached data interface is used to cache device data in the device data pool to the dynamic queue management module; and constructing the dynamic queue management module according to the dynamic list, the timeout, the data retrieval interface, and the cached data interface.

[0009] In one example, the step of reading a first piece of data to be processed from the dynamic queue management module based on the ID of the data processing module specifically includes: the dynamic queue management module receiving the ID of the data processing module through the data retrieval interface; determining whether a matching item of the data processing module is found based on the ID of the data processing module; if so, if it is determined that the device data in the queue is not empty in the matching items of the data processing module, then retrieving the earliest piece of device data from the queue, determining the earliest piece of device data as the first piece of data to be processed, returning the first piece of data to be processed to the data reading control module, and setting the time when the queue data was last read to the current time.

[0010] In one example, the method further includes: if it is determined that the device data of the queue is empty in the associated items of the data processing module, then deleting the associated item from the dynamic list; determining the time interval between the current time and the time before the last time the queue data in all other associated items was read from all other associated items in the dynamic list; determining whether there is a timeout associated item whose time interval exceeds the longest timeout timer; if not, returning empty data to the data reading control module.

[0011] In one example, the method further includes: if there is a timeout associated item with a time interval exceeding the longest timeout in the timeout register, then determining whether the device data in the queue of the timeout associated item is empty; if yes, then deleting the timeout associated item from the dynamic list; if no, replacing the ID of the data processing module in any timeout associated item with the ID of the data processing module; retrieving the earliest device data from the queue of the any timeout associated item, determining the earliest device data as the first data to be processed, returning the first data to be processed to the data reading control module, and setting the time when the queue data was last read to the current time.

[0012] In one example, the method further includes: if no associated item is matched with the data processing module, determining whether there is a timeout associated item with a time interval exceeding the longest timeout in the timeout; if not, returning empty data to the data reading control module.

[0013] In one example, caching the second data in the dynamic queue management module specifically includes: the dynamic queue management module receiving the ID of the data processing module and the second data through the cached data interface; calculating the key value of the second data by calling the data key value generation module; the data key values ​​of the same device must be the same; retrieving related items containing the key value from the dynamic list; if there is a related item containing the key value, storing the second data at the tail of the queue of the related item containing the key value; and returning empty data to the data read control module to indicate that caching was successful.

[0014] In one example, the method further includes: if there is no associated item containing the key value, adding a new associated item in the dynamic list; in the newly added associated item, setting the ID of the data processing module to the ID of the data processing module; setting the data key value of the device to the data key value of the device to be processed; setting the queue to a new empty queue; setting the time when the queue data was last read to the current time; and returning the second data to the data reading control module to indicate to the data reading control module that caching was not successful.

[0015] On the other hand, this application provides a device data sequential reading device applied to a data reading control module, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to: receive a call request from a data processing module and the ID of the data processing module; each data processing module corresponds to a different ID; lock the data reading control module itself; and read a first piece of data to be processed from a dynamic queue management module according to the ID of the data processing module; the dynamic queue management module includes a correspondence between the data processing module currently processing data and the corresponding device, and temporarily buffers data by reading data from a device data pool. A queue is used to store the data; different devices correspond to different queues; it is determined whether the first data was successfully read. If so, the first data is returned to the data processing module; if not, a second piece of data to be processed is read from the device data pool; the second data is data from any device; the second data is cached in the dynamic queue management module, and it is determined whether the second data was successfully cached; if not, it is determined that other data processing modules are not processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked; if so, it is determined that other data processing modules are processing the data of the device corresponding to the second data, and a second piece of data to be processed is read again from the device data pool until the second data caching fails.

[0016] On the other hand, embodiments of this application provide a non-volatile computer storage medium for sequential data reading of a device, storing computer-executable instructions applied to a data reading control module. The computer-executable instructions are configured to: receive a call request from a data processing module and the ID of the data processing module; each data processing module corresponds to a different ID; lock the data reading control module itself; and read a first piece of data to be processed from a dynamic queue management module according to the ID of the data processing module; the dynamic queue management module includes a correspondence between the data processing module currently processing data and the corresponding device, and a queue for temporarily caching the data by reading data from a device data pool; wherein different devices correspond to different queues; and determine the... If the first data is successfully read, return it to the data processing module; otherwise, read a second piece of data to be processed from the device data pool. The second data can be data from any device. Cache the second data to the dynamic queue management module and determine if the second data is successfully cached. If not, determine that other data processing modules are not processing data from the device corresponding to the second data, return the second data to the data processing module, and unlock the data reading control module itself. If yes, determine that other data processing modules are processing data from the device corresponding to the second data, and reread a second piece of data to be processed from the device data pool until the second data caching fails.

[0017] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0018] The data processing module ID is pre-set, and a temporary cache queue for device data is used. Through the dynamic queue management module, it is ensured that data from the same device is not processed in parallel. Especially when a large number of devices correspond to a small number of data processing modules, data from the same device is processed sequentially while ensuring high concurrency efficiency and high flexibility. This ensures the sequentiality of data processing, the performance of background service processing, and the flexibility of system expansion. Attached Figure Description

[0019] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of the framework of a device data sequential reading system provided in this application embodiment;

[0021] Figure 2 A flowchart illustrating a method for sequentially reading device data provided in an embodiment of this application;

[0022] Figure 3 A schematic diagram of the process for reading a first piece of data to be processed from the dynamic queue management module is provided for an embodiment of this application.

[0023] Figure 4 A schematic diagram illustrating the process of caching second data into a dynamic queue management module, provided for an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of a device for sequential reading of device data, provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0027] Figure 1 This is a schematic diagram of the framework of a device data sequential reading system provided in an embodiment of this application.

[0028] exist Figure 1 The sequential reading system includes a data reading control module, a dynamic queue management module, a device data pool, a data key value generation module, and several data processing modules.

[0029] The system comprises three modules: a data processing module for processing data uploaded from devices; a data read control module for providing processable data to the data processing module (acting as a central control module coordinating the collaboration between the dynamic queue management module, device data pool, and data key generation module); a dynamic queue management module for storing the mapping between data processing modules and corresponding devices, and for temporarily caching data read from the device data pool; and a queue for different devices to use as temporary buffers. The device data pool caches data uploaded from devices. The data key generation module generates identical key values ​​for data from the same device, such as the device's ID or MAC address. Data from different devices is distinguished by calculating the key values ​​of the device data.

[0030] It should be noted that the device data pool can be any device capable of storing a large amount of device data, such as message brokers, databases, etc.

[0031] In other words, the dynamic queue management module maintains the correspondence between data processing modules and their corresponding devices, and temporarily caches data that has been read from the device data pool but has not yet been handed over to the data processing modules. Generally, there may be tens of thousands of front-end devices, while there may only be a few or dozens of data processing modules running in the background. Only data processing modules that are currently processing data will have the above-mentioned correspondence in the dynamic queue management module (hence the name dynamic queue management). Once a data processing module becomes idle, that is, it is not processing data, the dynamic queue management module will not have information about that data processing module.

[0032] When constructing the dynamic queue management module, it's necessary to determine the associated items for each data processing module. This involves defining associated items, which include the data processing module's ID, the device's data key value, the queue, and the last time the queue data was read. Based on these associated items, a dynamic list is constructed. In other words, within the dynamic queue management module, each data processing module refers to the module currently processing data, and the associated items are dynamically added when caching the second piece of data.

[0033] It should be noted that the associated items exist dynamically. In the dynamic list, each data processing module ID needs to correspond to an associated item, that is, each data processing module ID needs to correspond to a queue of data from a device to be processed.

[0034] In addition, a timeout is constructed based on the maximum time required for each data processing module to process a single piece of device data from the device to be processed.

[0035] In addition, a data retrieval interface and a data caching interface are constructed. The data retrieval interface is used to read device data from the dynamic queue management module, and the data caching interface is used to cache device data in the device data pool to the dynamic queue management module.

[0036] Finally, a dynamic queue management module is constructed based on the dynamic list, the timeout, the data retrieval interface, and the cached data interface.

[0037] Specifically, during the sequential reading process, the data reading control module receives a call request from the data processing module along with the ID of the data processing module. Each data processing module corresponds to a different ID.

[0038] The data reading control module itself is locked, and the first piece of data to be processed is read from the dynamic queue management module according to the ID of the data processing module.

[0039] It's understandable that locking itself is to ensure that only one data processing module's read operation is responded to at a time. This effectively avoids the chaos that would result from performing two read operations simultaneously.

[0040] Determine whether the first data was read successfully. If so, return the first data to the data processing module.

[0041] If not, then read one second piece of data to be processed from the device data pool. It should be noted that the second piece of data can be data from any device.

[0042] The second data is cached in the dynamic queue management module. The module checks if the caching was successful. If so, it determines that another data processing module is processing the data from the device corresponding to the second data, and returns empty data to the data processing module. It's important to note that "other data processing modules" refers to any data processing module other than the one currently calling the data read control module. In other words, even if another data processing module is processing the data from the device corresponding to the second data, it will still return empty data to the data processing module.

[0043] If not, it is determined that other data processing modules are not processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked.

[0044] How exactly does this application embodiment sequentially read device data? The following will explain. Figure 2 Provide a detailed description.

[0045] Figure 2 This is a flowchart illustrating a method for sequentially reading device data according to an embodiment of this application. Certain input parameters or intermediate results in this process can be manually adjusted to help improve accuracy.

[0046] Figure 2 The process includes the following steps:

[0047] S201: The data reading control module receives the call request from the data processing module and the ID of the data processing module.

[0048] In other words, when the data processing module calls the data reading control module, it provides the ID of the data processing module. Each data processing module has a different ID, meaning that different data processing modules cannot have the same ID.

[0049] S202: Lock the data reading control module itself, and read a first piece of data to be processed from the dynamic queue management module according to the ID of the data processing module.

[0050] That is, read the earliest device data that should be processed by the data processing module from the dynamic queue management module.

[0051] S203: Determine whether the first data was read successfully.

[0052] If the read is successful, proceed to step S204; if the read fails, proceed to steps S205-S208.

[0053] S204: If so, the first data is returned to the data processing module.

[0054] S205: If not, read a second piece of data to be processed from the device data pool.

[0055] Specifically, the data acquisition module is invoked to read the second oldest piece of data to be processed from the device data pool, thus enabling the data acquisition module to read device data from the device data pool.

[0056] It should be noted that the data acquisition module can be implemented according to the actual type of equipment data pool used, and there are no restrictions on the selection of equipment data pool.

[0057] S206: Cache the second data in the dynamic queue management module and determine whether the second data has been successfully cached.

[0058] If so, it is determined that there is another data processing module processing the data of the device corresponding to the second data, and S205 is executed again.

[0059] S207: If not, then determine that other data processing modules are no longer processing the data of the device corresponding to the second data, return the second data to the data processing module, and unlock the data reading control module itself.

[0060] Understandably, S207 ensures that no two data processing modules will process data from the same device simultaneously, thus guaranteeing the sequential processing of data from the same device.

[0061] It should be noted that, although the embodiments in this application are based on... Figure 2 Steps S201 to S207 will be described sequentially, but this does not mean that steps S201 to S207 must be performed in a strict order. The reason this embodiment follows this order is... Figure 2 The order in which steps S201 to S207 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S201 to S207 can be appropriately adjusted according to actual needs.

[0062] pass Figure 2This method is effective because the data reading process is relatively simple, and the reading time is much shorter than the data processing time. Therefore, locking during reading will not cause performance problems. All data processing modules will not wait for each other, resulting in strong concurrency and high processing efficiency. There is no need for a unique correspondence between the front-end devices and the data processing modules, and their number can be adjusted arbitrarily.

[0063] Furthermore, by pre-setting the ID of the data processing module and temporarily caching the queue of data for each device, the dynamic queue management module ensures that data from the same device is not processed in parallel. Especially when a large number of devices correspond to a small number of data processing modules, the system achieves sequential processing of data from the same device while ensuring high concurrency efficiency and high flexibility. This ensures both the sequentiality of data processing and the performance of background service processing, as well as the flexibility of system expansion.

[0064] based on Figure 2 In addition to the method described herein, this application also provides some specific implementation schemes and extended schemes of the method, which will be further described below.

[0065] Figure 3 This is a schematic diagram illustrating the process of reading a first piece of data to be processed from the dynamic queue management module, as provided in an embodiment of this application.

[0066] The dynamic queue management module's data retrieval interface receives the ID of the data processing module as input and returns the device data that should be processed by that module as output. If no suitable data is available, it returns empty data. Furthermore, if a data processing module times out, it is assumed to be non-existent, and the device data cached for it will be processed by other data processing modules.

[0067] Figure 3 The process may include the following steps:

[0068] S301: Dynamic queue management module, which receives the ID of the data processing module through the data retrieval interface.

[0069] S302: Determine whether the associated item of the data processing module is matched based on the ID of the data processing module.

[0070] If no associated item of the data processing module is matched, then step S305 is executed.

[0071] S303: If the device data in the queue is determined to be non-empty in the associated items of the data processing module, then the earliest device data is taken out from the queue, the earliest device data is determined as the first data to be processed, the first data to be processed is returned to the data reading control module, and the time when the queue data was last read is set to the current time.

[0072] It should be noted that in the queue, device data is sorted in order of upload time, with the earliest data at the head of the queue and the latest data at the tail.

[0073] S304: If it is determined that the device data of the queue is empty in the associated items of the data processing module, then the associated item is deleted from the dynamic list.

[0074] S305: Determine the time interval between the current time and the time before the last time that queue data in all other associated items of the dynamic list was read, from all other associated items.

[0075] It should be noted that all other related items refer to the remaining related items in the dynamic list, excluding those related to the data processing module.

[0076] S306: Determine if there is a timeout associated item whose time interval exceeds the longest timeout in the timeout register.

[0077] S307: If not, return empty data to the data reading control module.

[0078] S308: If there is a timeout associated item with a time interval exceeding the longest timeout in the timeout unit, then determine whether the device data in the queue of the timeout associated item is empty.

[0079] S309: If so, the timeout associated item is deleted from the dynamic list.

[0080] S310: If not, replace the ID of the data processing module in any timeout association item with the ID of the data processing module. At this time, there may be multiple non-empty timeout association items.

[0081] S311: Take the earliest device data from the queue of any timeout associated item, determine the earliest device data as the first data to be processed, return the first data to be processed to the data reading control module, and set the time when the queue data was last read to the current time.

[0082] As can be seen, the dynamic queue management module has a timeout mechanism. When a data processing module encounters a problem, the device data it is responsible for processing will be transferred to another data processing module.

[0083] Figure 4 This is a schematic diagram illustrating the process of caching second data into the dynamic queue management module, provided as an embodiment of this application.

[0084] The cached data interface of the dynamic queue management module receives the ID of the data processing module and the second data as input parameters. If caching is successful, it returns empty data; otherwise, it returns the second data as input parameters.

[0085] Figure 4 The process may include the following steps:

[0086] S401: The dynamic queue management module receives the ID of the data processing module and the second data through the cache data interface.

[0087] S402: Calculate the key value of the second data by calling the data key value generation module.

[0088] It should be noted that the data key generation module can be implemented according to the actual device data format and content, and there are no restrictions on the type of device data.

[0089] S403: Retrieve from the dynamic list the associated item containing the key value.

[0090] S404: If there is an associated item containing the key value, then the second data is stored at the tail of the queue of the associated item containing the key value.

[0091] It should be noted that there will not be multiple associated entries containing the specified key value. If there are multiple entries, it means that multiple data processing modules are processing the data of the device corresponding to that key value, which cannot guarantee the processing order.

[0092] S405: Return empty data to the data read control module to indicate that the caching was successful.

[0093] S406: If there is no associated item containing the key value, then add a new associated item in the dynamic list.

[0094] S407: In the newly added associated item, set the ID of the data processing module to the ID of the data processing module.

[0095] S408: Set the data key value of the device to the data key value of the device to be processed.

[0096] S409: Set the queue to a new empty queue.

[0097] S410: Set the time when the queue data was last read to the current time.

[0098] S411: Return the second data to the data read control module to indicate to the data read control module that caching was unsuccessful.

[0099] Based on the same idea, some embodiments of this application also provide devices and non-volatile computer storage media corresponding to the above methods.

[0100] Figure 5 A schematic diagram of a device for sequentially reading data, provided in an embodiment of this application, is applied to a data reading control module and includes:

[0101] At least one processor; and,

[0102] A memory communicatively connected to the at least one processor; wherein,

[0103] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0104] Receive the call request from the data processing module and the ID of the data processing module; each data processing module corresponds to a different ID;

[0105] The data reading control module itself is locked. Based on the ID of the data processing module, a first piece of data to be processed is read from the dynamic queue management module. The dynamic queue management module includes the correspondence between the data processing module that is processing data and the corresponding device, as well as a queue for temporarily caching the data by reading data from the device data pool. Different devices correspond to different queues.

[0106] Determine whether the first data was successfully read. If so, return the first data to the data processing module.

[0107] If not, then read a second piece of data to be processed from the device data pool; the second piece of data can be data from any device.

[0108] The second data is cached in the dynamic queue management module, and it is determined whether the second data has been successfully cached.

[0109] If not, it is determined that other data processing modules are no longer processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked.

[0110] If so, it is determined that there is another data processing module processing the data of the device corresponding to the second data, and the second data to be processed is read again from the device data pool until the second data caching fails.

[0111] Some embodiments of this application provide a device for sequentially reading non-volatile computer storage media, storing computer-executable instructions, applied to a data reading control module, wherein the computer-executable instructions are configured as follows:

[0112] Receive the call request from the data processing module and the ID of the data processing module; each data processing module corresponds to a different ID;

[0113] The data reading control module itself is locked. Based on the ID of the data processing module, a first piece of data to be processed is read from the dynamic queue management module. The dynamic queue management module includes the correspondence between the data processing module that is processing data and the corresponding device, as well as a queue for temporarily caching the data by reading data from the device data pool. Different devices correspond to different queues.

[0114] Determine whether the first data was successfully read. If so, return the first data to the data processing module.

[0115] If not, then read a second piece of data to be processed from the device data pool; the second piece of data can be data from any device.

[0116] The second data is cached in the dynamic queue management module, and it is determined whether the second data has been successfully cached.

[0117] If not, it is determined that other data processing modules are no longer processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked.

[0118] If so, it is determined that there is another data processing module processing the data of the device corresponding to the second data, and the second data to be processed is read again from the device data pool until the second data caching fails.

[0119] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0120] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0121] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0127] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0128] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0129] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the technical principles of this application should fall within the protection scope of this application.

Claims

1. A method of sequentially reading device data, characterized by, The method includes: The data reading control module receives the call request from the data processing module and the ID of the data processing module; each data processing module corresponds to a different ID; The data reading control module itself is locked. Based on the ID of the data processing module, a first piece of data to be processed is read from the dynamic queue management module. The dynamic queue management module includes the correspondence between the data processing module currently processing data and the corresponding device, as well as a queue for temporarily caching the data by reading data from the device data pool. Different devices correspond to different queues. Determine whether the first data was successfully read. If so, return the first data to the data processing module. If not, then read a second piece of data to be processed from the device data pool; the second piece of data can be data from any device. The second data is cached in the dynamic queue management module, and it is determined whether the second data has been successfully cached. If not, then it is determined that other data processing modules are no longer processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked; If so, it is determined that there is another data processing module processing the data of the device corresponding to the second data, and the second data to be processed is read again from the device data pool until the second data caching fails.

2. The method according to claim 1, characterized in that, Before reading a first piece of data to be processed from the dynamic queue management module according to the ID of the data processing module, the method further includes: Determine the associated items for each data processing module; the associated items include the ID of the data processing module, the data key value of the device, the queue, and the time when the queue data was last read; A dynamic list is constructed based on the associated items of each data processing module; A timeout is constructed based on the maximum time required for each data processing module to process a piece of device data; Construct a data retrieval interface and a data caching interface. The data retrieval interface is used to read device data from the dynamic queue management module, and the data caching interface is used to cache device data in the device data pool to the dynamic queue management module. The dynamic queue management module is constructed based on the dynamic list, the timeout, the data retrieval interface, and the cached data interface.

3. The method according to claim 2, characterized in that, The step of reading a first piece of data to be processed from the dynamic queue management module according to the ID of the data processing module specifically includes: The dynamic queue management module receives the ID of the data processing module through the data retrieval interface; Based on the ID of the data processing module, determine whether a matching item of the data processing module is found; If so, if the device data in the queue is determined to be non-empty in the associated items of the data processing module, then the earliest device data is taken out from the queue, the earliest device data is determined as the first data to be processed, the first data to be processed is returned to the data reading control module, and the time when the queue data was last read is set to the current time.

4. The method according to claim 3, characterized in that, The method further includes: If it is determined that the device data of the queue is empty in the associated items of the data processing module, then the associated item will be deleted from the dynamic list; Determine the time interval between the current time and the time before the last time the queue data in all other associated items of the dynamic list was read from all other associated items; Determine if there are any timeout associated items whose time interval exceeds the longest timeout period in the timeout register; If not, empty data is returned to the data reading control module.

5. The method according to claim 4, characterized in that, The method further includes: If there is a timeout associated item with a time interval exceeding the longest timeout in the timeout register, then determine whether the device data in the queue of the timeout associated item is empty; If so, the timeout associated item will be removed from the dynamic list; If not, replace the ID of the data processing module in any timeout associated item with the ID of the data processing module. Take the earliest device data from the queue of any timeout associated item, determine the earliest device data as the first data to be processed, return the first data to be processed to the data reading control module, and set the time when the queue data was last read to the current time.

6. The method according to claim 3, characterized in that, The method further includes: If no matching item is found for the data processing module, it is determined whether there is a timeout item with a time interval exceeding the longest timeout period in the timeout mechanism. If not, empty data is returned to the data reading control module.

7. The method according to claim 2, characterized in that, The step of caching the second data in the dynamic queue management module specifically includes: The dynamic queue management module receives the ID of the data processing module and the second data through the cache data interface; The key value of the second data is calculated by calling the data key value generation module; the data key values ​​of the same device must be the same. Retrieve associated items containing the key value from the dynamic list; If there is an associated item containing the key value, then the second data is stored at the tail of the queue of the associated item containing the key value; Returning empty data to the data read control module indicates that caching was successful.

8. The method according to claim 7, characterized in that, The method further includes: If there is no associated item containing the key value, then add a new associated item in the dynamic list; In the newly added associated item, the ID of the data processing module is set to the ID of the data processing module; Set the device's data key value to the data key value of the device to be processed; Set the queue to a new empty queue; Set the time when the queue data was last read to the current time; The second data is returned to the data reading control module, indicating that the data was not cached successfully.

9. A device for sequentially reading device data, characterized in that, Applied to the data read control module, including: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Receive the call request from the data processing module and the ID of the data processing module; each data processing module corresponds to a different ID; The data reading control module itself is locked. Based on the ID of the data processing module, a first piece of data to be processed is read from the dynamic queue management module. The dynamic queue management module includes the correspondence between the data processing module that is processing data and the corresponding device, as well as a queue for temporarily caching the data by reading data from the device data pool. Different devices correspond to different queues. Determine whether the first data was successfully read. If so, return the first data to the data processing module. If not, then read a second piece of data to be processed from the device data pool; the second piece of data can be data from any device. The second data is cached in the dynamic queue management module, and it is determined whether the second data has been successfully cached. If not, then it is determined that other data processing modules are no longer processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked; If so, it is determined that there is another data processing module processing the data of the device corresponding to the second data, and the second data to be processed is read again from the device data pool until the second data caching fails.

10. A device for sequential data reading of a non-volatile computer storage medium, storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows, which are applied to the data reading control module: The data reading control module receives the call request from the data processing module and the ID of the data processing module; each data processing module corresponds to a different ID; The data reading control module itself is locked. Based on the ID of the data processing module, a first piece of data to be processed is read from the dynamic queue management module. The dynamic queue management module includes the correspondence between the data processing module that is processing data and the corresponding device, as well as a queue for temporarily caching the data by reading data from the device data pool. Different devices correspond to different queues. Determine whether the first data was successfully read. If so, return the first data to the data processing module. If not, then read a second piece of data to be processed from the device data pool; the second piece of data can be data from any device. The second data is cached in the dynamic queue management module, and it is determined whether the second data has been successfully cached. If not, then it is determined that other data processing modules are no longer processing the data of the device corresponding to the second data, the second data is returned to the data processing module, and the data reading control module itself is unlocked; If so, it is determined that there is another data processing module processing the data of the device corresponding to the second data, and the second data to be processed is read again from the device data pool until the second data caching fails.

Images