Real-time data acquisition method and system based on Internet of Things

Abstract

The invention relates to the field of data processing, in particular to a real-time data acquisition method and system based on the Internet of Things. The method comprises the following steps: establishing corresponding processes for data collected by a plurality of sensors, wherein the processes comprise process arrival time and service time; obtaining a process sequence according to the process arrival time and the service time, and calculating a process selection parameter according to the process sequence and the service time; reordering the process sequence to obtain the queue jumping times of the processes, and calculating the yielding degree of the processes according to the process selection parameters and the queue jumping times of the processes; source service parameters of the processes are calculated according to the yielding degrees of the processes, then service grade parameters of the processes are calculated, sorting of the process sequences is further adjusted according to the service grade parameters, the source service parameters represent the overall queue jumping degrees of the sensors corresponding to the processes, and the service grade parameters represent the priority service degrees of the processes. The problem of low data acquisition and processing efficiency in the prior art is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more specifically, to a real-time data collection method and system based on the Internet of Things. Background Art

[0002] The Internet of Things is an important direction for the development of modern technology, and real-time data collection is one of its core functions. Through various sensors and smart devices, the Internet of Things can collect real-time data in different environments, covering temperature, humidity, pressure, light, location information, etc. Sensors are connected to the cloud platform via wireless networks, allowing data to be transmitted and processed instantly. This real-time data collection not only improves the timeliness of data, but also enhances the accuracy and completeness of information, providing a solid foundation for subsequent analysis.

[0003] In industrial production, the role of real-time data collection systems is particularly significant. By deploying sensors at every stage of the production line, companies can monitor equipment status and production processes in real time, thereby quickly identifying potential failures and reducing downtime. This predictive maintenance method not only improves production efficiency but also reduces operating costs. In addition, the real-time nature of data also enables companies to respond quickly to changes in market demand, optimize production plans and inventory management, and improve overall operational flexibility.

[0004] At present, the patent document with the announcement number "CN109033387B" and the name "An Internet of Things Search System, Method and Storage Medium for Fusion of Multi-Source Data" discloses a method that can aggregate data generated by heterogeneous Internet of Things devices and industry data and Internet data from multiple sources onto a platform to provide data support for data fusion analysis. In this method, the sorting of data involves sorting by time.

[0005] The above methods usually cannot sort the data collection well, thus affecting the efficiency of data collection and processing. Summary of the invention

[0006] In order to solve the problem of low efficiency of data collection and processing in the prior art, the present invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides a real-time data collection method based on the Internet of Things, comprising: Establish corresponding processes for data collected by multiple sensors, where the processes include arrival time and service time of the processes; obtain process sequences based on the arrival time and service time of the processes, and calculate process selection parameters based on the process sequences and service time ,in, represents the process selection parameter of process i, represents the sequence number of process i in the process sequence, Represents the service time of process i; reorder the process sequence to obtain the number of times the process is interrupted, and calculate the degree of concession of the process according to the process selection parameter and the number of times the process is interrupted ,in, represents the concession degree of process i, represents the process selection parameter of process i, Indicates the number of times process i is queued. Indicates the number of times the t-th process of the queue-intercepting process i is interrupted. Indicates the number of processes that jump the queue i; according to the degree of concession of the process Calculate the source service parameter of the process, and then calculate the service level parameter of the process, and further adjust the order of the process sequence according to the service level parameter. The source service parameter is used to characterize the overall queue interruption degree of the sensor corresponding to the process, and the service level parameter is used to characterize the priority service degree of the process.

[0008] By dynamically adjusting the process sequence and priority, the delay in data processing can be reduced. According to the process's concession degree and service level parameters, resources can be allocated more reasonably to ensure that high-priority processes can obtain services in a timely manner. By calculating the number of queue jumps and the degree of concession, it can effectively avoid certain processes from not being processed for a long time and reduce the risk of process starvation. The source service parameters provide a reference for evaluating the overall performance of the sensor, which helps to optimize and improve the process from the sensor, and ultimately solve the problem of low efficiency in data collection and processing in current technology.

[0009] Preferably, the reordering of the process sequence includes: reordering the process sequence using a shortest job first algorithm.

[0010] The shortest job first algorithm can significantly reduce the average waiting time of the process and improve overall efficiency by giving priority to short jobs. Short job priority processing enables more processes to be completed quickly, better utilizes system resources, and avoids processes that occupy resources for a long time affecting the entire system.

[0011] Preferably, the data collected by the multiple sensors include: at least two sensors collecting data according to a fixed period.

[0012] Through multiple sensors, more comprehensive and rich information can be obtained, and fixed-period collection ensures timely data updates.

[0013] Preferably, the source service parameter of the process is calculated according to the concession degree of the process ,include: ,in, represents the source service parameter of sensor j, represents the concession degree of the lth process in sensor j, represents the concession degree of the lth process in sensor k, n represents the number of processes of sensor j, m represents the number of sensors, and max[] represents calculating the maximum value of all elements in brackets [].

[0014] The comparison of the concession degree between the processes of sensor j and other sensors is used to evaluate whether the process of sensor j does not have too much service time or queue jumping compared with the processes of other sensors, so as to judge whether the process of sensor j needs to be processed with priority. Considering the situation of the entire sensor can avoid the overall delay of the process of a certain sensor, and also avoid the situation where the data of this sensor is always delayed and may cause the system to get stuck.

[0015] Preferably, the service level parameter of the computing process ,include: ,in, represents the service level parameter of process i, which originates from sensor j, represents the concession degree of process i, represents the concession degree of the lth process in sensor j, It represents the source service parameter of sensor j corresponding to process i, and max[] represents the maximum value of all elements in brackets [].

[0016] By comprehensively considering the specific process and the situation of the sensor corresponding to the process to calculate the service level parameters, the priority service degree of the process can be more reasonably and comprehensively judged, thereby ranking it more reasonably.

[0017] Preferably, further adjusting the order of the process sequence according to the service level parameter includes: arranging the processes in the process sequence in descending order according to the service level parameter.

[0018] Preferably, after the at least two sensors collect data according to a fixed period, the method further comprises: performing missing completion and abnormal correction processing on the collected data.

[0019] In a second aspect, the present invention further provides a real-time data collection system based on the Internet of Things, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned real-time data collection method based on the Internet of Things.

[0020] The beneficial effects of the present invention are as follows: by dynamically adjusting the process sequence and priority, the delay in data processing can be reduced; according to the process's concession degree and service level parameters, resources can be allocated more reasonably to ensure that high-priority processes can obtain services in a timely manner; by calculating the number of queue jumps and the degree of concession, it is possible to effectively avoid certain processes from not being processed for a long time and reduce the risk of process starvation; the source service parameters provide a reference for evaluating the overall performance of the sensor, which helps to optimize and improve the process from the sensor, and ultimately solve the problem of low efficiency in data acquisition and processing in current technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 This is a flow chart of a real-time data collection method based on the Internet of Things provided by an embodiment of the present invention; Figure 2 The present invention provides a real-time data collection system based on the Internet of Things. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a flow chart of a real-time data collection method based on the Internet of Things according to an embodiment of the present invention, comprising the following steps: S101. Establish corresponding processes for data collected by multiple sensors.

[0025] In some embodiments, the receiving of data collected by the sensor includes: at least two sensors collect data according to a fixed period. For example, the temperature, current and voltage sensors collect data once every 5 seconds according to a fixed period. After the at least two sensors collect data according to the fixed period, it also includes: performing missing completion and abnormal correction processing on the collected data. The missing completion and abnormal correction processing on the collected data belongs to the well-known technology and will not be repeated here.

[0026] The IoT system usually consists of one or more servers and multiple sensors. The server is responsible for data processing, storage and analysis, while the sensor is responsible for collecting data from the environment or equipment. This architecture allows different sensors to send data to the server, which then integrates and analyzes the data to ultimately achieve monitoring, control and automation functions. When receiving sensor data, the server usually needs to establish a process first. The specific process is: the sensor sends a connection request to the server, and after the server accepts the connection request, it establishes a process to receive the data collected by the sensor. The process includes the arrival time and service time of the process. The arrival time refers to the time when the process enters the ready queue, and the service time refers to the time required to process the process.

[0027] S102. Obtain a process sequence according to the arrival time and service time of the process, and calculate a process selection parameter according to the process sequence and service time.

[0028] The process selection parameters ,in, represents the sequence number of process i in the process sequence, Represents the service time of process i.

[0029] The process sequence is obtained according to the arrival time and service time of the process. Specifically, the processes are first sorted according to their arrival time, with the process with the earlier arrival time being sorted first. If multiple processes arrive at the same time, they are sorted according to the service time, with the process with the shorter service time being sorted first.

[0030] The process selection parameters Indicates the priority of process i in the current process sequence. The larger the value, the easier it is for process i to be selected for execution. It is related to the service time of the process and its sequence number in the process sequence. For example, there are currently two processes a and b in the process sequence, with sequence numbers 3 and 5 respectively, and service times of 1 second and 3 seconds respectively. , , so process a is more likely to be selected for execution.

[0031] S103: reorder the process sequence to obtain the number of queue-jumping times of the process, and calculate the degree of concession of the process according to the process selection parameter of the process and the number of queue-jumping times.

[0032] The degree of concessions in the process ,in, represents the process selection parameter of process i, Indicates the number of times process i is queued. Indicates the number of times the t-th process of the queue-intercepting process i is interrupted. Indicates the number of processes that jump the queue i.

[0033] In some embodiments, the reordering of the process sequence includes: reordering the process sequence using the shortest job first algorithm. Reordering may cause queue jumping, so the number of times each process has been jumped can be obtained. For example, the processes are sorted according to the arrival time to obtain four processes with process numbers 1 / 2 / 3 / 4, with arrival times of 0 seconds, 1 second, 2 seconds and 3 seconds, and service times of 8 seconds, 4 seconds, 2 seconds and 3 seconds, respectively. The process sequence is reordered using the shortest job first algorithm, starting from the current time 0, and process 1 is executed. During the execution of process 1, processes 2, 3 and 4 have arrived. At this time, the service time of process 3 is the shortest, so process 3 is executed first, followed by processes 4 and 2. Process 2 is jumped in line twice, and processes 3 and 4 are not jumped in line. Process selection parameters for process 2 , the degree of concession In the calculation formula, process 2 is interrupted twice, so v=2, the first process to interrupt process 2 is process 3, and the first process to interrupt process 2 is process 4. That is, the sum of the number of times process 3 and process 4 of process 2 were interrupted in the queue, so .

[0034] S104. Calculate the source service parameter of the process according to the concession degree of the process, and then calculate the service level parameter of the process. Further adjust the order of the process sequence according to the service level parameter. The source service parameter is used to characterize the overall queue interruption degree of the sensor corresponding to the process, and the service level parameter is used to characterize the priority service degree of the process.

[0035] In some embodiments, the source service parameter of the process is calculated according to the concession degree of the process ,include: ,in, represents the source service parameter of sensor j, represents the concession degree of the lth process in sensor j, represents the concession degree of the lth process in sensor k, n represents the number of processes of sensor j, m represents the number of sensors, and max[] represents calculating the maximum value of all elements in brackets []. The larger the value is, the more likely the process corresponding to the sensor will be queued compared to processes corresponding to other sensors, and the process of the sensor will be often delayed. Therefore, it is necessary to increase the priority service level of the process corresponding to the sensor.

[0036] For example, an IoT system has three sensors A, B and C that collect data as sensors. The three sensors collect data in a cycle of 5 seconds. Currently, there are three cycles of data corresponding to processes waiting to be processed, that is, there are nine processes A1, B1, C1, A2, B2, C2, A3, B3 and C3 waiting to be processed. If you want to calculate the source service parameter of sensor A , in the formula middle, A, m represents the number of sensors, that is, m=3, and n represents the number of processes of sensor A, that is, n=3. Substituting the concession degree of each process, the source service parameter of sensor A can be calculated .when , it means that the process of sensor A has not given in too much to the processes of other sensors, which means that the process of sensor A has not had too much service time or queue jumping compared with the processes of other sensors, so there is no need to increase the priority service level of the process corresponding to this sensor. Otherwise, it is necessary to further increase the priority service level of the process corresponding to this sensor.

[0037] In some embodiments, the service level parameter of the computing process ,include: ,in, Represents the service level parameter of process i, The larger the value, the more priority should be given to serving or processing process i, which comes from sensor j. represents the concession degree of process i, represents the concession degree of the lth process in sensor j, It represents the source service parameter of sensor j corresponding to process i, and max[] represents the maximum value of all elements in brackets [].

[0038] when When ≤1, it means that the concession degree of process i is less than that of sensor Most of the processes of sensor j mean that the queue-jumping situation of process i is less than that of most of the processes of sensor j, so process i does not need to further improve the priority service level of process i. When >1, the queue-jumping of process i may surpass most of the processes of sensor j. In order to prevent excessive waiting, the priority service level of process i needs to be further improved.

[0039] Combination , the degree of concession of process i and its source service parameters of sensor j Calculated The priority service level of each process can be evaluated more reasonably and accurately, and then the priority service level of each process can be evaluated more reasonably and accurately. The process sequence is further adjusted to be arranged in descending order according to the service level parameter.

[0040] The above-mentioned real-time data collection method based on the Internet of Things provided by an embodiment of the present invention can reduce the delay of data processing by dynamically adjusting the process sequence and priority. According to the degree of concession of the process and the service level parameters, resources can be allocated more reasonably to ensure that high-priority processes can obtain services in time. By calculating the number of queue jumping and the degree of concession, it can effectively avoid certain processes from not being processed for a long time and reduce the risk of process starvation. The source service parameters provide a reference for evaluating the overall performance of the sensor, which helps to optimize and improve the process from the sensor, and ultimately solve the problem of low efficiency of data collection and processing in current technology.

[0041] The present invention also provides a real-time data collection system based on the Internet of Things. Figure 2 As shown, the system includes a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, a real-time data collection method based on the Internet of Things described in the present invention is implemented.

[0042] The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface, whose configuration and functions are known in the art and thus will not be described in detail here.

[0043] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0044] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0045] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A real-time data collection method based on the Internet of Things, characterized in that: include: Establish corresponding processes for data collected by multiple sensors, and the processes include arrival time and service time of the processes; The process sequence is obtained according to the arrival time and service time of the process, and the process selection parameter is calculated according to the process sequence and service time. ,in, represents the process selection parameter of process i, represents the sequence number of process i in the process sequence, represents the service time of process i; The process sequence is reordered to obtain the number of queue interruptions of the process, and the concession degree of the process is calculated according to the process selection parameter of the process and the number of queue interruptions. ,in, represents the concession degree of process i, represents the process selection parameter of process i, Indicates the number of times process i is queued. Indicates the number of times the t-th process of the queue-intercepting process i is interrupted. Indicates the number of processes that jump the queue i; According to the degree of concession of the process Calculate the source service parameter of the process, and then calculate the service level parameter of the process, and further adjust the order of the process sequence according to the service level parameter. The source service parameter is used to characterize the overall queue interruption degree of the sensor corresponding to the process, and the service level parameter is used to characterize the priority service degree of the process.

2. The real-time data collection method based on the Internet of Things according to claim 1 is characterized in that: The reordering of the process sequence comprises: The process sequence is reordered using a shortest job first algorithm.

3. The real-time data collection method based on the Internet of Things according to claim 1 is characterized in that: The data collected by the multiple sensors include: at least two sensors collect data according to a fixed period.

4. The real-time data collection method based on the Internet of Things according to claim 3 is characterized in that: Calculate the source service parameter of the process according to the concession level of the process ,include: ,in, represents the source service parameter of sensor j, represents the concession degree of the lth process in sensor j, represents the concession degree of the lth process in sensor k, n represents the number of processes of sensor j, m represents the number of sensors, and max[] represents calculating the maximum value of all elements in brackets [].

5. The real-time data collection method based on the Internet of Things according to claim 1 is characterized in that: Service level parameter of the computing process ,include: ,in, represents the service level parameter of process i, which originates from sensor j, represents the concession degree of process i, represents the concession degree of the lth process in sensor j, It represents the source service parameter of sensor j corresponding to process i, and max[] represents the maximum value of all elements in brackets [].

6. The real-time data collection method based on the Internet of Things according to claim 1 is characterized in that: The further adjusting the order of the process sequence according to the service level parameter comprises: The processes in the process sequence are arranged in descending order according to the service level parameter.

7. The real-time data collection method based on the Internet of Things according to claim 3 is characterized in that: After the at least two sensors collect data according to a fixed period, the method includes: performing missing completion and abnormal correction processing on the collected data.

8. A real-time data collection system based on the Internet of Things, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the real-time data collection method based on the Internet of Things as described in any one of claims 1 to 7.

Images