Carbon emission dynamic tracking method and system

Abstract

The invention discloses a carbon emission dynamic tracking method and system, and the method comprises the following steps: a monitoring terminal distributes collected sensor data to a message queue through a gateway, and the sensor data is used for executing a carbon emission tracking service; acquiring a monitoring terminal corresponding to a service in the service system and a requirement of the service on sensor data processing; determining the window size and the window delay time of the message queue based on the requirements for sensor data processing; the business system determines whether missing sensor data exists or not according to the sensor data obtained by the message queue in a window period, and then complements the missing data; inputting the complemented data into a tracking service model to obtain a tracking result; wherein the delay time of the window is configured according to the number of data delays counted in a period of time and the estimated time required by a service system for filling data.

Description

Technical Field

[0001] The present application relates to big data and carbon emission technology, and in particular to a method and system for dynamic tracking of carbon emissions. Background Art

[0002] The IoT field usually requires the collection of a large amount of data and the relevant calculations based on the data within a certain period of time. For example, in applications such as carbon emission tracking and measurement, many detection systems will perform real-time monitoring, and some systems will perform regular detection over a longer period of time. These applications may be various businesses such as generating data reports and predicting future events. In reality, data will be aggregated into message queues through different routes and then consumed by business systems. However, due to various reasons, data may be delayed, and the time order of its arrival in the message queue is not in accordance with the time when the data was generated. Although some tools provide solutions for processing out-of-order data, if some data is delayed too long, it will affect the actual business. Some solutions can use models or statistical methods to complete the data, but they need to comprehensively consider the timeliness and accuracy of data processing.

[0003] The above content is the inventor's knowledge and understanding of the relevant technology of this application, which does not necessarily constitute part of the prior art. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a carbon emission dynamic tracking method and system to balance the timeliness and accuracy of data.

[0005] On the one hand, an embodiment of the present application provides a method for dynamic tracking of carbon emissions, comprising the following steps:

[0006] The monitoring terminal distributes the collected sensor data to the message queue through the gateway, and the sensor data is used to perform carbon emission tracking services;

[0007] Obtain the monitoring terminals corresponding to the business in the business system, as well as the business's requirements for sensor data processing;

[0008] Based on the requirements for sensor data processing, determine the window size and window delay time of the message queue;

[0009] The business system determines whether there is missing sensor data based on the sensor data obtained by the message queue within a window period, and then completes the missing data;

[0010] Input the completed data into the tracking business model to obtain the tracking results;

[0011] The window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data.

[0012] In some embodiments, the delay time of the window is configured by this scheme:

[0013] Set constraints and the maximum delay time t allowed by the business max ≥Communication delay t c +Data completion time t f +Window delay time t w ;

[0014] The data completion time t f = backlog delay of data completion program t d +Data completion processing time t p , where data completion processing time = number of data completions * amount of data completion operations / concurrent data processing capability;

[0015] The number of data completions is calculated by counting the number of data arriving later than the window interval K and the distribution of the delayed data on the time axis, according to the window delay time t w The minimum setting step and the distribution of data arriving later than the window interval are counted to determine the function S, S(t w ) indicates setting t w After the delay, the proportion of the data quantity K not covered by the window delay time, and the amount of data that needs to be supplemented is estimated based on S*K;

[0016] Construct an evaluation function g, which is scored by the delay factor y and the accuracy factor z. The greater the total delay, the smaller the delay factor, the greater the number of values ​​to be filled, and the smaller the accuracy factor;

[0017] The evaluation function is solved by maximizing the score as the optimization goal, max g = (ay + bz), a and b are coefficients, and max represents the maximum value operation symbol.

[0018] In some embodiments, the window collects data according to the data generation time, and the window determines the width of the collected data production time according to the window size.

[0019] In some embodiments, it is determined whether to cache the discarded data in the queue according to the data identifier, and the cached data is reported to a preset business system according to a preset period to perform data repair.

[0020] In some embodiments, multiple message queues are included, each message queue corresponds to at least one window thread for collecting sensor data, and the data in each window thread is consumed by a corresponding service in the service system.

[0021] In some embodiments, the processing requirements include a data push frequency and a data volume pushed each time, and the window size is determined according to the push frequency and data volume of the data corresponding to the service.

[0022] In some embodiments, the business system selects a corresponding algorithm or artificial intelligence model to fill in the data based on the type and missing mode of the actual data.

[0023] In some embodiments, when the push frequency required by the service is greater than the first threshold and / or the maximum delay time is less than the second threshold, the window delay time is determined by the evaluation function g; otherwise, the window delay time is determined by looking up the table according to the push frequency.

[0024] In some embodiments, the following step is also included: adjusting the coefficient of the accuracy factor z corresponding to the service according to the error of the padded data.

[0025] On the other hand, an embodiment of the present application provides a carbon emission dynamic tracking system, including:

[0026] Memory, used to store programs;

[0027] A processor is used to load a program to execute the carbon emission dynamic tracking method.

[0028] Beneficial effects: This application configures the window delay time based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data. It can use the relevant function model to set the delay of the window in the business message queue, thereby achieving a balance between business data accuracy and delay size. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments.

[0030] Figure 1 is a flow chart of a method provided by an embodiment of the present application;

[0031] Figure 2 It is a system block diagram provided by an embodiment of the present application;

[0032] Figure 3 This is another system block diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described through the implementation method with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0034] Reference Figure 1 The present application embodiment provides a method for dynamic tracking of carbon emissions, comprising the following steps:

[0035] S1. The monitoring terminal distributes the collected sensor data to the message queue through the gateway. The sensor data is used to perform the carbon emission tracking service. It is understandable that the monitoring terminal can be connected to multiple different sensors, which can be used to monitor a specific object, such as a factory, a building or a road. Different services may require data collected by one or more detection terminals to complete the task. These sensors will carry certain business identifiers when reporting data, and specific services consume these data.

[0036] S2. Obtain the monitoring terminal corresponding to the business in the business system, as well as the business's requirements for sensor data processing. The requirements for sensor data processing include the frequency of consumption, the amount of data required for each consumption, and the requirements for data delay.

[0037] S3. Determine the window size and window delay time of the message queue based on the requirements for sensor data processing. In some embodiments, the window collects data according to the data generation time, and the window determines the width of the collected data production time according to the window size.

[0038] It is understandable that, in the embodiment, the actual generation time of the data is used as the basis for the window to collect data. The window size and the window movement mode (sliding distance) can be determined according to the data requirements. The frequency of reporting requirements limits the window size and the distance of window movement, and the amount of data required for each report determines the size of the window. Of course, in fact, it will also be limited by the frequency of data reporting of the monitoring terminal.

[0039] S4. The business system determines whether there is missing sensor data based on the sensor data obtained within a window period, and then completes the missing data.

[0040] It is understandable that the data collection window cannot completely collect all the data, and some data will arrive late. In this case, the missing data needs to be supplemented to avoid affecting the business. Therefore, in the process, the missing data is supplemented before being delivered to the specific business for processing. It is understandable that in this step, after taking the data out of the window of the message queue, it will be checked whether the data is missing, and then the data will be supplemented using known interpolation, prediction models, etc.

[0041] S5. Input the completed data into the tracking business model to obtain the tracking result.

[0042] It is understandable that the completed data can complete the corresponding carbon emission tracking business through the tracking business model. The specific business can be to display the current data situation in real time or to predict the data situation in the future. The actual emission distribution can also be displayed through a map.

[0043] The window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data.

[0044] In some embodiments, the delay time of the window is configured by this scheme:

[0045] Set constraints and the maximum delay time t allowed by the business max ≥Communication delay t c +Data completion time t f +Window delay time t w . It is understandable that the maximum delay time allowed by the business refers to the maximum allowed time for the business to receive data used for the business. For example, the business allows data to arrive with a delay of 3 seconds. For example, the data that should be reported at time point A is allowed to arrive in the business system at the latest at time point A + 3 seconds. It is understandable that this part of the time includes transmission delay, time to complete missing data and window delay time. Here, the window delay time is actually the waiting time for data set by the message queue. Once this data is exceeded, the data collected in this window will flow to the next link, which is the link to determine whether the data is complete and complete it. Therefore, in this constraint, the processing time for data completion is also considered.

[0046] The data completion time t f = backlog delay of data completion program t d +Data completion processing time t p , where data completion processing time = number of data completions * amount of data completion operations / concurrent data processing capability.

[0047] It is understandable that the time for completing a small amount of data is actually faster, but considering that different businesses need to complete data, the time is estimated here. The estimated data completion time takes into account the delay time caused by the average backlog in a period and the time of the data processing process.

[0048] The number of data completions is calculated by counting the number of data arriving later than the window interval K and the distribution of the delayed data on the time axis, according to the window delay time t w The minimum setting step and the distribution of data arriving later than the window interval are counted to determine the function S, S(t w ) indicates setting t w After the delay, the proportion not covered by the window delay time is the amount of data that needs to be supplemented based on S*K estimation.

[0049] In this scheme, the amount of data that arrives later than the window interval will be counted according to a certain period. For example, the window is from time point A to time point B, and the amount of data that arrives beyond time point B is K. Then, according to the minimum setting step of the delay time that can be set for the window, the distribution status is counted. Among them, the delay in setting the window can be set to a minimum of 1s. Of course, different message queues have different setting limits, and some allow 100ms to be set as a delay, or even more or less. It can be understood that by counting the distribution of these data of quantity K, a piecewise function S can be generated to represent the distribution.

[0050] For example, taking the minimum setting step as 1s, the piecewise function can be constructed in the following form:

[0051]

[0052] From the above function, we can see that when the delay is set to 1s, 25% of the data in K cannot arrive in time.

[0053] Construct an evaluation function g, which is evaluated by the delay factor y and the accuracy factor z. The larger the total delay, the smaller the delay factor, the larger the number of values ​​to be filled, and the smaller the accuracy factor;

[0054] Among them, the evaluation function is solved according to the maximization of the score as the optimization goal, max g = (ay + bz), a and b are coefficients, and max represents the maximum value operation symbol. It can be understood that the above constraints constrain the solution of the above formula. It can be understood that through the above function, a better solution can be solved. For different businesses, different systems and function representations can be set for the delay factor and the accuracy factor according to the characteristics of the business. For example, the delay factor y can be set as a piecewise function or as an inverse proportional function of the total delay. In order to set it as a piecewise function physics, it can be divided into multiple ranges and different scores can be assigned to each range, wherein the longer the time for presenting the delay interval, the lower the score. The accuracy factor z can also be set as a piecewise function, and the corresponding score in each proportional range can be set according to the proportion of fill data to data.

[0055] In some embodiments, the data discarded in the queue is determined whether to be cached based on the data identifier, and the cached data is reported to the preset business system according to the preset period to perform data repair. It is understandable that for some businesses, it is necessary to save data, but the data is required to be used for processing in a timely manner. Therefore, some data is processed by data completion, but real data is required for subsequent business processing. For data that does not arrive within the specified time, the data is repaired for the business that needs it.

[0056] In some embodiments, multiple message queues are included, each message queue corresponds to at least one window thread for collecting sensor data, and the data in each window thread is consumed by the corresponding business in the business system. It is understandable that different businesses can be processed by multiple message queues, and each message queue can be configured with multiple window threads to collect data from different businesses.

[0057] In some embodiments, the processing requirements include a data push frequency and a data volume pushed each time, and the window size is determined according to the push frequency and data volume of the data corresponding to the service.

[0058] In some embodiments, the business system selects a corresponding algorithm or artificial intelligence model to fill in the data according to the type of data and the missing mode. For example, a data interpolation solution can be used to complete the data. A regression model can be used to complete the data. An AI model can also be used for processing. It is determined specifically according to the data characteristics of the business and the requirements for the data.

[0059] In some embodiments, when the push frequency required by the service is greater than the first threshold and / or the maximum delay time is less than the second threshold, the window delay time is determined by the evaluation function g, otherwise, the window delay time is determined by looking up the table according to the push frequency. It can be understood that when the maximum delay allowed by the service is large, or the push frequency is low, it means that the service has enough time to wait, and the evaluation function g is often used to solve the problem. The solution is often not a suitable solution, because the solution tends to set a larger window delay. Therefore, the calculation can be reduced, and a fixed delay time can be directly set for such services. This process can be determined by setting a corresponding delay table.

[0060] In some embodiments, the following steps are also included, and the coefficient b of the accuracy factor z corresponding to the service is adjusted according to the error of the padded data. In some embodiments, the coefficient b of the accuracy factor z can be adjusted based on the effect of data padded. It can be understood that when the error worsens, the more padded data there is, the greater the impact on the accuracy factor. Without adjusting the calculation method of the accuracy factor z itself, the coefficient b can be used to adjust the degree of its impact on the overall evaluation. Therefore, the coefficient of the accuracy factor z can be adjusted according to the changing trend of the data error, so that the evaluation function can adapt to the changes in the actual situation. It can be understood that when the filling accuracy is worse, the weight given to the accuracy factor should be lower, so the size of b can be lowered.

[0061] Reference Figure 2 The present application embodiment provides a carbon emission dynamic tracking system, including:

[0062] exist Figure 2 It can be determined that multiple monitoring terminals are included, each of which can include one or more sensors. The data of the monitoring terminals are distributed to multiple message queues through multiple gateways, where the detection terminals / gateways are assigned to different message queues according to the business orientation of the data, and then consumed by the business system. In this solution, the delay size of the window in the message queue can be dynamically adjusted to balance the contradiction between data delay and data accuracy.

[0063] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute the carbon emission dynamic tracking method.

[0064] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0066] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for dynamic tracking of carbon emissions, characterized in that: The following steps are involved: The monitoring terminal distributes the collected sensor data to the message queue through the gateway, and the sensor data is used to perform carbon emission tracking services; Obtain the monitoring terminals corresponding to the business in the business system, as well as the business's requirements for sensor data processing; Based on the requirements for sensor data processing, determine the window size and window delay time of the message queue; The business system determines whether there is missing sensor data based on the sensor data obtained by the message queue within a window period, and then completes the missing data; Input the completed data into the tracking business model to obtain the tracking results; The window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data.

2. The method according to claim 1, characterized in that The delay time of the window is configured by the following scheme: Set constraints and the maximum delay time t allowed by the business max ≥Communication delay t c +Data completion time t f +Window delay time t w ; The data completion time t f = backlog delay of data completion program t d +Data completion processing time t p , where data completion processing time = number of data completions * amount of data completion operations / concurrent data processing capability; The number of data completions is calculated by counting the number of data arriving later than the window interval K and the distribution of the delayed data on the time axis, according to the window delay time t w The minimum setting step and the distribution of data arriving later than the window interval are counted to determine the function S, S(t w ) indicates setting t w After the delay, the proportion of the data quantity K not covered by the window delay time, and the amount of data that needs to be supplemented is estimated based on S*K; Construct an evaluation function g, which is scored by the delay factor y and the accuracy factor z. The greater the total delay, the smaller the delay factor, the greater the number of values ​​to be filled, and the smaller the accuracy factor; The evaluation function is solved by maximizing the score as the optimization goal, max g = (ay + bz), a and b are coefficients, and max represents the maximum value operation symbol.

3. The method according to claim 2, characterized in that The window collects data according to the data generation time, and the window determines the width of the collected data production time according to the window size.

4. The method according to claim 2, characterized in that: The discarded data in the queue is judged whether to be cached according to the data identifier, and the cached data is reported to the preset business system according to the preset period for data repair.

5. The method according to claim 2, characterized in that: It includes multiple message queues, each message queue corresponds to at least one window thread for collecting sensor data, and the data in each window thread is consumed by the corresponding business in the business system.

6. The method according to any one of claims 1 to 5, characterized in that: The processing requirements include the data push frequency and the amount of data pushed each time, and the window size is determined according to the push frequency and the amount of data corresponding to the service.

7. The method according to any one of claims 1 to 5, characterized in that: The business system selects the corresponding algorithm or artificial intelligence model to fill in the data based on the actual data type and missing mode.

8. The method according to any one of claims 1 to 5, characterized in that: When the push frequency required by the service is greater than the first threshold and / or the maximum delay time is less than the second threshold, the window delay time is determined by the evaluation function g; otherwise, the window delay time is determined by looking up the table according to the push frequency.

9. The method according to claim 1, characterized in that: The method also includes the following steps: adjusting the coefficient of the accuracy factor z corresponding to the business according to the error of the padded data.

10. A carbon emission dynamic tracking system, characterized in that: include: Memory, used to store programs; A processor, used for loading a program to execute the carbon emission dynamic tracking method as described in any one of claims 1 to 8.

Images