Carbon Emission Dynamic Tracking Method and System

Abstract

The present application discloses a method and system for dynamic carbon emission tracking, comprising the following steps: a monitoring terminal distributes collected sensor data to a message queue via a gateway, wherein the sensor data is used to execute a carbon emission tracking service; obtaining a monitoring terminal corresponding to a service in a service system, and the service's requirements for sensor data processing; determining a message queue window size and a window delay time based on the requirements for sensor data processing; the service system determines whether there is missing sensor data based on the sensor data obtained by the message queue within a window period, and then completes the missing data; the completed data is input into a tracking service model to obtain a tracking result; wherein the window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the service system to fill in the data.

Description

Technical Field

[0001] This application relates to big data and carbon emission technology, and in particular to a method and system for dynamic tracking of carbon emissions. Background Art

[0002] The IoT often requires collecting large amounts of data and performing relevant calculations based on this data over a specific time period. For example, in applications such as carbon emissions tracking and measurement, many monitoring systems perform real-time monitoring, while others perform periodic monitoring over longer time periods. These applications may encompass various businesses, such as generating data reports and predicting future events. In reality, data is aggregated through various routes into message queues and then consumed by business systems. However, for various reasons, data may be delayed, and the order in which it arrives in the message queue may not correspond to the time of its generation. While some tools offer solutions for handling out-of-order data, excessively delayed data can impact actual business operations. Some solutions can use models or statistical methods to complete data, but these require a comprehensive approach to both timeliness and accuracy.

[0003] The above content is the inventor's knowledge and understanding of the relevant technology of this application, and it does not necessarily constitute part of the prior art. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a method and system for dynamic carbon emission tracking to balance the timeliness and accuracy of data.

[0005] In one aspect, an embodiment of the present application provides a method for dynamic tracking of carbon emissions, comprising the following steps:

[0006] The monitoring terminal distributes the collected sensor data to the message queue through the gateway. The sensor data is used to perform carbon emission tracking services.

[0007] Obtain the monitoring terminals corresponding to the business in the business system, as well as the business's requirements for sensor data processing;

[0008] Based on the requirements for sensor data processing, determine the window size and window delay time of the message queue;

[0009] The business system determines whether there is any missing sensor data based on the sensor data obtained by the message queue within a window period, and then completes the missing data;

[0010] Input the completed data into the tracking business model to obtain the tracking results;

[0011] The window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data.

[0012] In some embodiments, the delay time of the window is configured by:

[0013] Set the constraint condition, the maximum delay time t allowed by the business max ≥Communication delay t c +Data completion time t f + window delay time t w ;

[0014] The data completion time t f = backlog delay of data completion program t d +Data completion processing time t p , where data completion processing time = number of data completions * amount of data completion operations / concurrent data processing capability;

[0015] The number of data completions is calculated by counting the number of data arriving later than the window interval K and the distribution of the delayed data on the time axis, according to the window delay time t w The minimum setting step and the distribution of the data arriving later than the window interval are counted to determine the function S, S(t w ) means setting t w After the delay, the proportion of data K not covered by the window delay time is used to estimate the amount of data that needs to be supplemented based on S*K;

[0016] Construct an evaluation function g, which is scored by the delay factor y and the accuracy factor z. The greater the total delay, the smaller the delay factor, the greater the number of values to be filled, and the smaller the accuracy factor.

[0017] The evaluation function is solved according to the maximization of score as the optimization goal, max g = (ay + bz), a and b are coefficients, and max represents the maximum value operation symbol.

[0018] In some embodiments, the window collects data according to the data generation time, and the window determines the width of the collected data production time according to the window size.

[0019] In some embodiments, it is determined whether to cache the discarded data in the queue based on the data identifier, and the cached data is reported to a preset business system according to a preset period for data repair.

[0020] In some embodiments, multiple message queues are included, each message queue corresponds to at least one window thread for collecting sensor data, and the data in each window thread is consumed by a corresponding business in the business system.

[0021] In some embodiments, the processing requirements include a data push frequency and a data volume pushed each time, and the window size is determined according to the push frequency and data volume of the data corresponding to the service.

[0022] In some embodiments, the business system selects a corresponding algorithm or artificial intelligence model to fill in the data based on the type and missing mode of the actual data.

[0023] In some embodiments, when the push frequency required by the service is greater than the first threshold and / or the maximum delay time is less than the second threshold, the window delay time is determined by the evaluation function g; otherwise, the window delay time is determined by looking up the table according to the push frequency.

[0024] In some embodiments, the following step is further included: adjusting the coefficient of the accuracy factor z corresponding to the service according to the error of the padded data.

[0025] On the other hand, an embodiment of the present application provides a carbon emissions dynamic tracking system, including:

[0026] Memory, used to store programs;

[0027] The processor is used to load the program to execute the carbon emission dynamic tracking method.

[0028] Beneficial effects: This application configures the window delay time based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data. It can use the relevant function model to set the delay of the window in the business message queue, thereby achieving a balance between business data accuracy and delay size. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments.

[0030] Figure 1 This is a flow chart of a method provided by an embodiment of the present application;

[0031] Figure 2 This is a system block diagram provided by an embodiment of the present application;

[0032] Figure 3 This is another system block diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions, and advantages of this application more clear, the following will refer to the drawings in the embodiments of this application to clearly and completely describe the technical solutions of this application through implementation methods. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] Reference Figure 1 , the embodiment of the present application provides a method for dynamic tracking of carbon emissions, comprising the following steps:

[0035] S1. The monitoring terminal distributes collected sensor data to a message queue via the gateway. This sensor data is used to execute the carbon emissions tracking service. It is understood that a monitoring terminal can be connected to multiple different sensors, each used to monitor a specific object, such as a factory, building, or road. Different services may require data collected by one or more monitoring terminals to complete a task. These sensors carry a specific service identifier when reporting data, and specific services consume this data.

[0036] S2. Obtain the monitoring terminals corresponding to the business in the business system and the business's requirements for sensor data processing. These requirements include the frequency of consumption, the amount of data required for each consumption, and data latency requirements.

[0037] S3. Determine the window size and window delay of the message queue based on the requirements for sensor data processing. In some embodiments, the window collects data based on the data generation time, and the window determines the width of the collected data production time based on the window size.

[0038] It will be appreciated that, in the embodiments, the actual time of data generation is used as the basis for window data collection. The window size and window movement method (sliding distance) can be determined based on data requirements. The required reporting frequency limits the window size and window movement distance, and the amount of data required for each report determines the window size. Of course, in practice, it is also limited by the frequency of data reporting by the monitoring terminal.

[0039] S4. The business system determines whether there is any missing sensor data based on the sensor data obtained within a window period, and then completes the missing data.

[0040] Understandably, the data collection window may not fully collect all data, and some data may arrive late. In this case, missing data needs to be supplemented to avoid impacting business operations. Therefore, in Benfang, missing data is supplemented before being delivered to specific businesses for processing. Understandably, in this step, after extracting data from the message queue window, the data is verified for missing data, and then supplemented using known interpolation and prediction models.

[0041] S5. Input the completed data into the tracking business model to obtain the tracking result.

[0042] It's understandable that the completed data can be used to track carbon emissions through the tracking business model. Specific services can include real-time display of current data, forecasts of future data, and maps showing actual emissions distribution.

[0043] The window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data.

[0044] In some embodiments, the delay time of the window is configured by:

[0045] Set the constraint condition, the maximum delay time t allowed by the business max ≥Communication delay t c +Data completion time t f + window delay time t w . It is understandable that the maximum delay time allowed by the business refers to the maximum allowed time for the business to receive data used for the business. For example, the business allows data to arrive with a delay of 3 seconds. For example, data that should be reported at time point A is allowed to arrive in the business system at the latest at time point A+3 seconds. It is understandable that this part of the time includes transmission delay, time to complete missing data and window delay time. Here, the window delay time is actually the waiting time set by the message queue for data. Once this data is exceeded, the data collected in the window will flow to the next link, which is the link to determine whether the data is complete and complete it. Therefore, in this constraint, the processing time for data completion is also considered.

[0046] The data completion time t f = backlog delay of data completion program t d +Data completion processing time t p , where data completion processing time = number of data completions * amount of data completion operations / concurrent data processing capability.

[0047] It's understandable that completing small amounts of data is actually quite fast, but given the different business needs for data completion, we estimate the time here. The estimated data completion time takes into account the average backlog delay over a period of time and the data processing time.

[0048] The number of data completions is calculated by counting the number of data arriving later than the window interval K and the distribution of the delayed data on the time axis, according to the window delay time t w The minimum setting step and the distribution of the data arriving later than the window interval are counted to determine the function S, S(t w ) means setting t w After the delay, the proportion not covered by the window delay time is the amount of data that needs to be supplemented based on S*K estimation.

[0049] In this scheme, the amount of data arriving after the window interval is counted at a certain period. For example, if the window is from time point A to time point B, the amount of data arriving after time point B is K. The distribution is then counted based on the minimum delay step that can be set for the window. The delay in the window setting can be set to a minimum of 1 second. Of course, different message queues have different setting limits, some allowing a delay of 100ms, or even more or less. It is understandable that by counting the distribution of this amount of data, K, a piecewise function S can be generated to represent the distribution.

[0050] For example, taking the minimum setting step as 1s, the piecewise function can be constructed in the following form:

[0051]

[0052] From the above function, we can see that when the delay is set to 1s, 25% of the data in the K number cannot arrive in time.

[0053] Construct an evaluation function g, which is evaluated by the delay factor y and the accuracy factor z. The greater the total delay, the smaller the delay factor, the greater the number of values to be filled, and the smaller the accuracy factor;

[0054] Among them, the evaluation function is solved according to the maximization of the score as the optimization goal, max g=(ay+bz), a and b are coefficients, and max represents the maximum value operation symbol. It can be understood that the above constraints constrain the solution of the above formula. It can be understood that through the above function, a better solution can be solved. For different businesses, different systems and function representations can be set for the delay factor and accuracy factor according to the characteristics of the business. For example, the delay factor y can be set as a piecewise function or as an inverse proportional function of the total delay. In order to set it as a piecewise function, it can be divided into multiple ranges and different scores can be assigned to each range, wherein the longer the time of presenting the delay interval, the lower the score. The accuracy factor z can also be set as a piecewise function, and the corresponding score in each proportion range can be set according to the proportion of filler data to the data.

[0055] In some embodiments, data discarded from the queue is determined to be cached based on its identifier, and the cached data is reported to the pre-set business system at a preset period for data repair. It is understandable that for some businesses, data needs to be preserved, but the data must also be processed in a timely manner. Therefore, some data is processed using data completion, but subsequent business processing requires real data. For data that does not arrive within the specified time, data repair is performed for the business that requires it.

[0056] In some embodiments, multiple message queues are included, each corresponding to at least one window thread for collecting sensor data. The data in each window thread is consumed by the corresponding business in the business system. It is understood that different businesses can be processed by multiple message queues, and each message queue can be configured with multiple window threads to collect data for different businesses.

[0057] In some embodiments, the processing requirements include a data push frequency and a data volume pushed each time, and the window size is determined according to the push frequency and data volume of the data corresponding to the service.

[0058] In some embodiments, the business system selects an appropriate algorithm or artificial intelligence model to fill in the data based on the type of data and the missing data. For example, data interpolation can be used to complete the data. Regression models can also be used to complete the data. AI models can also be used for processing. The specific method depends on the data characteristics and data requirements of the business.

[0059] In some embodiments, when the push frequency required by the service is greater than a first threshold and / or the maximum delay time is less than a second threshold, the window delay time is determined by the evaluation function g; otherwise, the window delay time is determined by looking up the table according to the push frequency. It is understandable that when the maximum delay allowed by the service is large, or when the push frequency is low, it means that the service has enough time to wait, and solving the problem through the evaluation function g often does not yield a suitable solution because the solutions tend to set a larger window delay. Therefore, calculations can be reduced by directly setting a fixed delay time for such services. This process can be determined by setting a corresponding delay table.

[0060] In some embodiments, the following steps are further included: adjusting the coefficient b of the accuracy factor z corresponding to the service based on the error of the padded data. In some embodiments, the coefficient b of the accuracy factor z can be adjusted based on the effect of data padded. It is understood that when the error worsens, the more padded data is, the greater the impact on the accuracy factor. Without adjusting the calculation method of the accuracy factor z itself, the coefficient b can be used to adjust its impact on the overall evaluation. Therefore, the coefficient of the accuracy factor z can be adjusted based on the changing trend of the data error, so that the evaluation function can adapt to changes in actual conditions. It is understandable that when the padded accuracy is worse, the weight assigned to the accuracy factor should be lower, so the value of b can be adjusted lower.

[0061] Reference Figure 2 , an embodiment of the present application provides a carbon emission dynamic tracking system, comprising:

[0062] exist Figure 2 Multiple monitoring terminals can be identified, each of which can include one or more sensors. Data from these terminals is distributed to multiple message queues via multiple gateways. The monitoring terminals / gateways are assigned to different message queues based on the business nature of the data, and then consumed by the business system. In this solution, the delay size of the message queue window can be dynamically adjusted to balance data latency and accuracy.

[0063] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other via the communication bus. The processor may invoke logic instructions in the memory to execute the method for dynamic carbon emissions tracking.

[0064] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable logic unit (PLC), a data processing logic unit based on quantum computing, and the like.

[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0066] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] Note that the above are only preferred embodiments of the present application and the technical principles employed. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for dynamic tracking of carbon emissions, characterized in that: The following steps are involved: The monitoring terminal distributes the collected sensor data to the message queue through the gateway. The sensor data is used to perform carbon emission tracking services. Obtain the monitoring terminals corresponding to the business in the business system, as well as the business's requirements for sensor data processing; Based on the requirements for sensor data processing, determine the window size and window delay time of the message queue; The business system determines whether there is any missing sensor data based on the sensor data obtained by the message queue within a window period, and then completes the missing data; Input the completed data into the tracking business model to obtain the tracking results; The window delay time is configured based on the number of data delays counted over a period of time and the estimated time required for the business system to fill in the data; The business system selects the corresponding algorithm or artificial intelligence model to fill in the data based on the type and method of missing data; The delay time of the window is configured by the following scheme: Set the constraint condition, the maximum delay time t allowed by the business max ≥Communication delay t c +Data completion time t f + window delay time t w ; The data completion time t f = backlog delay of data completion program t d +Data completion processing time t p , where data completion processing time = number of data completions * amount of data completion operations / concurrent data processing capability; The number of data completions is calculated by counting the number of data arriving later than the window interval K and the distribution of the delayed data on the time axis, according to the window delay time t w The minimum setting step and the distribution of data arriving later than the window interval are counted to determine the function S, S(t w ) means setting t w After the delay, the proportion of data K not covered by the window delay time is used to estimate the amount of data that needs to be supplemented based on S*K; Construct an evaluation function g, which is scored by the delay factor y and the accuracy factor z. The greater the total delay, the smaller the delay factor, the greater the number of values to be filled, and the smaller the accuracy factor. The evaluation function is solved according to the maximization of score as the optimization goal, max g = (ay + bz), a and b are coefficients, and max represents the maximum value operation symbol.

2. The method according to claim 1, characterized in that The window collects data according to the data generation time, and the window determines the width of the collected data production time according to the window size.

3. The method according to claim 1, characterized in that The discarded data in the queue is judged whether to be cached based on the data identifier, and the cached data is reported to the preset business system according to the preset period for data repair.

4. The method according to claim 1, wherein It includes multiple message queues, each message queue corresponds to at least one window thread for collecting sensor data, and the data in each window thread is consumed by the corresponding business in the business system.

5. The method according to any one of claims 1 to 4, characterized in that The processing requirements include the data push frequency and the amount of data pushed each time, and the window size is determined according to the push frequency and the amount of data corresponding to the service.

6. The method according to any one of claims 1 to 4, characterized in that When the push frequency required by the service is greater than the first threshold and / or the maximum delay time is less than the second threshold, the window delay time is determined by the evaluation function g; otherwise, the window delay time is determined by looking up the table according to the push frequency.

7. The method according to claim 1, characterized in that The method further includes the following steps: adjusting the coefficient of the accuracy factor z corresponding to the business according to the error of the padded data.

8. A carbon emission dynamic tracking system, characterized in that: include: Memory, used to store programs; A processor, configured to load a program to execute the carbon emission dynamic tracking method according to any one of claims 1 to 7.

Images