Building energy measurement and carbon emission monitoring method based on multimodal model

Abstract

The present invention discloses a method for building energy metering and carbon emission monitoring based on a multimodal model, comprising: in the process of building component production and structure construction, collecting the dial image of the meter through a camera; inputting the dial image into a pre-trained multimodal deep learning model API, and outputting the type of the meter; identifying the meter reading according to the meter type; and calculating the real-time energy consumption of various energy sources through an edge server according to the identified meter readings and the unit carbon emission factor corresponding to the energy type. E 总 and carbon emissions C 总 This method uses cameras to automatically collect energy consumption data to improve collection efficiency; uses multimodal deep learning technology to identify meter readings, which can overcome problems such as incomplete numbers and complex dials and improve recognition accuracy; uses edge servers to calculate energy consumption and carbon emissions, reducing data transmission pressure and improving timeliness.

Description

Technical Field

[0001] The present invention relates to the field of building energy consumption and carbon emission monitoring, and specifically to a method for measuring and monitoring energy consumption and carbon emissions during the production of building components and structure construction using a multimodal deep learning model. Background Art

[0002] The construction industry is one of the main areas of energy consumption and carbon emissions. Strengthening energy consumption management and carbon emission monitoring in the production of building components and structural construction is of great significance for achieving green and low-carbon development in the construction industry. Traditional energy consumption measurement for the production of building components and structural construction mainly adopts an extensive management model. Production and construction companies estimate the input of materials, equipment, labor and other factors based on experience, and lack a refined energy consumption statistics and assessment mechanism. Data from various energy meters are mostly manually transcribed and summarized on a monthly or quarterly basis, making it difficult to grasp the real-time energy consumption of production lines, construction processes and other links. The lag in energy consumption data is not conducive to energy efficiency comparison and energy-saving potential.

[0003] In recent years, information technologies such as the Internet of Things and big data have begun to be applied to the production of building components and the construction of structures. With the help of sensors to collect the operating parameters of production equipment and construction machinery in real time, and through digital twins to reconstruct the virtual image of the production line and construction site, visual monitoring of the manufacturing and construction process can be achieved, but there are still some limitations. First, due to factors such as cost and wiring, not all energy consumption metering points are equipped with smart meters, resulting in incomplete energy consumption data collection. In particular, in a large number of existing buildings, most of the energy meters used are old electronic meters, which lack data communication functions and have high replacement costs, making it inconvenient to collect and use energy consumption data. Secondly, the energy consumption data collected by some smart meters lack correlation with the operating status of production equipment and construction machinery, making it difficult to support energy efficiency analysis and optimization.

[0004] In addition, component production and structural construction involve multiple links such as raw material processing, component transportation, and on-site assembly. Energy consumption metering points are scattered and of various types, including electricity, oil, and gas. Different energy meters have different models and specifications, and the data is heterogeneous, making it difficult to achieve unified collection and comprehensive utilization of energy consumption data. There is a lack of standardized and unified calculation methods for the conversion of energy consumption and carbon emissions, and the carbon footprint accounting results are often delayed and inaccurate. Summary of the invention

[0005] In view of the problems of lagging energy consumption and carbon emission data and inaccurate calculation in the existing production of building components and structure construction, the present invention provides a building energy metering and carbon emission monitoring method based on a multimodal model, which can improve data collection efficiency and accuracy.

[0006] In order to solve the above technical problems, the present invention includes the following technical solutions:

[0007] A building energy metering and carbon emission monitoring method based on a multimodal model comprises the following steps:

[0008] S1. During the production of building components and the construction of structures, the dial image of the meter is collected by a camera and the image signal is transmitted to the edge server. The meter includes an electricity meter, a gas meter, and a water meter;

[0009] S2. Input the dial image into the pre-trained multimodal deep learning model API and output the type of the meter;

[0010] S3. Identify the meter reading according to the meter type;

[0011] S4. Based on the identified meter readings and the unit carbon emission factor corresponding to the energy type, the edge server calculates the real-time energy consumption of each energy type. E 总 and carbon emissions C 总 .

[0012] Furthermore, if the meter is a gas meter or an electricity meter, its digital display frame area is automatically intercepted, a reading prompt word is set, the reading prompt word and the intercepted digital display frame are imported into the multimodal model, and the multimodal model directly recognizes the reading in the digital display frame;

[0013] If the meter is a water meter, set the dial recognition prompt words, and use the multimodal model to give the number and type of dial partitions and small pointers; then intercept the digital display frame and small pointer dial of the water meter respectively; set the reading prompt words, and import the reading prompt words and the intercepted digital display frame into the multimodal model, and use the multimodal model to directly identify the reading in the digital display frame; for the small pointer dial, identify the reading of each pointer respectively, and then combine them to obtain the total reading of the small pointer dial.

[0014] further, ;

[0015] ;

[0016] in, V 燃气 is the gas meter reading, in m 3 , W 水 is the water meter reading, in t, P 电力 is the meter reading in kWh, Q 燃气 is the unit calorific value of natural gas, in MJ / m 3 , F 燃气 is the carbon emission factor of natural gas, in tCO2 / TJ,F 水 is the carbon emission factor of water, in kgCO2e / t, F 电力 is the carbon emission factor of electricity, with the unit of tCO2 / MWh.

[0017] Furthermore, after step S4, the following steps are further included:

[0018] S5. Upload the energy consumption and carbon emission data of various types calculated by the edge server to the cloud management platform.

[0019] Furthermore, if the meter is a water meter, the pointer position is extracted by performing color space conversion, threshold segmentation and morphological operations on the image, and the pointer angle is calculated to obtain the reading, including:

[0020] S3-1. Convert the image from RGB space to HSV space: HSV= f (RGB), where is the color space conversion function;

[0021] S3-2. Extracting hue channel from HSV color space H , used for color segmentation;

[0022] S3-3. Select the threshold segmentation of the pointer color area:

[0023] Setting Tuning Parameters T 1 and T 2 To select the pointer color area:

[0024] ;

[0025] S3-4. Use morphological operations to remove small area noise; A is the minimum area threshold, then:

[0026] Cleaned Mask = g (pointer mask, A );

[0027] in, g is the morphological operation function, A It is a preset minimum area threshold used to filter out noise points;

[0028] S3-5. Calculate the center of mass of the pointer C ( x , y ) and direction; center of mass C It is given by the average position of pixels within the region:

[0029] ;

[0030] in, N is the total number of pixels in the area;

[0031] S3-6. Calculate the pointer angle θ And converted to readings:

[0032] d x = x 最远点 - C x ; d y = C y - y 最远点 ; θ =arctan2( d y , d x );

[0033] Convert to clockwise from directly above: θ' =(90°- θ )mod360°;

[0034] reading R Calculated as: .

[0035] Further, step S3-3 is replaced by using a local adaptive threshold algorithm to improve the reliability of pointer recognition, specifically: the threshold interval is recorded as T local (x,y), where

[0036] ;

[0037] in, x and y Represents the pixel coordinates in the image; k 1 is the window size used in the local adaptive thresholding algorithm, k 2 is used to adjust the offset of the final threshold.

[0038] Then get the pointer color area according to the selected threshold interval:

[0039] .

[0040] Due to the adoption of the above technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention uses a camera to automatically collect energy consumption data instead of manual meter reading, thereby reducing labor intensity and improving collection efficiency; adopts multimodal deep learning technology to identify meter readings, which can overcome problems such as incomplete numbers and complex dials, and improve recognition accuracy; performs energy consumption and carbon emission calculations on the edge side, reduces data transmission pressure, and improves timeliness; concentrates on monitoring energy consumption and carbon emissions on a cloud management platform, macro-controls the energy efficiency of building complexes, and supports regional energy conservation and emission reduction decisions; supports abnormal early warning, which helps to timely discover and handle abnormal energy usage behaviors and improve management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a method for building energy metering and carbon emission monitoring based on a multimodal model in one embodiment of the present invention;

[0042] Figure 2 It is a water meter dial pointer reading identification in one embodiment of the present invention;

[0043] Figure 3 The following is an example of meter classification and readings in one embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following is a further detailed description of the building energy metering and carbon emission monitoring method based on a multimodal model provided by the present invention in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer in conjunction with the following description. It should be noted that the accompanying drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0045] like Figures 1 to 3 As shown, the building energy metering and carbon emission monitoring method based on the multimodal model provided in this embodiment includes the following steps:

[0046] S1. During the production of building components and the construction of structures, the dial images of meters are collected by cameras and the image signals are transmitted to the edge server. The meters include electricity meters, gas meters, and water meters. The collection frequency is set according to demand to balance data timeliness and transmission costs. The edge server is responsible for identification and analysis.

[0047] For example, the camera is deployed at the front end of electricity meters, gas meters, water meters and other meters. The acquisition frequency is set to once an hour, the image resolution is 1920×1080 pixels, and the size of each image is about 2MB.

[0048] S2. Input the dial image into the pre-trained multimodal deep learning model API and output the type of the meter.

[0049] The pre-trained multimodal deep learning model sets the meter type prompt: "Analyze the meter types in the image: electricity meter T=1, gas meter T=2, water meter T=3. The format of outputting the meter type is T=type number", where T is the meter type. The multimodal deep learning model classifies the meter image transmitted to the edge server, that is, determines whether the meter in the image is an electricity meter, a gas meter, or a water meter. The multimodal deep learning model makes full use of image features and combines the meter type prompt to accurately distinguish different types of meters.

[0050] S3. Identify the meter reading according to the meter type.

[0051] If it is classified as a gas meter or an electricity meter, the digital display frame area is automatically captured and the reading prompt is set: "Identify the reading in the digital display frame of the meter. When T=1, the output format is = value, when T=2, the output format is = value". The reading prompt words and the intercepted digital display frame are imported into the multimodal model, and the multimodal model directly recognizes the reading in the digital display frame. The multimodal model can overcome problems such as incomplete and ambiguous numbers and accurately recognize the reading.

[0052] If it is classified as a water meter, set the dial recognition prompt: "This is an image of a water meter. Please analyze the structure and components of the water meter dial, and output the following information: the dial partition type is digital display frame + small pointer dial, the output format is Z=1, N=the number of small pointer dials, only digital display frame, the output format is Z=0, N=0", and the number and type of dial partitions and small pointers are given by the multimodal model, where Z is the water meter dial partition type and N is the number of small pointer dials. Then intercept the digital display frame and small pointer dial of the water meter respectively. Set the reading prompt: "Identify the reading in the digital display frame of the meter, and the output format is = value", and import the reading prompt words and the intercepted digital display frame into the multimodal model, and the multimodal model directly recognizes the reading in the digital display frame. For the small pointer dial, the reading of each pointer is identified separately, and then combined to obtain the total reading of the small pointer dial.

[0053] S4. On the edge server, based on the identified meter readings and the unit carbon emission factor corresponding to the energy type, the real-time energy consumption and carbon emissions of each type of energy are calculated according to the following formula.

[0054]

[0055]

[0056] in, V 燃气 is the gas meter reading in cubic meters (m 3 ),W 水 is the water meter reading in tons (t), P 电力 is the meter reading in kilowatt-hours (kWh), Q 燃气 The unit calorific value of natural gas is expressed in megajoules per cubic meter (MJ / m 3 ), F 燃气 is the carbon emission factor of natural gas, expressed in tons of carbon dioxide equivalent / terajoule (tCO2 / TJ), F 水 is the carbon emission factor of water, expressed in kgCO2e / t, F 电力 is the carbon emission factor of electricity, expressed in tons of carbon dioxide per megawatt-hour (tCO2 / MWh).

[0057] Based on the pain points of energy consumption management in the production and construction of building components, the present invention makes full use of artificial intelligence technologies such as multimodal deep learning, information technologies such as the Internet of Things and edge computing, and innovatively proposes a building energy metering and carbon emission monitoring method based on a multimodal model. This method uses a camera to replace manual meter reading, realizes automatic recognition of energy meter readings through a preset multimodal model, and completes the calculation of energy consumption and carbon emissions on site. The data is uploaded to the management platform in real time through the Internet of Things, supporting visual display of energy consumption and abnormal warning. The whole process is automated to improve the efficiency of energy consumption management; localized processing ensures data timeliness; unified platform management maximizes data value. It can effectively guide energy conservation and carbon reduction in the process of building component manufacturing and structural construction, and help the construction industry achieve green and low-carbon transformation.

[0058] In a specific embodiment, the building energy metering and carbon emission monitoring method based on the multimodal model also includes: S5. Uploading the energy consumption and carbon emission data of various energy sources calculated by the edge server to the cloud management platform. The platform receives and aggregates data from different engineering projects, and presents the energy consumption structure and carbon emission level of the engineering projects in real time through a visual interface, providing data support for management decisions. When abnormal data is found, the platform automatically triggers an early warning and notifies the management personnel to handle it in time.

[0059] In a specific embodiment, in step S3, in order to improve recognition accuracy, a review prompt word may be set and the reading may be reviewed using different multimodal models.

[0060] In a specific embodiment, in step S3, different types of meters use corresponding specific prompt words to improve recognition accuracy.

[0061] In a specific embodiment, Figure 2As shown, the identification of the water meter dial pointer in step S3 can be carried out by the following steps: extracting the pointer position by performing color space conversion, threshold segmentation and morphological operations on the image, and calculating the pointer angle to obtain the reading. Specifically, it includes:

[0062] S3-1: Convert the image from RGB space to HSV color space: HSV= f (RGB), where is the color space conversion function;

[0063] S3-2: Extracting the hue channel from the HSV color space H , used for color segmentation;

[0064] S3-3: Select the threshold segmentation of the pointer color area:

[0065] Setting Tuning Parameters T 1 and T 2 To select the pointer color area:

[0066]

[0067] in, T 1 and T 2. Manual adjustment based on the color of the pointer and lighting conditions; for example, select the threshold of the red area, generate a mask image, and adjust the tonal value range parameters. T 1=0.8, T 2=0.1;

[0068] S3-4: Use morphological operations to remove small area noise; A is the minimum area threshold, then:

[0069] Cleaned Mask = g (pointer mask, A );

[0070] in, g is the morphological operation function, A It is a preset minimum area threshold used to filter out noise points; for example, by using morphological operations to remove noise in the mask image and remove connected domains with an area less than 50 pixels;

[0071] S3-5: Calculate the center of mass of the pointer C ( x , y ) and direction; center of mass C It is given by the average position of pixels within the region:

[0072] ;

[0073] in, Nis the total number of pixels in the area;

[0074] S3-6: Calculate pointer angle and reading R ;

[0075] Calculate the pointer angle θ : d x = x 最远点 - C x ; d y = C y - y 最远点 ; θ =arctan2( d y , d x );

[0076] Set the pointer angle θ Convert to clockwise from directly above: θ' =(90°- θ )mod360°;

[0077] reading R Calculated as: R = θ' / 36°.

[0078] As an example, when calculating the reading based on the pointer angle, the small dial ranges from 0 to 10, with every 36 degrees representing 1 scale.

[0079] In a specific embodiment, in step S3-3, a local adaptive threshold algorithm is used to improve the reliability of pointer recognition so as to be applicable to images under different lighting conditions;

[0080] ；

[0081] in, x and y represents the pixel coordinates in the image, k 1 is the window size used in the local adaptive threshold algorithm, which can be set to an odd value according to actual needs, such as 3, 5, 7, ..., 21. k 2 is used to adjust the offset of the final threshold, with a value range of 0 to 255; Indicates , Centered at The scope of the local area. Here Refers to the HSV color space Channel, namely the hue channel, is used for color segmentation and pointer identification, the same as before.

[0082] Then, based on the selected threshold T local Get the pointer color area:

[0083] .

[0084] In a specific embodiment, Figure 3 As shown, according to the identified meter types and readings, the gas meter reading is 0.917m³, the water meter reading is 0.9976m³, and the electricity meter reading is 13.0129kWh; thus, the current total carbon emissions are obtained, multiplied by the carbon emission factor of the corresponding energy, to obtain the carbon emissions. In this embodiment, according to the "Building Carbon Emission Calculation Standard (GB / T51366-2019)", the unit calorific value carbon emission factor of natural gas is 55.54tCO2 / TJ, and the carbon emission factor of water is 0.168kgCO2e / t; the carbon emission factor of electricity is taken as the national power grid average value of 0.5703tCO2 / MWh in 2022. Assume that the low calorific value of natural gas is 38.931MJ / m 3 The calculation steps are shown in Table 1-4, and the current total carbon emissions are 9.5717 kgCO2e.

[0085] Table 1 Calculation of natural gas carbon emissions

[0086]

[0087] Table 2 Calculation of electricity carbon emissions

[0088]

[0089] Table 3 Calculation of water carbon emissions

[0090]

[0091] Table 4 Calculation of total carbon emissions

[0092]

[0093] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A building energy metering and carbon emission monitoring method based on a multimodal model, characterized in that: The steps include: S1. During the production and construction of building components, the dial image of the meter is collected by the camera and the image signal is transmitted to the edge server. The meter includes an electricity meter, a gas meter, and a water meter; S2. Input the dial image into the pre-trained multimodal deep learning model API and output the type of the meter; S3. Identify the meter reading according to the meter type; S4. Based on the identified meter readings and the unit carbon emission factor corresponding to the energy type, the edge server calculates the real-time energy consumption of each energy type. E 总 and carbon emissions C 总 ; If the meter is a gas meter or an electricity meter, the digital display frame area is automatically intercepted, a reading prompt is set, the reading prompt and the intercepted digital display frame are imported into the multimodal model, and the multimodal model directly recognizes the reading in the digital display frame; If the meter type is a water meter, set the dial recognition prompt word, and use the multimodal model to give the number and type of dial partitions and small pointers; then intercept the digital display frame and small pointer dial of the water meter respectively; Set the reading prompt words, and import the reading prompt words and the intercepted digital display frame into the multimodal model, so that the multimodal model can directly identify the reading in the digital display frame; for the small pointer dial, identify the reading of each pointer separately, and then combine them to obtain the total reading of the small pointer dial; When the meter type is a water meter, the pointer position is extracted by performing color space conversion, threshold segmentation and morphological operations on the image, and the pointer angle is calculated to obtain the reading, including: S3-1. Convert the image from RGB space to HSV space: HSV= f (RGB), where f is the color space conversion function; S3-2. Extracting hue channel from HSV color space H , used for color segmentation; S3-3. Select the threshold segmentation of the pointer color area: Setting the tuning parameters T 1 and T 2 To select the pointer color area: ； S3-4. Use morphological operations to remove small area noise; A is the minimum area threshold, then: Cleaned Mask = g (pointer mask, A ); in, g is the morphological operation function, A It is a preset minimum area threshold used to filter out noise points; S3-5. Calculate the center of mass of the pointer C ( x , y ) and direction; center of mass C It is given by the average position of pixels within the region: ； in, N is the total number of pixels in the area; S3-6. Calculate the pointer angle θ And converted to readings: d x = x 最远点 - C x ; d y = C y - y 最远点 ; θ =arctan2( d y , d x ); Convert to clockwise from directly above: θ' =(90°- θ )mod360°; reading R Calculated as: .

2. The building energy metering and carbon emission monitoring method based on multimodal model as claimed in claim 1, characterized in that: ； ； in, V 燃气 is the gas meter reading, in m 3 , W 水 is the water meter reading, in t, P 电力 is the meter reading in kWh, Q 燃气 is the unit calorific value of natural gas, in MJ / m 3 , F 燃气 is the carbon emission factor of natural gas, in tCO2 / TJ, F 水 is the carbon emission factor of water, in kgCO2e / t, F 电力 is the carbon emission factor of electricity, with the unit of tCO2 / MWh.

3. The building energy metering and carbon emission monitoring method based on multimodal model as claimed in claim 1, characterized in that: After step S4, the following steps are also included: S5. Upload the energy consumption and carbon emission data of various types calculated by the edge server to the cloud management platform.

4. The building energy metering and carbon emission monitoring method based on multimodal model as claimed in claim 1, characterized in that: Step S3-3 is replaced by using a local adaptive threshold algorithm to improve the reliability of pointer recognition, specifically: the threshold interval is recorded as T local (x,y), where ； in, x and y Represents the pixel coordinates in the image; k 1 is the window size used in the local adaptive thresholding algorithm, k 2 is used to adjust the offset of the final threshold; Then get the pointer color area according to the selected threshold interval: 。

Images