Multi-modal data fusion method and system of energy-saving building data sensing system

Abstract

The invention relates to the technical field of data processing, in particular to a multi-modal data fusion method and system for an energy-saving building data sensing system, and the method comprises the steps: obtaining environment state data, a current scene time period and a historical scene time period of each dimension at each moment; according to the difference of the environment state data at adjacent moments under the same dimension in the current scene time period, combining the feature difference between the current scene time period and the historical scene time period to obtain an environment unique index; according to the similarity of the environment state data of the historical scene time period and the current scene time period in the same dimension, obtaining a reference scene time period in combination with the unique indexes of the environment; and according to the environment state data of the reference scene time period and the current scene time period in each dimension, combining with the environment unique index to construct a fusion weight, and fusing the multi-modal data of the current scene time period. According to the invention, the fusion weight is highly matched with the current scene feature, and accurate fusion of multi-modal data can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-modal data fusion method and system of an energy-saving building data perception system. BACKGROUND

[0002] An energy-saving building is a building that reduces building energy consumption through climate adaptability design combined with planning layout, and by implementing preset energy-saving strategies. It reduces energy consumption and improves energy utilization efficiency through various technical means. The core of an energy-saving building is to build a closed-loop control system that can accurately perceive and automatically control. In order to accurately perceive each device and energy system in the building, real-time collection, transmission and processing of environmental conditions during building operation are required through Internet of Things, sensor network and other technical means. However, the building environment is a highly complex and dynamic system, involving energy consumption status, environmental parameters, and personnel activities of various devices and other factors. These data will generate various modal information during collection. In order to accurately assess the environmental conditions in the building, it is necessary to fuse these different modal data. By fusing multi-modal data, data prediction, data feature extraction and other operations can be realized to accurately assess the environmental conditions in the building.

[0003] The existing multi-modal data fusion method sets fixed weights for different data dimensions according to experience to achieve the purpose of weighted fusion of modal data. However, the data perceived in the building is complex and dynamic, which may result in different feature performances of the environmental state of the same space scene at different time sequences, making the fixed weight fusion result poor in scene description effect. SUMMARY

[0004] In order to solve the technical problem that the existing multi-modal data fusion method is affected by the scene environment, making the fixed weight fusion result poor in scene description effect, the purpose of the present application is to provide a multi-modal data fusion method and system of an energy-saving building data perception system, and the technical solution adopted is as follows: In the first aspect, the present application provides a multi-modal data fusion method of an energy-saving building data perception system, comprising: In the energy-saving building scene, the environmental state data of each dimension at each time is obtained; wherein the time includes the current time and the historical time before the current time; Divide the current scene time period and the historical scene time period; according to the difference of the environmental state data of the same dimension at adjacent time in the current scene time period, combine the feature difference between the current scene time period and the historical scene time period, and obtain the environmental unique index of the current scene time period in each dimension; According to the similarity of the environment state data of each historical scene time period and the current scene time period at the same dimension corresponding time, combined with the environment unique index of the current scene time period at the corresponding same dimension, the historical scene time period is screened to obtain a reference scene time period; According to the difference of the environment state data of each reference scene time period and the current scene time period in each dimension, combined with the environment unique index, the fusion weight of the current scene time period in each dimension is constructed, and the multi-modal data of the current scene time period is fused.

[0005] Preferably, the environment unique index of the current scene time period in each dimension is obtained according to the difference of the environment state data of adjacent time in the same dimension in the current scene time period, combined with the feature difference between the current scene time period and the historical scene time period, and specifically includes: The environment stability index of the current scene time period in each dimension is obtained according to the similarity of the environment state data between each two adjacent time in each dimension in the current scene time period; The environment unique index of the current scene time period in each dimension is obtained according to the difference distance between the environment stability index of the current scene time period and each historical scene time period in the same dimension.

[0006] Preferably, the environment stability index of the current scene time period in each dimension is obtained according to the similarity of the environment state data between each two adjacent time in each dimension in the current scene time period, and specifically includes: The environment state data is an environment state vector composed of environment values of a plurality of positions in each dimension at each time; In the current scene time period, each environment stability index of the current scene time period in the arbitrary dimension is determined based on the cosine similarity of the environment state vectors between each time and the adjacent next time.

[0007] Preferably, the environment unique index of the current scene time period in each dimension is obtained according to the difference distance between the environment stability index of the current scene time period and each historical scene time period in the same dimension, and specifically includes: Based on all the environment stability indexes in each dimension in each scene time period, an environment change sequence of each scene time period in each dimension is formed; wherein, the scene time period includes the current scene time period and each historical scene time period; The DTW distance between the environment change sequence of the current scene time period and each historical scene time period in the same dimension is taken as the change difference factor of the current scene time period and each historical scene time period in each dimension; The mean of the change difference factor of the current scene time period and all historical scene time periods in the same dimension is taken as the environment unique indicator of the current scene time period in each dimension.

[0008] Preferably, the reference scene time period is obtained by screening the historical scene time periods according to the similarity of the environment state data of each historical scene time period and the current scene time period at the same dimension and the environment unique indicator of the current scene time period in the same dimension, specifically including: The similarity weight value corresponding to each dimension of each historical scene time period is determined based on the similarity between the environment state data of each historical scene time period and the current scene time period in the same dimension. The correlation indicator between each historical time period and the current scene time period is obtained according to the similarity weight value corresponding to each dimension of each historical scene time period and the environment unique indicator of the current scene time period in the same dimension. The reference scene time period is obtained by screening all historical scene time periods according to the correlation indicator.

[0009] Preferably, the correlation indicator between each historical time period and the current scene time period is obtained according to the similarity weight value corresponding to each dimension of each historical scene time period and the environment unique indicator of the current scene time period in the same dimension, specifically including: For any one historical scene time period, the environment unique indicator of the current scene time period in the same dimension is weighted using the similarity weight value corresponding to each dimension, and the normalized result of the weighted sum of all dimensions is taken as the correlation indicator between the any one historical scene time period and the current scene time period.

[0010] Preferably, the reference scene time period is obtained by screening all historical scene time periods according to the correlation indicator, specifically including: The historical scene time period corresponding to the correlation indicator greater than or equal to the preset similarity threshold is taken as the reference scene time period.

[0011] Preferably, the fusion weight of the current scene time period in each dimension is constructed according to the difference of the environment state data of each reference scene time period and the current scene time period in each dimension and the environment unique indicator, specifically including: The change consistency factor is obtained by negatively correlating the change difference factor. The stability coefficient of each dimension is obtained based on the mean and standard deviation of the change consistency factor corresponding to each dimension of all reference scene time periods, and the product between the stability coefficient of each dimension and the environment unique indicator of the current scene time period in each dimension is normalized to obtain the fusion weight of the current scene time period in each dimension.

[0012] Preferably, the dividing the current scene time period and the historical scene time period specifically comprises: An arbitrary time is recorded as a first time, and a next time adjacent to the first time is recorded as a second time; Based on the difference between the environment state vectors of the first time and the second time in each dimension, a change factor of the first time and the second time in each dimension is determined; and the average of the change factors of the first time and the second time in all dimensions is taken as a change index between the first time and the second time; When the change index is greater than or equal to a preset change threshold, the second time is taken as a change time; all times between every adjacent two change times constitute a scene time period; a scene time period in which the current time is located is a current scene time period, and a scene time period in which the historical time is located is a historical scene time period.

[0013] In a second aspect, the present application provides a multi-modal data fusion system of an energy-saving building data perception system, which is used to realize the steps of a multi-modal data fusion method of an energy-saving building data perception system, and specifically comprises: A data acquisition module is used to acquire environment state data in each dimension at each time in an energy-saving building scene; wherein the time includes a current time and a historical time before the current time; A uniqueness analysis module is used to divide a current scene time period and a historical scene time period; based on the difference between the environment state data of adjacent times in the same dimension in the current scene time period, and in combination with the feature difference between the current scene time period and the historical scene time period, an environment uniqueness index of the current scene time period in each dimension is obtained; A scene screening module is used to screen historical scene time periods to obtain reference scene time periods according to the similarity of the environment state data of the same dimension corresponding times between each historical scene time period and the current scene time period, and in combination with the environment uniqueness index of the current scene time period in the corresponding same dimension; A data fusion module is used to construct a fusion weight of the current scene time period in each dimension according to the difference of the environment state data of each reference scene time period and the current scene time period in each dimension, and in combination with the environment uniqueness index, and to fuse the multi-modal data of the current scene time period.

[0014] The embodiments of the present application have at least the following beneficial effects: The application firstly collects multi-dimensional environment state data, to provide a data basis for subsequent multi-modal data fusion analysis process. Then, by locking the current scene and the historical scene in the dynamic building environment in the time dimension, the unique characteristics of the current scene time period in the environment state data in each dimension can be represented by using the unique indicators of the environment under the current environment scene. Further, by comprehensively considering the similarity between the historical scene and the current scene, and the uniqueness of the current scene, the historical scene is filtered, and the reference scene is accurately filtered to find the historical data benchmark of the current scene. Finally, by analyzing the difference between the reference scene and the current scene, and the uniqueness of the current scene, the exclusive fusion weight of the current scene is constructed, to ensure that the fusion weight is highly matched with the characteristics of the current scene, and the accurate fusion of multi-modal data can be realized. Reliable basis is provided for the environment evaluation and equipment control of energy-saving buildings. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0016] Figure 1 is a step flow chart of a multi-modal data fusion method of an energy-saving building data sensing system provided by the present application; Figure 2 is a sub-step flow chart of step S200 provided by the present application; Figure 3 is a step flow chart of a reference scene time period acquisition method provided by the present application; Figure 4 is a structural schematic diagram of a multi-modal data fusion system of an energy-saving building data sensing system provided by the present application. DETAILED DESCRIPTION

[0017] In order to further illustrate the technical means and effects taken by the present application to achieve the predetermined invention purpose, the specific embodiments, structure, features and effects of a multi-modal data fusion method and system of an energy-saving building data sensing system according to the present application are described in detail as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0019] The application provides a multi-modal data fusion method and system of an energy-saving building data sensing system.

[0020] Please refer to Figure 1 which shows a step flowchart of a multi-modal data fusion method of an energy-saving building data sensing system according to an embodiment of the application, and the method comprises the following steps. Step S100, in an energy-saving building scene, acquiring environmental state data in each dimension at each time point; wherein the time point comprises a current time point and a historical time point.

[0021] In the energy-saving building scene, sensors are arranged at each monitoring position to monitor the environmental changes of each position in the energy-saving building in different data category dimensions.

[0022] In an embodiment, the monitoring position can include a load-bearing wall, a building passage, a building house, a building corner and the like of the energy-saving building. In other embodiments, the implementer can set up the scene according to the experience of professionals, and can set up according to the experience of professionals, and the purpose is to monitor the environmental state of the building.

[0023] In an embodiment, various sensors are arranged at each monitoring position, and the sensors can include temperature sensors, humidity sensors, illumination sensors, air quality sensors and the like of the environment sensors, which are used to monitor the environmental values at each position of the building. The frequency of data collection of each sensor is pre-set to 1 time / minute, and then a plurality of historical time points before the current time point need to be acquired to provide data reference for the actual scene at the current time point, and the number of historical time points can be set by the implementer according to the specific implementation scene.

[0024] It should be understood that one kind of data value corresponds to one dimension of environmental state data, and then the environmental state data can include temperature, humidity, natural light intensity, CO2 concentration, and the like, and one data corresponds to one dimension.

[0025] In other embodiments, a camera can also be arranged at each monitoring position to monitor personnel activity information at each monitoring position. For example, a video frame image is collected by using the camera, and human behavior information of the video frame image is extracted by using a neural network to form one dimension of data, which will not be introduced too much here. It should be understood that the image data also corresponds to one data dimension. The embodiment mainly analyzes the real environmental state collected by the plurality of sensors, and the implementer can set different data dimensions according to the specific need of extracted feature information in the specific implementation scene.

[0026] Further, the environment values of the same dimension at each monitoring position are included, and in order to more accurately describe the environment at each moment, the sensor data of different positions needs to be integrated. Therefore, in the embodiment, the environment values of several positions in each dimension at each moment are combined to form an environment state vector, and the environment state data in each dimension at each moment is the environment state vector.

[0027] More specifically, for a certain moment, the environment values of a kind of data at all monitoring positions are arranged in a fixed position order to form a vector, which is referred to as an environment state vector. For example, for temperature, the temperature at all monitoring positions at each moment is combined to form a vector, which is the environment state data in the temperature dimension at each moment.

[0028] In the spatial scenario of a building, the building environment is a complex and dynamic system. To achieve accurate perception of various positions and energy systems in the building, various types of data need to be comprehensively and real-time collected. By reasonably deploying various types of sensors at key positions and building areas, data such as environment state, personnel activity, and equipment operation energy consumption can be covered to ensure the comprehensiveness and representativeness of the data. The data collection frequency can ensure the real-time nature of the data, laying a foundation for subsequent dynamic scenario division and accurate data analysis. Ultimately, the environment state data of each dimension collected is the data source for the entire multi-modal data fusion work.

[0029] Step S200, divide the current scenario time period and the historical scenario time period; according to the difference between the environment state data of adjacent moments in the same dimension in the current scenario time period, and in combination with the feature difference between the current scenario time period and the historical scenario time period, obtain the environment unique index of the current scenario time period in each dimension.

[0030] In the spatial scenario of a building, the building environment is complex and dynamic, and the time sequence and the activities in the building will affect the environment state of the building. Therefore, the difference between the environment state data of each dimension at each adjacent moment can be analyzed. When the difference between the environment state data of adjacent moments in the building is small, it indicates that the environment changes slightly or does not change. When the difference between the environment state data of adjacent moments in the building is large, it indicates that the environment changes. Accordingly, the historical time sequence can be divided into multiple time periods, each time period corresponds to an environment scenario, and effective division of the dynamic building environment is achieved, laying a foundation for subsequent targeted analysis of the scenario data.

[0031] Further, the building environment data follows the basic law without external interference, and the change trend changes with external interference. By analyzing the environment change, consistency features and difference features in different environment scenarios, the basic uniqueness of the environment scenario is finally evaluated, which can judge the detection ability of the perception data to the current scenario change. The perception data with high basic uniqueness can better reflect the unique change of the scenario.

[0032] Based on this feature, as shown in Figure 2 The sub-step flowchart of step S200 can be implemented by steps S201 to S203.

[0033] In step S201, all time points are divided into scene time periods according to the difference between the environmental state data in each dimension at each time point and the adjacent time point, and the scene time periods include a current scene time period and a historical scene time period.

[0034] In this embodiment, taking any two adjacent time points as an example, specifically, any one time point is recorded as a first time point, and the next time point adjacent to the first time point is recorded as a second time point.

[0035] Firstly, based on the difference between the environmental state vectors in each dimension at the first time point and the second time point, a change factor in each dimension at the first time point and the second time point is determined.

[0036] Specifically, the Euclidean distance between the environmental state vectors in each dimension at the first time point and the second time point is normalized to obtain the change factor in each dimension at the first time point and the second time point. The normalization method can adopt a max-min normalization method, and the normalization method and the calculation method of the Euclidean distance are known technologies, which will not be described in detail here.

[0037] It should be understood that for each two adjacent time points, one dimension corresponds to one change factor, which reflects the change degree of the environmental state data in the corresponding dimension between the two adjacent time points.

[0038] Secondly, the average of the change factors in all dimensions at the first time point and the second time point is taken as a change index between the first time point and the second time point.

[0039] The change index is a balanced representation of the change degree of the environmental state in all dimensions between the two adjacent time points. The greater the value of the change index between the two adjacent time points, the greater the change of the building environmental state from the first time point to the second time point, and the greater the possibility that the first time point and the second time point are in different environmental states. The smaller the value of the change index between the two adjacent time points, the smaller the change of the building environmental state from the first time point to the second time point, or no change, and the greater the possibility that the first time point and the second time point are in the same environmental state.

[0040] Thirdly, when the change index is greater than or equal to a preset change threshold, the second time point is taken as a change time point.

[0041] In this embodiment, the value of the change threshold is 0.7, which can be set by the implementer according to the specific implementation scenario. When the change index between two adjacent time points is greater than or equal to the change threshold, it indicates that the difference between the environmental state data of the two adjacent time points is large, and the possibility that the two adjacent time points are in different environmental scenes is greater, so the time point with the later time sequence in the adjacent time points is taken as the change time point. When the change index between two adjacent time points is less than the change threshold, it indicates that the difference between the environmental state data of the two adjacent time points is small, and the possibility that the two adjacent time points are in the same environmental scene is greater, so the time point is not marked.

[0042] At this point, all time points can be screened according to the same method, and the change time point is marked, and then the change time point represents the time point at which the environmental scene changes.

[0043] In the fourth step, all time points between each adjacent two change time points constitute a scene time period; the scene time period in which the current time point is located is the current scene time period, and the scene time period in which the historical time point is located is the historical scene time period.

[0044] For example, assuming that there are 20 time points, the 8th time point and the 13th time point are change time points, then the time points between the first time point and the 8th time point constitute a scene time period, the time points between the 8th time point and the 13th time point constitute a scene time period, and the time points between the 13th time point and the 20th time point constitute a scene time period.

[0045] It should be noted that the change time point represents the time point at which the environmental scene changes, so the change time point belongs to the scene time period with the later time sequence in the adjacent two scene time periods, for example, the 8th time point belongs to the second scene time period, and the 13th time point belongs to the third scene time period.

[0046] At this point, the historical time sequence can be divided into multiple scene time periods, each scene time period corresponds to an environmental scene, the dynamic building environment is effectively divided, which lays a foundation for subsequent targeted analysis of scene data, that is, accurately judges the environmental scene in which the current time point is located, selects the data processing and fusion strategy suitable for the current scene, ensures that the fusion result conforms to the actual situation of the current scene, and provides a data basis.

[0047] In step S202, the environmental stability index of the current scene time period in each dimension is obtained according to the similarity of the environmental state data between each two adjacent time points in each dimension in the current scene time period.

[0048] Specifically, under any one dimension in the current scene time period, each environment stability indicator of the current scene time period in the any one dimension is determined based on the cosine similarity of the environment state vectors between each time instant and the adjacent next time instant.

[0049] When the cosine similarity of the environment state vectors between every two adjacent time instants is greater, it indicates that the corresponding environment stability is stronger under the same environment scene, that is, the environment stability indicator has a greater value. Therefore, the environment stability indicator measures the environment stability characteristics between adjacent time instants under the same environment scene.

[0050] As a specific example, taking the tth time instant and the t+1th time instant in the current scene time period as an example, the acquisition method of one environment stability indicator corresponding to two adjacent time instants can be expressed by a formula as follows: wherein, denotes the environment stability indicator corresponding to the tth time instant and the t+1th time instant in the i th dimension in the current scene time period, denotes the environment state vector of the tth time instant in the i th dimension in the current scene time period, denotes the environment state vector of the tth time instant in the i+1th dimension in the current scene time period, denotes the cosine similarity between two vectors, denotes the exponential function with the natural constant e as the base.

[0051] The embodiment realizes nonlinear quantification of stability by twice negative correlation calculation on the cosine similarity, converts the measurement result of the similarity into the evaluation result of the stability, and adapts subsequent uniqueness and scene similarity calculation to filter transient interference.

[0052] It should be noted that each environment stability indicator of each historical scene time period in each dimension can be acquired in the same way, and the environment stability indicator reflects the data stability characteristics between adjacent time instants.

[0053] In step S203, the environment uniqueness indicator of the current scene time period in each dimension is obtained according to the difference distance between the environment stability indicators of the current scene time period and each historical scene time period in the same dimension.

[0054] In the first step, all the environment stability indicators in each dimension in each scene time period are used to form an environment change sequence of each scene time period in each dimension; wherein the scene time period includes the current scene time period and each historical scene time period.

[0055] It should be understood that in each scene time period, every two adjacent time points correspond to an environment stability indicator in each dimension. For each dimension, all environment stability indicators in the scene time period are arranged in time sequence to form an environment change sequence of each scene time period in each dimension.

[0056] In the second step, the DTW distance between the environment change sequence of the current scene time period and the environment change sequence of each historical scene time period in the same dimension is taken as the change difference factor of the current scene time period and each historical scene time period in each dimension.

[0057] Specifically, considering that the time lengths of different scene time periods are different, the embodiment takes the DTW distance as a measurement means of difference distance for feature analysis. The change difference factor represents the difference degree of the environment features of the current scene and the historical scene in the same dimension.

[0058] In the third step, the average value of the change difference factors of the current scene time period and all historical scene time periods in the same dimension is taken as the environment unique indicator of the current scene time period in each dimension.

[0059] For any dimension, the difference degrees of the environment features between the current scene time period and all historical scene time periods are integrated, and the average value is used to reflect the balance. If the environment difference degrees between the current scene time period and all historical scene time periods are large, it means that the current scene time period has a higher unique degree of basic features in the dimension, which means that the data is more likely to detect the change in the current scene.

[0060] If the environment difference degrees between the current scene time period and all historical scene time periods are small, it means that the current scene time period does not have data difference in the dimension, and further means that the current scene time period has a lower unique degree of basic features in the dimension.

[0061] In step S300, according to the similarity of the environment state data of each historical scene time period and the current scene time period in the same dimension at the corresponding time, and in combination with the environment unique indicator of the current scene time period in the corresponding same dimension, the historical scene time period is filtered to obtain a reference scene time period.

[0062] The behavior purpose of the user in the building is relatively fixed, and the equipment demand and use condition in the building when the same purpose is completed are approximately close, so the reference scene similar to the current scene can be filtered by analyzing the perceptual similarity, change description consistency feature, and scene similarity feature, to provide a reference basis for the correlation between the data features and the current scene for subsequent analysis.

[0063] As a specific example, as shown in FIG. 6, the reference scene time period is filtered according to the similarity of the perceptual similarity, the change description consistency feature, and the scene similarity feature. Figure 3As shown, the reference scenario time period acquisition method can be implemented by steps S301 to S303.

[0064] In step S301, based on the similarity between the environment state data of each historical scenario time period and the current scenario time period in the same dimension, the similarity weight value corresponding to each dimension of each historical scenario time period is determined.

[0065] The greater the similarity between the environment state data of two different scenario time periods in the same dimension, the more similar the data change rules of the two environment scenarios in the same dimension, and the higher the consistency of the data change description of the two environment scenarios when evaluating the similarity and correlation between the two environment scenarios. The smaller the similarity between the environment state data of two different scenario time periods in the same dimension, the greater the difference in the data change rules of the two environment scenarios in the same dimension, and the lower the consistency of the data change description of the two environment scenarios when evaluating the similarity and correlation between the two environment scenarios.

[0066] As a specific example, for any dimension, the environment state data of all time points in each scenario time period is arranged in time sequence to form the environment state sequence corresponding to each scenario time period, wherein the scenario time period includes the current scenario time period and the historical scenario time period. The DTW distance of the environment state sequence of each historical scenario time period and the current scenario time period in the same dimension is processed in a negative correlation manner to obtain the similarity weight value corresponding to each dimension of each historical scenario time period and the current scenario time period.

[0067] More specifically, taking any historical scenario time period and any dimension as an example, the similarity weight value acquisition method can be represented by the following formula: wherein represents the similarity weight value corresponding to the nth historical scenario time period and the current scenario time period in the ith dimension, represents the DTW distance of the environment state sequence of the nth historical scenario time period and the current scenario time period in the ith dimension, represents the exponential function with natural constant e as the base.

[0068] The greater the value of the historical scenario time period and the current scenario time period, the greater the difference between the two environment scenarios in the same dimension, and the greater the negative correlation processing result The similarity between the two environment scenario data is evaluated.

[0069] In step S302, according to the similarity weight corresponding to each dimension of each historical scene time period, and in combination with the environment unique indicator of the current scene time period in the same dimension, an association indicator between each historical time period and the current scene time period is obtained.

[0070] Specifically, for any one historical scene time period, the environment unique indicator of the current scene time period in the same dimension is weighted by using the similarity weight corresponding to each dimension, and the normalization result of the weighted sum result of all dimensions is taken as the association indicator between the any one historical scene time period and the current scene time period.

[0071] As a specific example, taking the nth historical scene time period as an example, the association indicator between the nth historical scene time period and the current scene time period can be expressed by a formula as follows: In the formula, represents the association indicator between the nth historical scene time period and the current scene time period, represents the similarity weight corresponding to the ith dimension between the nth historical scene time period and the current scene time period, represents the environment unique indicator of the current scene time period in the ith dimension, N represents the total number of historical scene time periods, and represents a normalization function.

[0072] It should be noted that the normalization method is a known technology, and the implementer can select according to the specific implementation scene, for example, the maximum and minimum normalization method can be selected.

[0073] Environment unique indicator reflects the environment scene recognition degree of the ith dimension in the current scene, and also reflects whether the data of the dimension can effectively distinguish the current scene from other scenes.

[0074] For each dimension, the greater the data feature similarity between the historical scene and the current scene, that is, the greater the corresponding similarity weight, and the greater the recognition degree of the dimension to the current scene, and the higher the contribution value of the dimension to the measurement of the similarity between the two scenes.

[0075] represents the weighted sum result of by using the similarity weight as the weight, represents the cumulative sum of the weights of all dimensions, and represents the weighted sum result of by using the weight as the weight, ​​​​​The multi-dimensional comprehensive contribution is obtained by synthesizing the contribution values of all dimensions, and reflects the overall similarity and relevance between the historical scene and the current scene.

[0076] In step S303, the reference scene time period is obtained by screening all historical scene time periods according to the correlation index.

[0077] Specifically, the historical scene time period corresponding to the correlation index greater than or equal to the preset similarity threshold is taken as the reference scene time period. The value of the similarity threshold is 0.7, and the implementer can set it according to the specific implementation scene.

[0078] When the correlation index between the historical scene time period and the current scene time period is greater than or equal to the similarity threshold, it means that the more similar the comprehensive performance of the data in each dimension between the historical scene time period and the current scene time period is, the greater the possibility that they belong to the same scene is, and thus the higher the value of taking the historical scene time period as a reference is. When the correlation index between the historical scene time period and the current scene time period is less than the similarity threshold, it means that the more dissimilar the comprehensive performance of the data in each dimension between the historical scene time period and the current scene time period is, the smaller the possibility that they belong to the same scene is, and thus the lower the value of taking the historical scene time period as a reference is, so the historical scene time period is not further analyzed.

[0079] In step S400, the fusion weight of each dimension of the current scene time period is constructed according to the difference of the environmental state data of each reference scene time period and the current scene time period in each dimension, and the environmental unique index is combined to fuse the multi-modal data of the current scene time period.

[0080] In the similar scene, if the data fluctuation is similar and obviously different from other scenes, the data may be unique to the current scene. Based on this, by analyzing the difference, consistency of the environmental state data of each dimension between the reference scene and the current scene, and the basic uniqueness of the current scene, the possibility of each dimension of the environmental state data belonging to the current scene can be quantified, and the possibility of the dimension where various data belong to the scene can be analyzed to construct a more accurate fusion weight that conforms to the scene characteristics.

[0081] Firstly, the change difference factor is negatively correlated to obtain a change consistency factor.

[0082] Specifically, the change difference factor of each dimension of the current scene time period and each historical scene time period represents the difference degree of the environment features of the current scene and the historical scene in the same dimension, and the change consistency factor obtained by negatively correlating the change difference factor reflects the similarity degree of the environment features of the current scene time period and each historical scene time period in the same dimension.

[0083] In the embodiment, the negative exponential function is adopted in the form of , and the change difference factor of the i th dimension of the previous scene time period and the n th historical scene time period is negatively correlated, , where e represents the natural constant. represents the i th dimension of the current scene time period, represents the i th dimension of the n th historical scene time period.

[0084] At this point, the change consistency factor of each dimension of the current scene time period and each historical scene time period reflects the data feature consistency size between the two environment scenes.

[0085] Secondly, the stability coefficient of each dimension is obtained based on the mean and standard deviation of the change consistency factors of all reference scene time periods in each dimension, and the product of the stability coefficient of each dimension and the environment unique index of the current scene time period in each dimension is normalized to obtain the fusion weight of the current scene time period in each dimension.

[0086] As a specific example, the calculation formula of the fusion weight of the i th dimension can be represented as , where represents the fusion weight of the i th dimension, represents the standard deviation of the change consistency factors of the current scene time period and all reference scene time periods in the i th dimension, represents the mean of the change consistency factors of the current scene time period and all reference scene time periods in the i th dimension, represents the change consistency factor corresponding to the i th dimension; represents the environment unique index of the current scene time period in the i th dimension, represents the stability coefficient of the i th dimension, represents the exponential function with the natural constant e as the base, is a normalization function, which can be processed by maximum and minimum normalization, which is not limited here.

[0087] In the embodiment, the ratio of the standard deviation and the mean of the change consistency factor corresponding to each dimension of all reference scene time periods is negatively correlated to obtain the stability coefficient of each dimension.

[0088] The ratio of the standard deviation and the mean The dispersion coefficient of the change consistency factor of the i-th dimension between the current scene time period and all reference scene time periods, which reflects the dispersion degree of the change consistency factor. The greater the mean of the change consistency between the current scene and the reference scene, the smaller the standard deviation, and the greater the value of the stability coefficient of the i-th dimension. It is explained that the current scene and the reference scene have greater overall similarity in the i-th dimension and have relatively stable fluctuations, and it is further explained that the data fluctuation of the i-th dimension is the commonality of similar scenes, which has the basis to become the data exclusive feature of the i-th dimension.

[0089] At the same time, the environment unique index reflects the data recognition degree of the i-th dimension in the current scene. The greater the value of the environment unique index, the higher the data comprehensive similarity and stability in the i-th dimension, and the higher the recognition degree of the current scene, and then the possibility that the data feature fluctuation of the i-th dimension is the exclusive feature of the current scene is greater.

[0090] The product of the two reflects the possibility that the data feature of each dimension belongs to the current scene, and the feature possibility index can be converted into a unified scale fusion weight through maximum and minimum value normalization. The size of the fusion weight directly reflects the importance of the data feature of each dimension in constructing the current scene feature. The greater the weight, the greater the contribution of the data feature of the dimension to the scene feature, which provides a quantitative basis for determining the importance of each data in subsequent multi-modal data fusion.

[0091] Finally, by using the fusion weight of each dimension, multi-dimensional data fusion can be realized, and then data prediction, data feature extraction and other purposes can be realized. This is a well-known technology in the field of multi-dimensional data fusion. For example, by means of a neural network model with cross attention mechanism, the multi-dimensional environmental state data in the current scene time period where the current time is located is extracted and fused, wherein the attention weight of the neural network model adopts the fusion weight of each dimension described above.

[0092] ​To sum up, the embodiment of the present application firstly accurately judges the current environment scene, selects the data processing and fusion strategy suitable for the scene, ensures that the fusion result meets the actual situation of the current scene, locks the stability of the dynamic building environment through the time dimension, and ensures that the fusion weight is highly matched with the current scene characteristics. Secondly, the fusion weight is determined based on the relevance of the data characteristics and the current scene. In the neural network model of the cross attention mechanism, the fusion weight can make the model pay more attention to the important perception data in the feature extraction and fusion process, reduce the interference of irrelevant or secondary data, thereby improving the accuracy of multi-modal data fusion, and finally construct a fusion result that can accurately reflect the characteristics of the current scene, providing a reliable basis for the environment evaluation and equipment control of energy-saving buildings.

[0093] As shown in Figure 4 The embodiment of the present application also provides a multi-modal data fusion system of an energy-saving building data perception system, which is used to realize the steps of a multi-modal data fusion method of an energy-saving building data perception system. The multi-modal data fusion system of the energy-saving building data perception system specifically comprises: A data acquisition module is used to acquire the environment state data in each dimension at each time in the energy-saving building scene. The time includes the current time and the previous historical time. A uniqueness analysis module is used to divide the current scene time period and the historical scene time period. According to the difference between the environment state data of adjacent time in the same dimension in the current scene time period, and in combination with the feature difference between the current scene time period and the historical scene time period, the environment unique index of the current scene time period in each dimension is obtained. A scene screening module is used to screen the historical scene time period to obtain the reference scene time period according to the similarity of the environment state data of the same dimension corresponding time of each historical scene time period and the current scene time period, and in combination with the environment unique index of the current scene time period in the corresponding same dimension. A data fusion module is used to construct the fusion weight of each dimension of the current scene time period according to the difference of the environment state data of each reference scene time period and the current scene time period in each dimension, in combination with the environment unique index, and fuse the multi-modal data of the current scene time period.

[0094] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A multimodal data fusion method for an energy-saving building data perception system, characterized in that: The method comprises the following steps: In an energy-saving building scenario, obtain environmental status data in each dimension at each moment; wherein the moment includes the current moment and previous historical moments; Divide the current scene time period into the historical scene time period; based on the difference in environmental status data at adjacent moments in the same dimension within the current scene time period, combined with the feature differences between the current scene time period and the historical scene time period, obtain the unique environmental indicators of each dimension for the current scene time period; Based on the similarity of the environmental status data of each historical scene time period and the current scene time period at the corresponding moment of the same dimension, combined with the unique environmental indicators of the current scene time period in the corresponding same dimension, the historical scene time periods are screened to obtain the reference scene time period; According to the difference in environmental status data in each dimension between each reference scene time period and the current scene time period, combined with the unique environmental indicators, the fusion weight of the current scene time period in each dimension is constructed, and the multimodal data of the current scene time period is fused.

2. The multimodal data fusion method for an energy-saving building data perception system according to claim 1 is characterized in that: The unique environmental indicators of each dimension for the current scene time period are obtained based on the differences in environmental status data at adjacent moments in the same dimension within the current scene time period and the feature differences between the current scene time period and the historical scene time period, specifically including: The environmental stability index in each dimension of the current scene time period is obtained according to the similarity of the environmental state data between every two adjacent moments in each dimension in the current scene time period; According to the difference distance between the environmental stability index of the current scenario time period and each historical scenario time period in the same dimension, the unique environmental index of the current scenario time period in each dimension is obtained.

3. The multimodal data fusion method of the energy-saving building data perception system according to claim 2 is characterized in that: The process of obtaining the environmental stability index in each dimension of the current scene time period according to the similarity of the environmental state data between every two adjacent moments in each dimension in the current scene time period specifically includes: The environmental state data is an environmental state vector composed of environmental values ​​at a plurality of positions in each dimension at each moment; In any dimension within the current scene time period, each environmental stability index in the current scene time period in the any dimension is determined based on the cosine similarity of the environmental state vector between each moment and the next adjacent moment.

4. The multimodal data fusion method for an energy-saving building data perception system according to claim 3 is characterized in that: The method of obtaining the unique environmental index of each dimension of the current scene time period according to the difference between the environmental stability index of the current scene time period and each historical scene time period in the same dimension specifically includes: Based on all environmental stability indicators in each dimension within each scenario time period, an environmental change sequence in each dimension for each scenario time period is constructed; wherein the scenario time period includes the current scenario time period and each historical scenario time period; The DTW distance between the environmental change sequence of the current scene time period and each historical scene time period in the same dimension is used as the change difference factor of the current scene time period and each historical scene time period in each dimension; The mean of the change difference factors between the current scenario time period and all historical scenario time periods in the same dimension is taken as the unique environmental indicator of the current scenario time period in each dimension.

5. The multimodal data fusion method for an energy-saving building data perception system according to claim 1 is characterized in that: The method of filtering the historical scene time periods to obtain the reference scene time period based on the similarity of the environmental status data of each historical scene time period and the current scene time period at the corresponding moment in the same dimension, combined with the unique environmental indicators of the current scene time period in the corresponding same dimension, specifically includes: Based on the similarity between the environmental state data of each historical scene time period and the current scene time period in the same dimension, determine the similarity weight corresponding to each historical scene time period in each dimension; Based on the similarity weight corresponding to each dimension of each historical scene time period, combined with the unique environmental index of the current scene time period corresponding to the same dimension, the correlation index between each historical time period and the current scene time period is obtained; According to the correlation index, all historical scenario time periods are screened to obtain a reference scenario time period.

6. The multimodal data fusion method for an energy-saving building data perception system according to claim 5 is characterized in that: The correlation index between each historical time period and the current time period is obtained based on the similarity weight corresponding to each dimension of each historical time period and the unique environmental index corresponding to the same dimension of the current time period, specifically including: For any historical scene time period, the similarity weight corresponding to each dimension is used to weight the unique environmental indicators of the current scene time period in the same dimension, and the normalized result of the weighted average results of all dimensions is used as the correlation indicator between the any historical scene time period and the current scene time period.

7. The multimodal data fusion method for an energy-saving building data perception system according to claim 5 is characterized in that: The method of screening all historical scenario time periods according to the correlation indicators to obtain a reference scenario time period specifically includes: The historical scenario time period corresponding to the correlation index being greater than or equal to the preset similarity threshold is used as the reference scenario time period.

8. The multimodal data fusion method of the energy-saving building data perception system according to claim 4 is characterized in that: The method of constructing the fusion weight of each dimension of the current scene time period based on the difference in environmental status data of each reference scene time period and the current scene time period in each dimension and combining the unique environmental indicators specifically includes: Performing negative correlation processing on the change difference factor to obtain the change consistency factor; The stability coefficient of each dimension is obtained based on the mean and standard deviation of the change consistency factor corresponding to each dimension of all reference scene time periods. The product of the stability coefficient of each dimension and the unique environmental index of each dimension of the current scene time period is normalized to obtain the fusion weight of each dimension of the current scene time period.

9. The multimodal data fusion method of the energy-saving building data perception system according to claim 3 is characterized in that: The division of the current scene time period and the historical scene time period specifically includes: Record any moment as the first moment, and record the next moment adjacent to the first moment as the second moment; Determine the change factor in each dimension at the first moment and the second moment based on the difference between the environmental state vector at each dimension at the first moment and the second moment; and use the average of the change factors at all dimensions at the first moment and the second moment as the change index between the first moment and the second moment; When the change index is greater than or equal to the preset change threshold, the second moment is taken as the change moment; all moments between each two adjacent change moments constitute a scene time period; the scene time period where the current moment is located is the current scene time period, and the scene time period where the historical moment is located is the historical scene time period.

10. A multimodal data fusion system for energy-saving building data perception system, characterized by: The system is used to implement the steps of a multimodal data fusion method of an energy-saving building data perception system as described in any one of claims 1 to 9. The multimodal data fusion system of the energy-saving building data perception system specifically includes: A data acquisition module is used to obtain environmental status data in each dimension at each moment in an energy-saving building scenario; wherein the moment includes the current moment and previous historical moments; The uniqueness analysis module is used to divide the current scene time period into historical scene time periods. Based on the differences in environmental state data at adjacent moments in the same dimension within the current scene time period and the feature differences between the current scene time period and historical scene time periods, the uniqueness index of the environment in each dimension of the current scene time period is obtained. A scene screening module is used to screen historical scene time periods to obtain reference scene time periods based on the similarity of environmental status data of each historical scene time period and the current scene time period at the corresponding moment in the same dimension, combined with the unique environmental indicators of the current scene time period in the corresponding same dimension; The data fusion module is used to construct the fusion weight of each dimension of the current scene time period based on the differences in the environmental status data of each reference scene time period and the current scene time period in each dimension, combined with the unique environmental indicators, and fuse the multimodal data of the current scene time period.

Images