Intelligent station building multi-sensor fusion monitoring method adopting swan gap system

Abstract

The invention provides an intelligent station building multi-sensor fusion monitoring method adopting a swan-gap system, belongs to the technical field of intelligent monitoring, is used for monitoring the environment, equipment state and safety of an intelligent station building, and comprises the following steps: creating a corresponding virtual equipment interface for a target sensor through an equipment abstraction layer of the swan-gap system; wherein the target sensor is any one of a temperature and humidity sensor, a gas sensor, a smoke sensor, an illumination sensor, a current and voltage sensor, a vibration sensor, a pressure sensor and a positioning sensor; acquiring sensor data of a target sensor in the smart station building based on the virtual device interface by an edge computing node in the swan gap system; after the edge computing node processes the sensor data, analysis and reasoning are carried out based on the processed sensor data, and a reasoning result is obtained; the reasoning result comprises at least one of a prediction problem prompt in the smart station building, an equipment abnormal state prompt in the smart station building and a prevention measure.

Description

Technical Field

[0001] The present invention relates to Internet of Things technology and a multi-sensor fusion monitoring method for a smart station building using the Hongmeng system. Background Art

[0002] Smart substations refer to new types of power facilities that are formed by intelligently transforming traditional substations or distribution rooms through the integration of modern information technology. They are mainly used in the process of power transmission and distribution to ensure the safety, reliability and efficiency of power supply. Smart substations are usually set up in key locations such as urban central areas, industrial parks, and transportation hubs.

[0003] With the development of IoT technology, more and more smart stations are beginning to introduce IoT-based monitoring solutions. These solutions can use wireless sensor networks (WSN), Zigbee, LoRaWAN and other communication protocols to connect various sensors to the Internet and realize data remote transmission and cloud server processing.

[0004] Although the existing monitoring systems can achieve remote management and maintenance of power equipment to a certain extent, in actual applications, they face higher security risks due to the large amount of network communications involved. In addition, in some cases, the network bandwidth may not be sufficient to support the real-time transmission of large amounts of data, resulting in lags and delays in the remote management of power equipment. Summary of the invention

[0005] The present invention provides a multi-sensor fusion monitoring method for a smart station building using the Hongmeng system, so as to solve the problem of lag and delay in the remote management and maintenance of power equipment by the existing monitoring system.

[0006] In the first aspect, the present invention provides a multi-sensor fusion monitoring method for a smart station house using the Hongmeng system, which is used to monitor the environment, equipment status and safety of the smart station house, wherein the smart station house is provided with a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor and a positioning sensor, and the method comprises: Creating a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of the temperature and humidity sensor, the gas sensor, the smoke sensor, the light sensor, the current and voltage sensor, the vibration sensor, the pressure sensor, and the positioning sensor; The edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station based on the virtual device interface; After processing the sensor data, the edge computing node performs analysis and reasoning based on the processed sensor data to obtain a reasoning result; the reasoning result includes at least one of a predicted problem prompt in the smart station house, an abnormal status prompt of equipment in the smart station house, and preventive measures.

[0007] In a possible design of the first aspect, creating a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system includes: The device abstraction layer of the Hongmeng system assigns a corresponding identifier to the target sensor, and completes the registration of the target sensor in the Hongmeng system according to the identifier; Configuring attribute information of the target sensor through the device abstraction layer; the attribute information at least includes a sampling frequency and an alarm condition of the target sensor; The device abstraction layer creates a corresponding virtual device interface for each target sensor, and the virtual device interface is used to provide read and write functions and configuration functions.

[0008] In a possible design of the first aspect, the edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station based on the virtual device interface, including: The edge computing node receives the sensor data regularly sent by the target sensor in the smart station house through the built-in communication module based on the virtual device interface; or The edge computing node sends a data collection request to the target sensor in the smart station house based on the virtual device interface; The target sensor collects the sensor data according to the data collection request, and sends the sensor data to the edge computing node through a built-in communication module.

[0009] In a possible design of the first aspect, after the edge computing node processes the sensor data, it performs analysis and reasoning based on the processed sensor data to obtain a reasoning result, including: The edge computing node pre-processes the sensor data; The edge computing node performs spatial alignment processing on the pre-processed sensor data; The edge computing node inputs the sensor data after spatial alignment processing into the multimodal data reasoning model to obtain the reasoning result.

[0010] In a possible design of the first aspect, the edge computing node inputs the sensor data after the spatial alignment processing into the multimodal data reasoning model to obtain the reasoning result, including: The data input layer of the multimodal data reasoning model receives the sensor data and transmits the sensor data to the feature extraction layer of the multimodal data reasoning model; The feature extraction layer extracts corresponding data features from the sensor data and transmits the data features to the hidden layer of the multimodal data inference model; The hidden layer performs splicing processing on the data features to obtain fused sensor data features, and transmits the fused sensor data features to the fully connected layer of the multimodal data inference model; The fully connected layer generates a sensor feature representation corresponding to the fused sensor data feature according to the fused sensor data feature, and transmits it to the output layer of the multimodal data inference model; The output layer generates corresponding inference results according to the sensor feature representation.

[0011] In a possible design of the first aspect, after the edge computing node obtains sensor data of the target sensor in the smart station house based on the virtual device interface, the method further includes: When the edge computing node determines that the sensor data exceeds a preset threshold, it triggers an alarm and executes a preset emergency processing operation.

[0012] In a possible design of the first aspect, the method further includes: The edge computing module of the Hongmeng system monitors the storage usage of the edge computing node in real time; The edge computing module dynamically allocates storage resources to the edge computing node according to the storage usage.

[0013] In a possible design of the first aspect, the method further includes: The edge computing node transmits the sensor data and the reasoning result to the central control device of the Hongmeng system through distributed soft bus technology; The central control device continuously optimizes and updates the multimodal data reasoning model deployed on the edge computing node based on the sensor data and the reasoning result.

[0014] In a possible design of the first aspect, the method further includes: The central control device receives a user's viewing request; the viewing request is input based on a user monitoring interface provided by the central control device; The central control device displays the operating status of the smart station to the user according to the user's viewing request, and the operating status includes the sensor data, the terminal device status in the smart station and the alarm information.

[0015] In a possible design of the first aspect, the method further includes: The central control device receives a control instruction from a user; the control instruction is input based on a user control interface provided by the central control device; The central control device sends an action command to the edge computing node corresponding to the control instruction or the terminal device in the smart station house according to the control instruction, and the action command is used to instruct the edge computing node or the terminal device to perform a corresponding action.

[0016] In a second aspect, the present invention provides a smart station multi-sensor fusion monitoring system using the Hongmeng system, which is used to monitor the environment, equipment status and safety of the smart station. The smart station is provided with a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor and a positioning sensor. The system includes: a device abstraction layer of the Hongmeng system, and multiple edge computing nodes; The device abstraction layer of the Hongmeng system is used to create a corresponding virtual device interface for the target sensor; wherein the target sensor is any one of the temperature and humidity sensor, the gas sensor, the smoke sensor, the light sensor, the current and voltage sensor, the vibration sensor, the pressure sensor, and the positioning sensor; The edge computing node in the Hongmeng system is used to obtain sensor data of the target sensor in the smart station based on the virtual device interface; After processing the sensor data, the edge computing node performs analysis and reasoning based on the processed sensor data to obtain a reasoning result; the reasoning result includes at least one of a predicted problem prompt in the smart station house, an abnormal status prompt of equipment in the smart station house, and preventive measures.

[0017] In a third aspect, the present invention provides an electronic device, including a memory and a processor; The memory is connected to the processor and is used for storing programs; The processor is used to implement a multi-sensor fusion monitoring method for a smart station using the Hongmeng system as described in any one of the first aspects of the present application by running a program in the memory.

[0018] In a fourth aspect, the present invention provides a computer program product, comprising computer program instructions. When the computer program instructions are executed by a processor, the processor executes a multi-sensor fusion monitoring method for a smart station using the Hongmeng system as described in any one of the first aspects of the present invention.

[0019] In a fifth aspect, the present invention provides a chip comprising a processor and a data interface, wherein the processor reads and runs a program stored in a memory through the data interface to execute a multi-sensor fusion monitoring method for a smart station building using the Hongmeng system as described in any one of the first aspects of the present invention.

[0020] In the sixth aspect, the present invention provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, a multi-sensor fusion monitoring method for a smart station building using the Hongmeng system is implemented as any one of the items in the first aspect of the present application.

[0021] In the above scheme, a corresponding virtual device interface is created for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor, and a positioning sensor; the edge computing node in the Hongmeng system obtains the sensor data of the target sensor in the smart station based on the virtual device interface; after the edge computing node processes the sensor data, it analyzes and infers based on the processed sensor data to obtain the inference result; the inference result includes at least one of the prediction problem prompts in the smart station, the abnormal state prompts of the equipment in the smart station, and the preventive measures. It can be seen that in the technical scheme of the present application, the Hongmeng system is adopted to realize the monitoring of the multi-sensor fusion of the smart station. Based on the powerful multi-device collaboration capability of the Hongmeng system, the sensor data collected by different sensors in the smart station can be processed locally through the edge computing node, which means that the sensor data does not need to be transmitted to the cloud server or the central control device, and can be processed in real time on the local device, reducing the delay and improving the response speed. The edge computing node enables the Hongmeng system to effectively fuse data from multiple sensors. Through data fusion algorithms, data from different sensors can be merged and analyzed, redundant or irrelevant information can be removed, and valuable feature data can be extracted. This is very important for monitoring systems because it can provide a more comprehensive and accurate perception of environmental conditions. For example, the combination of data from temperature and humidity sensors and gas sensors can achieve accurate monitoring of the station environment and automatically trigger emergency response measures. Equipment in smart stations needs to respond promptly to emergencies (such as fire, gas leaks, abnormal temperatures, etc.). Edge computing enables edge devices to quickly calculate and make decisions when these events occur, thereby shortening response time. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present invention and, together with the description, serve to explain the principles of the embodiments of the present invention.

[0023] Figure 1It is a structural diagram of a smart station multi-sensor fusion monitoring system using the Hongmeng system provided by an embodiment of the present invention; Figure 2 It is a flowchart of a multi-sensor fusion monitoring method for a smart station house using the Hongmeng system provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a multimodal data inference model in a multi-sensor fusion monitoring method for a smart station house using the Hongmeng system provided in an embodiment of the present invention; Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0024] The above drawings have shown specific embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the embodiments of the present invention in any way, but to illustrate the concepts of the embodiments of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0027] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0028] First, some terms involved in the embodiments of the present invention are explained.

[0029] Hongmeng system: is a distributed operating system for all scenarios, designed to provide a unified operating system platform for multiple devices and achieve seamless collaborative experience among devices.

[0030] Smart substation: Smart substation refers to a new type of power facility formed by intelligent transformation of traditional substations or distribution rooms using modern information technology. They are mainly used in the process of power transmission and distribution to ensure the safety, reliability and efficiency of power supply. Smart substations can realize remote monitoring, fault warning, automatic control and other functions by integrating various intelligent technologies and equipment, such as smart meters, environmental monitors, video surveillance cameras, etc.

[0031] Hongmeng's Distributed Soft Bus (DSBus): allows seamless connection and communication between different devices, even if they are produced by different manufacturers or use different hardware platforms. In a smart station, this allows various types of sensors to be easily connected to the control center or other smart devices to form a collaborative network.

[0032] Smart substations refer to new types of power facilities that are formed by intelligently transforming traditional substations or distribution rooms through the integration of modern information technology. It is mainly used in the process of power transmission and distribution to ensure the safety, reliability and efficiency of power supply. Smart substations can be set up in urban centers close to high-density residential, commercial or industrial areas to meet the power needs of these areas; or, they can be set up in industrial parks to provide stable power support for factories and enterprises, especially high-tech manufacturing companies that rely on continuous power supply; or, they can be set up in places such as airports and railway stations to ensure the normal operation of transportation infrastructure; or, they can be set up in places far away from the urban power grid to provide local power supply and solve power problems in rural or mountainous areas.

[0033] With the development of smart power stations, the requirements for their monitoring systems are also increasing. The existing smart power station monitoring system has introduced a monitoring solution based on the Internet of Things technology to achieve remote management and maintenance of power equipment. Specifically, the monitoring solution based on the Internet of Things technology can use communication protocols such as wireless sensor networks (WSN), Zigbee, and LoRaWAN to connect various sensors to the Internet and realize data remote transmission and cloud server processing.

[0034] Although monitoring solutions based on IoT technology have the characteristics of low cost and easy scalability, and can realize remote management and maintenance of power equipment to a certain extent, in actual applications, due to the involvement of a large amount of network communication, they face higher security risks, such as data leakage and network attacks. In addition, in some cases, especially in remote areas, the network bandwidth may not be sufficient to support the real-time transmission of large amounts of data, and the stability and service quality of the cloud server will also directly affect the availability of the monitoring system.

[0035] Based on this, the embodiment of the present invention proposes a multi-sensor fusion monitoring method for a smart station house using the Hongmeng system, so as to realize the monitoring of the multi-sensor fusion of the smart station house by using the Hongmeng system. Based on the powerful multi-device collaboration capability of the Hongmeng system, the sensor data collected by different sensors in the smart station house can be processed locally through the edge computing node, which means that the sensor data does not need to be transmitted to the cloud server or the central control device in full, and can be processed in real time on the local device, reducing latency and improving response speed. The edge computing node enables the Hongmeng system to effectively fuse data from multiple sensors. Through the data fusion algorithm, the data of different sensors can be merged and analyzed, redundant or irrelevant information can be removed, and valuable feature data can be extracted. This is very important for the monitoring system because it can provide a more comprehensive and accurate perception of the environmental status. For example, the combination of data from temperature and humidity sensors and gas sensors can realize accurate monitoring of the station house environment and automatically trigger emergency response measures. The equipment in the smart station house needs to respond to emergencies (such as fire, gas leakage, abnormal temperature, etc.) in a timely manner. Edge computing enables edge devices to quickly calculate and make decisions when these events occur, thereby shortening the reaction time.

[0036] The technical solution provided in the implementation mode of the present application can be implemented by a server and / or a terminal device. It should be noted that the server involved in the embodiment of the present application can be an independent physical server 150, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, big data and artificial intelligence platforms.

[0037] The terminal device involved in the embodiments of the present application may be a device that provides voice and / or data connectivity to a user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. For example, a mobile phone (or "cellular" phone) and a computer with a mobile terminal, for example, a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device, which exchanges voice and / or data with a wireless access network. For example, Personal Communication Service (PCS) phones, cordless phones, Session Initiation Protocol (SIP) phones, Wireless Local Loop (WLL) stations, Personal Digital Assistants (PDAs) and other devices.

[0038] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0039] The following is an introduction to the smart station multi-sensor fusion monitoring system using the Hongmeng system provided in an embodiment of the present invention.

[0040] Figure 1 This is a schematic diagram of the structure of a smart station multi-sensor fusion monitoring system using the Hongmeng system provided by an embodiment of the present invention. Figure 1 As shown, the multi-sensor fusion monitoring system 100 of a smart station 110 adopting the Hongmeng system of this embodiment includes smart stations 110 arranged at different locations (only one is shown in the figure), and each smart station 110 is provided with a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor and a positioning sensor, as well as a device abstraction layer in the central control device 130 of the Hongmeng system, and multiple edge computing nodes 120 based on the Hongmeng system serving different smart stations 110. These devices work together to monitor the environment, equipment status and safety of the smart station 110.

[0041] Among them, the device abstraction layer of the Hongmeng system is used to create a corresponding virtual device interface for the target sensor; wherein the target sensor is any one of the temperature and humidity sensor, the gas sensor, the smoke sensor, the light sensor, the current and voltage sensor, the vibration sensor, the pressure sensor and the positioning sensor.

[0042] Specifically, when creating a virtual device interface corresponding to a target sensor, the device abstraction layer may first assign a corresponding identifier to the target sensor, and complete the registration of the target sensor in the Hongmeng system according to the identifier; then, configure the attribute information of the target sensor; the attribute information includes at least the sampling frequency and alarm condition of the target sensor. Afterwards, a corresponding virtual device interface is created for each target sensor, that is, each target sensor is mapped to a virtual device interface, and the virtual device interface is used to provide read and write functions and configuration functions. Users can configure the attribute information of the target sensor through the virtual device interface.

[0043] The edge computing node 120 in the Hongmeng system obtains the sensor data of the target sensor in the smart station 110 based on the virtual device interface. It can be understood that the edge computing node 120 can periodically obtain sensor data from the target sensor, or the edge computing node 120 can also receive sensor data sent by the target sensor in real time.

[0044] That is, the edge computing node 120 receives the sensor data regularly sent by the target sensor in the smart station 110 through the built-in communication module based on the virtual device interface. Alternatively, the edge computing node 120 sends a data collection request to the target sensor in the smart station 110 based on the virtual device interface; the target sensor collects the sensor data according to the data collection request, and sends the sensor data to the edge computing node 120 through the built-in communication module.

[0045] Exemplarily, when the target sensor is a temperature and humidity sensor, the sensor data collected is temperature and humidity data, which is used to monitor the indoor and outdoor temperature and humidity of the smart station 110 to ensure the suitability of the environment, such as the adaptation of the equipment working environment and air quality management. When the target sensor is a gas sensor, the sensor data collected is the concentration of harmful gases in the air, which is used to monitor the harmful gases in the environment where the smart station 110 is located in real time to ensure the air quality and safety in the smart station 110. When the target sensor is a smoke sensor, the sensor data collected is smoke concentration data, which is used for fire warning. When the target sensor is a light sensor, the sensor data collected is ambient light intensity data, which is used to monitor the light conditions in the smart station 110, and the lighting system can be automatically adjusted to achieve energy saving and emission reduction. When the target sensor is a current and voltage sensor, the sensor data collected is power-related data such as current, voltage, and power, which is used to monitor the health status of the power system in real time and detect electrical faults or equipment abnormalities. When the target sensor is a vibration sensor, the sensor data collected is the vibration data of the mechanical equipment, which is used to monitor the vibration of the terminal equipment in the smart station 110 to detect terminal equipment failures, such as overload or damage of motors, fans, pumps and other equipment. When the target sensor is a pressure sensor, the sensor data collected is the air pressure data in the environment or terminal equipment, which is used to monitor the pressure state of the gas or liquid to ensure that the terminal equipment operates within the normal pressure range. When the target sensor is a positioning sensor, the sensor data collected is location information, which is used to track the location information of the smart station 110 or personnel to ensure the rational scheduling and management of resources.

[0046] After acquiring the sensor data, the edge computing node 120 processes the sensor data, and performs analysis and reasoning based on the processed sensor data to obtain reasoning results; the reasoning results include at least one of the predicted problem prompts in the smart station 110, the abnormal status prompts of the equipment in the smart station 110, and preventive measures.

[0047] In this embodiment, the processing of sensor data by the edge computing node 120 includes at least one preprocessing of denoising, outlier processing, and normalization processing, and after preprocessing, the preprocessed sensor data is subjected to time alignment and space alignment processing.

[0048] Among them, time alignment is to ensure that sensor data from different sensors are aligned in time. For example, different sensor data can be made comparable at the same time point through interpolation, smoothing and other methods. Spatial alignment is for sensors distributed in different locations. The edge computing node 120 can perform spatial alignment according to the physical location of the sensor to ensure the spatial consistency of the data. For example, in a large factory, the data of multiple temperature and humidity sensors can be aligned through spatial interpolation.

[0049] The sensor data after time alignment and space alignment will be input by the edge computing node 120 into the analysis and reasoning of the multimodal data inference model to obtain the corresponding reasoning results. The multimodal data inference model is trained by the central control device 130 of the Hongmeng system based on the sensor data and reasoning results, and deployed on the edge computing node 120. In actual applications, the central control device 130 will continuously optimize and update the multimodal data inference model deployed on each edge computing node 120 according to the sensor data and reasoning results uploaded by different edge computing nodes 120, so as to achieve more accurate intelligent reasoning and decision support. Among them, the central control device 130 and the edge computing node 120 realize data transmission through distributed soft bus technology.

[0050] In this embodiment, the edge computing node 120 can also respond in time when faced with an emergency. Specifically, when the edge computing node 120 determines that the sensor data exceeds the preset threshold, it triggers an alarm and performs a preset emergency processing operation. For example, when the smoke concentration data collected by the smoke sensor set in a certain smart station 110 exceeds the preset threshold, the edge computing node 120 immediately triggers an alarm and sends a start instruction to the corresponding fire extinguishing equipment, so that the fire extinguishing equipment that receives the start instruction starts and performs the fire extinguishing operation. It can be seen that when an emergency occurs in the terminal device in the smart station 110 (such as fire, gas leakage, abnormal temperature, etc.). The edge computing node 120 can quickly calculate and make decisions when these events occur, thereby shortening the reaction time and achieving real-time response.

[0051] This method of enabling edge computing devices to process data locally greatly reduces the amount of data sent to the cloud and reduces bandwidth pressure. Especially when sensitive data is involved, edge computing can perform preliminary analysis and screening locally and only upload important or abnormal data to the cloud, which also effectively protects data privacy and security.

[0052] In some embodiments, the edge computing module of the Hongmeng system can also monitor the storage usage of the edge computing node 120 in real time; and dynamically allocate storage resources to the edge computing node 120 according to the storage usage. In this way, the resources of the edge computing node 120 can be intelligently scheduled according to actual needs. For example, when the computing resources of a certain edge computing node 120 are tight, the edge computing module can allocate some tasks to other idle edge computing nodes 120 to ensure the overall efficiency of the system.

[0053] In some embodiments, the central control device 130 can also receive a viewing request from a user; the viewing request is input based on a user monitoring interface provided by the central control device; the central control device 130 displays the operating status of the smart station 110 to the user according to the viewing request of the user, and the operating status includes the sensor data, the terminal device status in the smart station 110, and the alarm information.

[0054] In some embodiments, the central control device 130 receives a control instruction from a user; the control instruction is input based on a user control interface provided by the central control device 130; the central control device 130 sends an action command to the edge computing node 120 corresponding to the control instruction or the terminal device in the smart station 110 according to the control instruction, and the action command is used to instruct the edge computing node 120 or the terminal device to perform a corresponding action.

[0055] That is, the central control device 130 provides a unified monitoring platform, on which there are a user monitoring interface and a user control interface. Among them, the user monitoring interface can respond to the user's viewing request and display the operating status of the smart station 110 to the user. The user control interface can respond to the user's control instructions to control the terminal devices in the smart station 110 to perform corresponding actions, such as adjusting the sampling frequency of the sensor, starting or stopping certain devices, setting alarm thresholds, etc.

[0056] Based on the solution provided by the embodiment of the present invention, a corresponding virtual device interface is created for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor, and a positioning sensor; the edge computing node 120 in the Hongmeng system obtains the sensor data of the target sensor in the smart station 110 based on the virtual device interface; after the edge computing node 120 processes the sensor data, it analyzes and infers based on the processed sensor data to obtain the inference result; the inference result includes at least one of the prediction problem prompt in the smart station 110, the abnormal state prompt of the equipment in the smart station 110, and the preventive measures. It can be seen that in the technical solution of the present application, the Hongmeng system is adopted to realize the monitoring of the multi-sensor fusion of the smart station 110. Based on the powerful multi-device collaboration capability of the Hongmeng system, the sensor data collected by different sensors in the smart station 110 can be processed locally through the edge computing node 120, which means that the sensor data does not need to be transmitted to the cloud server or the central control device 130, and can be processed in real time on the local device, reducing the delay and improving the response speed. The edge computing node 120 enables the Hongmeng system to effectively fuse data from multiple sensors. Through the data fusion algorithm, the data from different sensors can be merged and analyzed, redundant or irrelevant information can be removed, and valuable feature data can be extracted. This is very important for the monitoring system because it can provide a more comprehensive and accurate perception of the environmental status. For example, the combination of data from temperature and humidity sensors and gas sensors can achieve accurate monitoring of the station environment and automatically trigger emergency response measures. The equipment in the smart station 110 needs to respond promptly to emergencies (such as fire, gas leaks, abnormal temperatures, etc.). Edge computing enables edge devices to quickly calculate and make decisions when these events occur, thereby shortening the response time.

[0057] The above is a description of the smart station multi-sensor fusion monitoring system using the Hongmeng system according to an embodiment of the present invention. The following is an introduction to the smart station multi-sensor fusion monitoring method using the Hongmeng system provided by an embodiment of the present invention in conjunction with the accompanying drawings.

[0058] Figure 2 1 is a flow chart of a multi-sensor fusion monitoring method for a smart station house using the Hongmeng system provided by an embodiment of the present invention. Figure 2 As shown, the multi-sensor fusion monitoring method for a smart station house using the Hongmeng system provided in this embodiment is used to monitor the environment, equipment status and safety of the smart station house, and the smart station house is provided with a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor and a positioning sensor. Specifically, the method provided in this embodiment may include the following steps: 2100. Create a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of the temperature and humidity sensor, the gas sensor, the smoke sensor, the light sensor, the current and voltage sensor, the vibration sensor, the pressure sensor and the positioning sensor.

[0059] 2200. The edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station based on the virtual device interface.

[0060] 2300. After processing the sensor data, the edge computing node performs analysis and reasoning based on the processed sensor data to obtain a reasoning result; the reasoning result includes at least one of a predicted problem prompt in the smart station house, an abnormal status prompt of equipment in the smart station house, and preventive measures.

[0061] When creating a virtual device interface corresponding to a target sensor, the device abstraction layer may first assign a corresponding identifier to the target sensor, and complete the registration of the target sensor in the Hongmeng system according to the identifier; then, configure the attribute information of the target sensor; the attribute information includes at least the sampling frequency and alarm condition of the target sensor. Afterwards, a corresponding virtual device interface is created for each of the target sensors, that is, each target sensor is mapped to a virtual device interface, and the virtual device interface is used to provide read and write functions and configuration functions. Users can configure the attribute information of the target sensor through the virtual device interface. For example, users can configure the sampling frequency, alarm conditions, etc. of the target sensor through the virtual device interface.

[0062] After creating a corresponding virtual device interface for the target sensor through the device abstraction layer, the edge computing node can communicate with the target sensor based on the virtual device interface. In other words, the edge computing node can obtain sensor data of the target sensor in the smart station based on the virtual device interface. It can be understood that the edge computing node can periodically obtain sensor data from the target sensor, or the edge computing node can also receive sensor data sent by the target sensor in real time.

[0063] That is, the edge computing node receives the sensor data regularly sent by the target sensor in the smart station through the built-in communication module based on the virtual device interface. Alternatively, the edge computing node sends a data collection request to the target sensor in the smart station based on the virtual device interface; the target sensor collects the sensor data according to the data collection request, and sends the sensor data to the edge computing node through the built-in communication module.

[0064] Exemplarily, when the target sensor is a temperature and humidity sensor, the sensor data collected is temperature and humidity data, which is used to monitor the indoor and outdoor temperature and humidity of the smart station to ensure the suitability of the environment, such as the adaptation of the equipment working environment and air quality management. When the target sensor is a gas sensor, the sensor data collected is the concentration of harmful gases in the air, which is used to monitor the harmful gases in the environment where the smart station is located in real time to ensure the air quality and safety in the smart station. When the target sensor is a smoke sensor, the sensor data collected is smoke concentration data, which is used for fire warning. When the target sensor is a light sensor, the sensor data collected is ambient light intensity data, which is used to monitor the light conditions in the smart station, and the lighting system can be automatically adjusted to achieve energy saving and emission reduction. When the target sensor is a current and voltage sensor, the sensor data collected is power-related data such as current, voltage, and power, which is used to monitor the health status of the power system in real time and detect electrical faults or equipment abnormalities. When the target sensor is a vibration sensor, the sensor data collected is vibration data of mechanical equipment, which is used to monitor the vibration of terminal equipment in the smart station to detect terminal equipment failures, such as overload or damage of motors, fans, pumps and other equipment. When the target sensor is a pressure sensor, the sensor data collected is the air pressure data in the environment or terminal device, which is used to monitor the pressure state of the gas or liquid and ensure that the terminal device operates within the normal pressure range. When the target sensor is a positioning sensor, the sensor data collected is location information, which is used to track the location information of the smart station or personnel to ensure the reasonable scheduling and management of resources.

[0065] After acquiring the sensor data, the edge computing node processes the sensor data, and performs analysis and reasoning based on the processed sensor data to obtain reasoning results; the reasoning results include at least one of predicted problem prompts in the smart station house, abnormal status prompts of equipment in the smart station house, and preventive measures.

[0066] Among them, after acquiring the sensor data, the edge computing node first preprocesses the sensor data, then performs time alignment and spatial alignment on the preprocessed sensor data, and inputs the sensor data after spatial alignment into the multimodal data inference model to obtain the inference result.

[0067] The processing of sensor data by the edge computing node includes at least one preprocessing of denoising, outlier processing and normalization processing, and after the preprocessing, the preprocessed sensor data is subjected to time alignment and space alignment processing.

[0068] Among them, time alignment is to ensure that sensor data from different sensors are aligned in time. For example, interpolation, smoothing and other methods can be used to make different sensor data comparable at the same time point. Spatial alignment is for sensors distributed in different locations. Edge computing nodes can perform spatial alignment based on the physical location of the sensors to ensure spatial consistency of the data. For example, in a large factory, the data of multiple temperature and humidity sensors can be aligned through spatial interpolation.

[0069] The sensor data after time alignment and space alignment will be input by the edge computing node into the analysis and reasoning of the multimodal data inference model to obtain the corresponding reasoning results. The multimodal data inference model is trained by the central control device of the Hongmeng system based on the sensor data and reasoning results, and deployed on the edge computing node. In actual applications, the central control device will continuously optimize and update the multimodal data inference model deployed on each edge computing node based on the sensor data and reasoning results uploaded by different edge computing nodes to achieve more accurate intelligent reasoning and decision support. Among them, the central control device and the edge computing node realize data transmission through distributed soft bus technology.

[0070] Figure 3 A schematic diagram of the structure of a multimodal data inference model in a multi-sensor fusion monitoring method for a smart station house using the Hongmeng system provided in an embodiment of the present invention. Figure 3 As shown, after the edge computing node inputs the sensor data after spatial alignment processing into the multimodal data reasoning model, the data input layer of the multimodal data reasoning model receives the sensor data and passes the sensor data to the feature extraction layer of the multimodal data reasoning model; the feature extraction layer extracts corresponding data features from the sensor data, and passes the data features to the hidden layer of the multimodal data reasoning model; the hidden layer concatenates the data features to obtain fused sensor data features, and passes the fused sensor data features to the fully connected layer of the multimodal data reasoning model; the fully connected layer generates a sensor feature representation corresponding to the fused sensor data features based on the fused sensor data features, and passes it to the output layer of the multimodal data reasoning model; the output layer generates a corresponding reasoning result based on the sensor feature representation.

[0071] Among them, the technology of comprehensive analysis and processing of multi-sensor data through deep learning or machine learning models. It can learn complex patterns from a large amount of historical data and provide more accurate prediction and classification results. The following is the specific implementation method of neural network fusion and its application scenario in multi-sensor data processing.

[0072] Specifically, in the data input layer, data from different sensors are taken as input. These data can include temperature, humidity, gas concentration, light intensity, current voltage, vibration, pressure, etc. In the feature extraction layer, including convolutional layers and pooling layers, useful features are automatically extracted from the raw data. For example, convolutional neural networks (CNNs) can extract local features in images or time series data through convolutional layers and pooling layers; recurrent neural networks (RNNs) and their variants (such as LSTM, GRU) can capture long-term dependencies in time series data through recursive structures. For multi-sensor data, the neural network can be designed as a multi-input model, with each input branch responsible for processing different types of data. Then, the features of different sensors are fused through shared hidden layers or splicing operations to form a comprehensive feature representation.

[0073] For different types of sensor data, neural networks can use different feature extraction methods to make full use of the characteristics and data structure of each sensor. For example, convolutional neural networks can be used to extract data features corresponding to sensor data with spatial structures, such as temperature and humidity data collected by temperature and humidity sensors, light data collected by light sensors, and location data collected by positioning sensors. Time series analysis can be used to extract data features corresponding to sensor data with time dependencies, such as gas concentration data collected by gas sensors, smoke concentration data collected by smoke sensors, current and voltage data collected by current and voltage sensors, pressure data collected by pressure sensors, and vibration data collected by vibration sensors. Frequency domain analysis can be used to extract corresponding data features from sensor data containing periodic or high-frequency components, such as current and voltage data collected by current and voltage sensors, vibration data collected by vibration sensors, and pressure data collected by pressure sensors. Among them, time series analysis includes RNN, LSTM, and GRU, and frequency domain analysis includes Fourier transform or wavelet transform.

[0074] The output layer output of the multimodal data reasoning model includes the prediction problem prompt in the smart station, the abnormal status prompt of the equipment in the smart station, and the reasoning result of at least one of the preventive measures, so that the solution of this embodiment can be based on the powerful multi-device collaboration capability of the Hongmeng system. The sensor data collected by different sensors in the smart station can be processed locally through the edge computing node, that is, the sensor data does not need to be transmitted to the cloud server or the central control device, and can be processed in real time on the local device, which reduces the response delay and improves the response speed. The edge computing node enables the Hongmeng system to effectively fuse data from multiple sensors. Through the data fusion algorithm, the data from different sensors can be merged and analyzed, redundant or irrelevant information can be removed, and valuable feature data can be extracted. This is very important for monitoring systems because it can provide a more comprehensive and accurate perception of environmental status.

[0075] In some implementations, after the edge computing node obtains the sensor data of the target sensor in the smart station based on the virtual device interface, it can trigger an alarm and perform a preset emergency processing operation when it is determined that the sensor data exceeds a preset threshold. For example, when the smoke concentration data collected by the smoke sensor exceeds the preset threshold, the edge computing node immediately triggers an alarm and sends a start instruction to the corresponding fire extinguishing equipment, so that the fire extinguishing equipment that receives the start instruction starts and performs the fire extinguishing operation. It can be seen that when an emergency occurs in the terminal device in the smart station (such as fire, gas leakage, abnormal temperature, etc.). The edge computing node can quickly calculate and make decisions when these events occur, thereby shortening the reaction time and achieving real-time response.

[0076] This method of enabling edge computing devices to process data locally greatly reduces the amount of data sent to the cloud and reduces bandwidth pressure. Especially when sensitive data is involved, edge computing can perform preliminary analysis and screening locally and only upload important or abnormal data to the cloud, which also effectively protects data privacy and security.

[0077] Based on the above scheme of the embodiment of the present invention, a corresponding virtual device interface is created for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor, and a positioning sensor; the edge computing node in the Hongmeng system obtains the sensor data of the target sensor in the smart station house based on the virtual device interface; after the edge computing node processes the sensor data, it analyzes and infers based on the processed sensor data to obtain the inference result; the inference result includes at least one of the predicted problem prompts in the smart station house, the abnormal state prompts of the equipment in the smart station house, and the preventive measures. Real-time processing of sensor data on the local device is realized without the need to transmit all to the cloud server or the central control device for processing, which reduces the response delay and improves the response speed. In addition, the edge computing node enables the Hongmeng system to effectively fuse data from multiple sensors. Through the data fusion algorithm, the data of different sensors can be merged and analyzed, redundant or irrelevant information can be removed, and valuable feature data can be extracted. The monitoring of this embodiment provides a more comprehensive and accurate environmental status perception.

[0078] In some embodiments, the edge computing module of the Hongmeng system can also monitor the storage usage of the edge computing node in real time; and dynamically allocate storage resources to the edge computing node based on the storage usage. In this way, the resources of the edge computing node can be intelligently scheduled according to actual needs. For example, when the computing resources of an edge computing node are tight, the edge computing module can assign some tasks to other idle edge computing nodes to ensure the overall efficiency of the system.

[0079] For example, in a smart station environment, multiple edge computing nodes are deployed, and each edge computing node is connected to different types of sensors, such as temperature and humidity sensors, gas sensors, smoke sensors, etc. These edge computing nodes are responsible for collecting and processing local data and uploading part of the data to the central control device. Over time, the storage space of some edge computing nodes gradually decreases, while the storage space of other nodes is still sufficient.

[0080] At this time, since the edge computing module of the Hongmeng system continuously monitors the storage usage of each edge computing node, including available space, read and write speed, I / O load and other indicators, when the storage space of an edge computing node is close to full, the edge computing module will automatically trigger the expansion mechanism to migrate part of the data to other idle edge computing nodes or cloud storage. For example, historical data can be archived to low-cost cloud storage to release more active storage space.

[0081] If the computing resources of an edge computing node are tight, the edge computing module can assign some tasks to other idle edge computing nodes to ensure the overall efficiency of the system. For example, when one edge computing node is processing a large amount of temperature and humidity data, another edge computing node can take over the processing task of gas concentration data of some gas sensors.

[0082] In this way, by dynamically allocating storage resources, the storage space of each edge computing node can be fully utilized to avoid resource waste and improve overall resource utilization. In addition, the intelligent scheduling mechanism can flexibly adjust task allocation according to actual needs to ensure the efficient operation of the system, especially in the case of multi-task concurrency, which improves the response speed and processing capacity. At the same time, automated resource management and task scheduling reduce the need for manual intervention, and also reduce maintenance costs and management complexity.

[0083] In some embodiments, the central control device can also receive a viewing request from a user; the viewing request is input based on a user monitoring interface provided by the central control device; the central control device displays the operating status of the smart station to the user based on the viewing request of the user, and the operating status includes the sensor data, the status of the terminal device in the smart station, and the alarm information.

[0084] For example, on the central control device, users can view the operating status of the entire smart station in real time through a graphical user monitoring interface. The user monitoring interface displays data from multiple sensors, the status of terminal devices, and alarm information. For example, users can view the temperature and humidity change trends in a certain area of ​​the smart station, whether the gas concentration exceeds the standard, whether there is a smoke alarm, etc.

[0085] Specifically, the central control device may receive a user's viewing request, which is based on an operation input on the user monitoring interface. For example, the user may choose to view sensor data of a specific area, or view all alarm information.

[0086] At this time, according to the user's viewing request, the central control device obtains the latest data from the edge computing node and the terminal device, and displays the operating status of the smart station on the user monitoring interface. The display content includes sensor data, such as temperature, humidity, gas concentration, etc., terminal device status, such as whether the device is online or offline, the working mode of the terminal device, etc., and alarm information, such as smoke alarm, equipment failure alarm, etc.

[0087] The user monitoring interface also provides a wealth of data visualization tools, allowing users to intuitively view data through charts, dashboards, etc. For example, a line chart can be used to display temperature trends, and a bar chart can be used to compare energy consumption in different areas.

[0088] In this way, through the unified user monitoring interface, users can grasp the operating status of the smart station in real time, quickly discover and handle abnormal situations, and improve monitoring efficiency and response speed. The graphical interface design makes data display more intuitive and easy to understand, and users can easily obtain the required information, which improves the system's usability and user experience. At the same time, users can remotely access the user monitoring interface through a web browser or mobile application, and view the operating status of the smart station anytime and anywhere, realizing remote monitoring and management of the smart station.

[0089] In some embodiments, a central control device receives a control instruction from a user; the control instruction is input based on a user control interface provided by the central control device; the central control device sends an action command to an edge computing node corresponding to the control instruction or a terminal device in the smart station building according to the control instruction, and the action command is used to instruct the edge computing node or the terminal device to perform a corresponding action.

[0090] That is to say, the central control device provides a unified monitoring platform, on which there are a user monitoring interface and a user control interface. Among them, the user monitoring interface can respond to the user's viewing request and display the operating status of the smart station to the user. The user control interface can respond to the user's control instructions and control the terminal devices in the smart station to perform corresponding actions, such as adjusting the sampling frequency of the sensor, starting or stopping certain devices, setting alarm thresholds, etc.

[0091] For example, on the central control device, users can remotely control the terminal device through the user control interface. For example, users can adjust the sampling frequency of sensors, start or stop certain devices, set alarm thresholds, etc. according to actual conditions. These operations can be completed with a simple click or slide, greatly simplifying the management process.

[0092] Specifically, the central control device receives the user's control instruction, which is based on the operation input on the user control interface. For example, the user can select a sensor and adjust its sampling frequency, or select a device and start or stop it. According to the user's control instruction, the central control device sends an action command to the edge computing node or terminal device corresponding to the instruction. For example, when the user adjusts the sampling frequency of a temperature and humidity sensor, the central control device will send the new sampling frequency parameters to the corresponding edge computing node; when the user starts a device, the central control device will send a start command to the device.

[0093] After receiving the action command, the edge computing node or terminal device executes the corresponding action and feeds back the execution result to the central control device. The central control device will display the execution result on the user control interface to confirm whether the operation is successful.

[0094] In this embodiment, through the user control interface, users can easily control the terminal devices in the smart station, simplify the management process, and reduce the complexity of manual operation. Users can accurately adjust the parameters and behaviors of the equipment according to actual needs to ensure that the operating status of the system meets expectations, improve the control accuracy and system stability. At the same time, users can remotely access the user control interface through a web browser or mobile application, control the equipment in the smart station anytime and anywhere, and realize remote management and maintenance.

[0095] Based on the above scheme of the embodiment of the present invention, through real-time monitoring and intelligent scheduling, the storage and computing resources of the edge computing nodes can be dynamically adjusted according to actual needs, thereby improving resource utilization and system flexibility. Through a unified user monitoring interface, users can view the operating status of the smart station in real time, improve monitoring efficiency and user experience, and support remote monitoring. Through the user control interface, users can remotely control terminal devices, simplify management processes, improve control accuracy, support remote management, and enhance security. This not only improves the intelligence level of the smart station, but also provides users with more convenient and efficient management and decision-making support, ensuring efficient, safe and reliable operation of the system.

[0096] The present application embodiment provides an electronic device, see Figure 4 As shown, the electronic device 400 includes a memory 402 and a processor 403 connected to the memory 402 via a bus 403 .

[0097] The memory 402 is used to store programs.

[0098] Processor 401 is used to create a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system by executing any one of the multi-sensor fusion monitoring methods for smart station buildings using the Hongmeng system in any of the above-mentioned embodiments; wherein the target sensor is any one of a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor, and a positioning sensor; the edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station building based on the virtual device interface; after processing the sensor data, the edge computing node performs analysis and reasoning based on the processed sensor data to obtain a reasoning result; the reasoning result includes at least one of a predicted problem prompt in the smart station building, an abnormal status prompt of the equipment in the smart station building, and preventive measures, so as to solve the problem of lag and delay in the remote management and maintenance of power equipment in the existing monitoring system.

[0099] The specific processing process of the processor 401 can refer to the introduction of the above method embodiment, and the specific implementation of the processor 401 can also refer to the introduction of the above embodiment.

[0100] Specifically, a bus may include a pathway for transmitting information between various components of a computer system.

[0101] The processor 401 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0102] The processor 401 may include a main processor, and may also include a baseband chip, a modem, etc.

[0103] The memory 402 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 402 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0104] The processor 401 executes the program stored in the memory 402 and calls other devices, which can be used to implement the various steps of any one of the multi-sensor fusion monitoring methods for smart station buildings using the Hongmeng system provided in the above-mentioned embodiments of the present application.

[0105] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs the program stored in the memory through the data interface to execute the multi-sensor fusion monitoring method for a smart station building using the Hongmeng system introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the embodiment of the multi-sensor fusion monitoring method for a smart station building using the Hongmeng system.

[0106] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps of the multi-sensor fusion monitoring method for a smart station using the Hongmeng system according to various embodiments of the present application described in any of the above embodiments of this specification.

[0107] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0108] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor in any of the above embodiments of this specification according to the steps of the smart station multi-sensor fusion monitoring method using the Hongmeng system according to various embodiments of the present application, and specifically the following steps can be implemented: 2100. Create a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of the temperature and humidity sensor, the gas sensor, the smoke sensor, the light sensor, the current and voltage sensor, the vibration sensor, the pressure sensor and the positioning sensor.

[0109] 2200. The edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station based on the virtual device interface.

[0110] 2300. After processing the sensor data, the edge computing node performs analysis and reasoning based on the processed sensor data to obtain a reasoning result; the reasoning result includes at least one of a predicted problem prompt in the smart station house, an abnormal status prompt of equipment in the smart station house, and preventive measures.

[0111] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0112] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0113] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0114] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0115] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0116] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0119] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0120] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0121] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-sensor fusion monitoring method for a smart station house using the Hongmeng system is used to monitor the environment, equipment status and safety of the smart station house. The smart station house is provided with a temperature and humidity sensor, a gas sensor, a smoke sensor, a light sensor, a current and voltage sensor, a vibration sensor, a pressure sensor and a positioning sensor, and is characterized in that: The method comprises: Creating a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system; wherein the target sensor is any one of the temperature and humidity sensor, the gas sensor, the smoke sensor, the light sensor, the current and voltage sensor, the vibration sensor, the pressure sensor, and the positioning sensor; The edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station based on the virtual device interface; After processing the sensor data, the edge computing node performs analysis and reasoning based on the processed sensor data to obtain a reasoning result; the reasoning result includes at least one of a predicted problem prompt in the smart station house, an abnormal status prompt of equipment in the smart station house, and preventive measures.

2. The method according to claim 1, characterized in that: The method of creating a corresponding virtual device interface for the target sensor through the device abstraction layer of the Hongmeng system includes: The device abstraction layer of the Hongmeng system assigns a corresponding identifier to the target sensor, and completes the registration of the target sensor in the Hongmeng system according to the identifier; Configuring attribute information of the target sensor through the device abstraction layer; the attribute information at least includes a sampling frequency and an alarm condition of the target sensor; The device abstraction layer creates a corresponding virtual device interface for each target sensor, and the virtual device interface is used to provide read and write functions and configuration functions.

3. The method according to claim 2, characterized in that The edge computing node in the Hongmeng system obtains sensor data of the target sensor in the smart station based on the virtual device interface, including: The edge computing node receives the sensor data regularly sent by the target sensor in the smart station house through the built-in communication module based on the virtual device interface; or The edge computing node sends a data collection request to the target sensor in the smart station house based on the virtual device interface; The target sensor collects the sensor data according to the data collection request, and sends the sensor data to the edge computing node through a built-in communication module.

4. The method according to claim 1, characterized in that: After the edge computing node processes the sensor data, it performs analysis and reasoning based on the processed sensor data to obtain a reasoning result, including: The edge computing node pre-processes the sensor data; The edge computing node performs spatial alignment processing on the pre-processed sensor data; The edge computing node inputs the sensor data after spatial alignment processing into the multimodal data reasoning model to obtain the reasoning result.

5. The method according to claim 4, characterized in that The edge computing node inputs the sensor data after spatial alignment processing into the multimodal data reasoning model to obtain the reasoning result, including: The data input layer of the multimodal data reasoning model receives the sensor data and transmits the sensor data to the feature extraction layer of the multimodal data reasoning model; The feature extraction layer extracts corresponding data features from the sensor data and transmits the data features to the hidden layer of the multimodal data inference model; The hidden layer performs splicing processing on the data features to obtain fused sensor data features, and transmits the fused sensor data features to the fully connected layer of the multimodal data inference model; The fully connected layer generates a sensor feature representation corresponding to the fused sensor data feature according to the fused sensor data feature, and transmits it to the output layer of the multimodal data inference model; The output layer generates corresponding inference results according to the sensor feature representation.

6. The method according to claim 1, characterized in that After the edge computing node obtains sensor data of the target sensor in the smart station house based on the virtual device interface, the method further includes: When the edge computing node determines that the sensor data exceeds a preset threshold, it triggers an alarm and executes a preset emergency processing operation.

7. The method according to claim 1, characterized in that The method further comprises: The edge computing module of the Hongmeng system monitors the storage usage of the edge computing node in real time; The edge computing module dynamically allocates storage resources to the edge computing node according to the storage usage.

8. The method according to claim 4, characterized in that The method further comprises: The edge computing node transmits the sensor data and the reasoning result to the central control device of the Hongmeng system through distributed soft bus technology; The central control device continuously optimizes and updates the multimodal data reasoning model deployed on the edge computing node based on the sensor data and the reasoning result.

9. The method according to claim 8, characterized in that The method further comprises: The central control device receives a user's viewing request; the viewing request is input based on a user monitoring interface provided by the central control device; The central control device displays the operating status of the smart station to the user according to the user's viewing request, and the operating status includes the sensor data, the terminal device status in the smart station and the alarm information.

10. The method according to claim 8, characterized in that The method further comprises: The central control device receives a control instruction from a user; the control instruction is input based on a user control interface provided by the central control device; The central control device sends an action command to the edge computing node corresponding to the control instruction or the terminal device in the smart station house according to the control instruction, and the action command is used to instruct the edge computing node or the terminal device to perform a corresponding action.

Images