Heterogeneous data-based clutter monitoring system and method

Abstract

The application discloses a kind of based on heterogeneous data's sundry monitoring system and method, wherein system includes multiple data acquisition terminals and data processing gateway, data acquisition terminal is used to by biological monitoring device to carry out real-time detection to data acquisition area to obtain first type detection data and send to data processing gateway, by distance monitoring device to data acquisition area is periodically detected to obtain second type detection data and send to data processing gateway, after receiving high-frequency sampling information, obtain second type original detection data and send to data processing gateway;Data processing gateway judges to respectively process second type detection data, second type original detection data and first type detection data, to judge the existence of sundry in data acquisition area in combination with each determination result in preset time period.It avoids that personnel or other biological movement causes sundry monitoring false alarm, maintenance cost is low and not easy to cause personnel privacy problem, improves the timeliness and accuracy of sundry stacking monitoring.

Description

Technical Field

[0001] This invention relates to the field of regional detection technology, and in particular to a debris monitoring system and method based on heterogeneous data. Background Technology

[0002] With the rapid development of urban economies, the number of high-rise buildings is constantly increasing. Currently, emergency evacuation routes in high-rise buildings are prone to debris accumulation, posing a serious challenge to public safety. For example, debris piled up in emergency evacuation routes not only hinders evacuation but also increases the danger of emergencies, creating more hidden dangers for public safety. Therefore, improving the monitoring capabilities for debris accumulation in emergency evacuation routes of high-rise buildings is an important and urgent task.

[0003] Most existing debris monitoring technologies utilize video capture, which offers wide coverage but suffers from high deployment and maintenance costs. Furthermore, issues involving personnel privacy can easily lead to disputes, thus limiting their effectiveness. Additionally, current monitoring technologies often overlook the impact of personnel behavior within building evacuation routes on debris monitoring. Therefore, there is an urgent need for a debris monitoring system that overcomes these limitations: low cost, no privacy concerns, and ensures that personnel within building evacuation routes do not negatively affect debris monitoring. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a clutter monitoring system and method based on heterogeneous data, which solves the problems of existing clutter monitoring technology, such as deployment and maintenance costs, easy privacy issues, and the impact of people in building evacuation routes on clutter monitoring, leading to false alarms.

[0005] In a first aspect, this application provides a debris monitoring system based on heterogeneous data, including multiple data acquisition terminals and a data processing gateway that communicates with all the data acquisition terminals respectively. Each data acquisition terminal has a corresponding data acquisition area, and each data acquisition terminal operates in a target acquisition terminal working mode, with the data acquisition area corresponding to the target acquisition terminal set as the target acquisition area.

[0006] The target acquisition terminal is used to detect the target acquisition area in real time using a biological monitoring device to obtain a first type of detection data and send the first type of detection data to the data processing gateway. It is also used to periodically detect the target acquisition area using a distance monitoring device to obtain a second type of detection data and send the second type of detection data to the data processing gateway. Furthermore, it is used to use the distance monitoring device to perform high-frequency detection on the target acquisition area for a preset time period after receiving high-frequency sampling information to obtain a second type of raw detection data and send the second type of raw detection data to the data processing gateway.

[0007] The data processing gateway is used to determine whether there may be debris in the target collection area based on the second type of detection data. If so, it sends high-frequency sampling information to the target collection terminal and performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living organisms in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it determines that there is no debris in the target collection area within the corresponding preset time period. If it determines that there are no living organisms in the target collection area, it determines that there are debris in the target collection area within the preset time period.

[0008] In one embodiment of this application, the target acquisition terminal periodically probes the target acquisition area using a distance monitoring device to obtain a second type of detection data, including:

[0009] The target acquisition terminal uses a distance monitoring device to detect the target acquisition area once every preset time period to obtain the second type of raw sampling data.

[0010] The second type of original sampling data is denoised to obtain the second type of denoised sampling data. The second type of denoised sampling data is divided into multiple preset windows by using a sliding window method, and the average distance of each preset window is calculated. The average distances of all preset windows are then combined to form the second type of detection data.

[0011] In one embodiment of this application, the data processing gateway determines whether debris may exist in the target acquisition area based on the second type of detection data, including:

[0012] The data processing gateway determines whether the average distance of each preset window in the second type of detection data is greater than or equal to its corresponding area distance threshold. If so, it determines that there are no debris in the current target acquisition area; otherwise, it determines that there may be debris in the current target acquisition area.

[0013] In one embodiment of this application, the data processing gateway performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target acquisition area, including:

[0014] The data processing gateway performs denoising processing on the second type of raw detection data to obtain the second type of denoised raw data, and transmits the second type of denoised raw data to the debris detection neural network so that the debris detection neural network outputs the detection result.

[0015] The aforementioned debris detection neural network is obtained by training a preset neural network based on the original detection data training dataset. The original detection data training dataset includes multiple sets of second-class original sampling data and a label corresponding to each set of second-class original sampling data. The label indicates whether debris may be present or not.

[0016] In one embodiment of this application, the preset neural network is a CNN+LSTM structure neural network.

[0017] In one embodiment of this application, the biological monitoring device includes an infrared sensor and a network sniffer, and the first type of detection data includes infrared detection data and sniffer detection data; the data processing gateway determines whether biological activity may exist in the target collection area based on the first type of detection data within a corresponding preset time period, including:

[0018] If an infrared signal is detected in the infrared detection data, it is determined that there may be living organisms in the target collection area; otherwise, it is determined that there is network device information in the sniffing detection data. If there is information, it is determined that there may be living organisms in the target collection area; otherwise, it is determined that there are no living organisms in the target collection area.

[0019] In one embodiment of this application, the distance monitoring device is at least one of a distance sensor, ultrasonic radar, lidar, or point cloud radar.

[0020] In one embodiment of this application, the debris monitoring system based on heterogeneous data further includes a cloud server that is communicatively connected to the data processing gateway. The data processing gateway is also used to send the first type of detection data to the cloud server and send the second type of raw detection data and the determination result of whether there is debris in the target collection area within the corresponding preset time period to the cloud server.

[0021] The cloud server is used to store the first type of detection data, the second type of raw detection data, and the determination results of whether there are debris in the target collection area within the corresponding preset time period, and is also used to display the determination results of whether there are debris in the target collection area.

[0022] In one embodiment of this application, the data acquisition terminal and the data processing gateway achieve data transmission through LoRa transmission technology, and the data processing gateway and the cloud server achieve data transmission through 5G transmission technology.

[0023] Secondly, this application provides a debris monitoring method based on heterogeneous data, implemented based on the heterogeneous data-based debris monitoring system, comprising:

[0024] The target acquisition terminal uses a biological monitoring device to detect the target acquisition area in real time to obtain a first type of detection data, and sends the first type of detection data to the data processing gateway. At the same time, it uses a distance monitoring device to periodically detect the target acquisition area to obtain a second type of detection data, and sends the second type of detection data to the data processing gateway.

[0025] The data processing gateway determines whether there may be debris in the target acquisition area based on the second type of detection data. If so, it sends high-frequency sampling information to the target acquisition terminal; otherwise, it does not send information to the target acquisition terminal.

[0026] After receiving high-frequency sampling information, the target acquisition terminal uses the distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain the second type of raw detection data, and sends the second type of raw detection data to the data processing gateway.

[0027] The data processing gateway performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living organisms in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it determines that there is no debris in the target collection area within the corresponding preset time period. If it determines that there are no living organisms in the target collection area based on the first type of detection data within the corresponding preset time period, it determines that there is debris in the target collection area within the preset time period.

[0028] Compared with the prior art, one or more embodiments of the above solutions may have the following advantages or beneficial effects:

[0029] The heterogeneous data-based debris monitoring system provided in this invention collects biological and distance detection data through data acquisition terminals and transmits the collected data to a data processing gateway. This allows the gateway to monitor debris within a specific area based on the distance and biological detection data, preventing false alarms caused by personnel or other biological activity. Furthermore, the data acquisition devices in the terminals have low deployment and maintenance costs and are less likely to cause privacy issues. Multiple data acquisition terminals are installed in different data acquisition areas and networked with the data processing gateway to collect data from different areas. This facilitates application in various situations requiring debris monitoring, such as in corridors to monitor multiple emergency evacuation routes. This allows managers to achieve more comprehensive and accurate monitoring and analysis of these routes, avoiding the limitations of traditional monitoring technologies in terms of range and accuracy. Real-time monitoring and early warning are achieved, significantly improving the timeliness and accuracy of debris monitoring and providing stronger support for fire prevention and control in high-rise buildings.

[0030] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0031] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0032] Figure 1 The diagram shown is a network structure diagram of the debris monitoring system based on heterogeneous data described in the embodiments of this application.

[0033] Figure 2 This is an illustration of the application scenario where the heterogeneous data-based debris monitoring system described in this application is used in high-rise buildings.

[0034] Figure 3 The diagram shows the signal interaction between the data acquisition terminal and the data processing gateway in the heterogeneous data-based debris monitoring system described in this application embodiment.

[0035] Figure 4 The diagram shown is a structural schematic of the data acquisition terminal in the debris monitoring system based on heterogeneous data described in this application embodiment.

[0036] Figure 5The diagram shown is a schematic representation of the data processing gateway in the heterogeneous data-based debris monitoring system described in this application embodiment.

[0037] Figure 6 The diagram shown is a flowchart illustrating the debris monitoring method based on heterogeneous data as described in the embodiments of this application.

[0038] Figure 7 The diagram shows a process schematic of the debris monitoring method based on heterogeneous data described in the embodiments of this application. Detailed Implementation

[0039] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples, so that the process of how the present invention uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that, as long as there is no conflict, the various embodiments and features in the various embodiments of the present invention can be combined with each other, and the resulting technical solutions are all within the protection scope of the present invention.

[0040] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0041] Sniffing technology is a wireless technology used to detect and identify Bluetooth devices. It has advantages such as low power consumption, low cost and easy deployment. Sniffing only requires people to wear devices with information capture capabilities, such as mobile phones, tablets, wristbands or headphones, and turn on Bluetooth or WiFi.

[0042] The following embodiments of this application provide a debris monitoring system and method based on heterogeneous data, which solves the problems of existing debris monitoring technologies, such as deployment and maintenance costs, easy privacy issues, and the impact of personnel in building evacuation routes on debris monitoring, leading to false alarms.

[0043] Existing contact recognition technologies have problems such as being unable to recognize signals from multiple contact points, being unable to effectively identify contact signals, and being unable to accurately identify multi-click events.

[0044] The following will describe in detail the principle and implementation of a debris monitoring system and method based on heterogeneous data according to this embodiment, with reference to the accompanying drawings, so that those skilled in the art can understand the debris monitoring system and method based on heterogeneous data according to this embodiment without creative effort.

[0045] This application's debris monitoring system based on heterogeneous data can be applied to multiple emergency evacuation routes in a building to achieve debris monitoring in these routes. Figure 2 This application demonstrates an intended use case for the heterogeneous data-based debris monitoring system described in this embodiment, applied to high-rise buildings. By applying this system to multiple emergency evacuation routes within a building, management personnel can achieve more comprehensive and accurate monitoring and analysis of these routes. This avoids the limitations of traditional monitoring technologies in terms of monitoring range and accuracy, enabling real-time monitoring and early warning. It significantly improves the timeliness and accuracy of debris accumulation monitoring, providing stronger support for fire prevention and control in high-rise buildings. Furthermore, this heterogeneous data-based debris monitoring system can also be applied to other multi-area debris monitoring scenarios to achieve debris monitoring in corresponding multi-area areas; however, this application does not impose any fixed limitations on its application.

[0046] like Figure 1 As shown in the figure, this embodiment provides a debris monitoring system based on heterogeneous data, which specifically includes multiple data acquisition terminals and a data processing gateway. The multiple data acquisition terminals communicate with the data processing gateway respectively, and each data acquisition terminal has a corresponding data acquisition area.

[0047] The data acquisition method of each data acquisition terminal and the data interaction process between each data acquisition terminal and the data processing gateway are the same. Therefore, in order to more clearly explain the working method of multiple data acquisition terminals, it is set that the data acquisition terminals all work in the working mode of the target acquisition terminal, and the data acquisition area corresponding to the target acquisition terminal is set as the target acquisition area.

[0048] The specific target acquisition terminal is mainly used to detect the target acquisition area in real time through biological monitoring devices to obtain the first type of detection data, and send the first type of detection data to the data processing gateway. It is also used to periodically detect the target acquisition area through distance monitoring devices to obtain the second type of detection data, and send the second type of detection data to the data processing gateway. Furthermore, after receiving high-frequency sampling information, it is used to use distance monitoring devices to conduct high-frequency detection of the target acquisition area within a preset time period to obtain the second type of raw detection data, and send the second type of raw detection data to the data processing gateway.

[0049] The data processing gateway is mainly used to determine whether there may be debris in the target collection area based on the second type of detection data. If so, it sends high-frequency sampling information to the target collection terminal and performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living organisms in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it determines that there is no debris in the target collection area within the corresponding preset time period. If it determines that there are no living organisms in the target collection area, it determines that there are debris in the target collection area within the preset time period.

[0050] The debris monitoring system based on heterogeneous data provided in this invention collects biological detection data and distance detection data by setting up a data acquisition terminal, and transmits the collected detection data to a data processing gateway. This allows the data processing gateway to monitor debris in a specific area based on the distance detection data and biological detection data, avoiding false alarms caused by the movement of people or other organisms. At the same time, the data acquisition device in the data acquisition terminal has low deployment and maintenance costs and is less likely to cause personnel privacy issues.

[0051] The following provides a detailed description of the structure and operation of the data acquisition terminal and data processing gateway in the heterogeneous data-based debris monitoring system of this embodiment.

[0052] refer to Figure 1 As shown, multiple data acquisition terminals and the data processing gateway can form a star network topology through communication. Specifically, data transmission between the data acquisition terminals and the data processing gateway can be achieved using LoRa transmission technology. For example, the data processing gateway can preset M frequency points (M≤64) through a LoRa transmission module, and N data acquisition terminals can be connected to each frequency point. In this case, the data processing gateway can support M*N data acquisition terminals. After initial power-on, the data acquisition terminal generates a random time offset within s seconds to avoid random access conflicts caused by devices on the same frequency point. The time offset obtained after initial power-on remains unchanged in each reporting cycle, thereby effectively avoiding data packet loss caused by access conflicts and realizing the self-organizing network communication function. The data processing gateway, as the center of all data acquisition terminals, has the functions of data processing and reception, and command issuance.

[0053] In practical applications, each data acquisition terminal needs to be installed appropriately for its designated data acquisition area. During installation, the positions of the biomonitoring devices and distance monitoring devices must be calibrated. By observing the terminal feedback values ​​several times in real time, it is ensured that the positions and angles of the biomonitoring devices and distance monitoring devices are correctly installed, and calibration is performed to eliminate system errors and reduce background noise. Simultaneously, the data acquisition area targeted by each data acquisition terminal is defined, recording the terminal ID and its installation location, and the data acquisition area for each terminal (e.g., the corridor area in a high-rise building). A unique ID for each data acquisition terminal is mapped and numbered, for example: Building_Sensor_001A, Building_Sensor_002A, etc. A data record table is established for each data acquisition terminal, allowing for easy querying of the terminal settings.

[0054] refer to Figure 3 As shown, the following section uses the target sampling terminal as an example to explain in detail the working method of the data acquisition terminal and the information interaction method between the data acquisition terminal and the data processing gateway. The target sampling terminal can be any data acquisition terminal, and the data acquisition area corresponding to the target acquisition terminal is the target acquisition area.

[0055] In this embodiment, the debris monitoring system based on heterogeneous data can detect the target collection area in real time through biological monitoring devices to obtain the first type of detection data, and then send the detected first type of detection data to the data processing gateway.

[0056] In one embodiment of this application, the biological monitoring device in the target acquisition terminal may include an infrared sensor and a network sniffer. The infrared sensor can detect clearly visible organisms such as humans or animals within the data acquisition area; the network sniffer can detect organisms within the data acquisition area carrying devices with Bluetooth or wireless functionality, such as mobile phones, tablets, wristbands, earphones, or animal locators, including those hiding in corners or behind foreign objects. This embodiment uses an infrared sensor and a network sniffer as biological monitoring devices in the target acquisition terminal, enabling the detection of both visible organisms such as humans or animals within the target acquisition area and, simultaneously, the detection of organisms in less easily identifiable corners or behind foreign objects within the data acquisition area. This achieves more comprehensive identification of humans and animals within the data acquisition area, contributing to improved intelligence and accuracy of the debris monitoring system based on heterogeneous data.

[0057] In one embodiment of this application, the network sniffer may employ an ESP32 to achieve real-time information capture.

[0058] The target acquisition terminal uses infrared sensors and network sniffers to detect the target acquisition area in real time to obtain the first type of detection data, which includes infrared detection data and sniffing detection data. The first type of detection data is then transmitted to the data processing gateway in real time or as a package of data over a period of time. Table 1 shows an example of the data frame format after data is packaged within a time period. Table 2 shows one data format of the sniffing detection data in Table 1.

[0059] Table 1

[0060]

[0061] Table 2

[0062]

[0063] In this embodiment, the debris monitoring system based on heterogeneous data, during normal operation, also uses the target acquisition terminal to periodically probe the target acquisition area through a distance monitoring device to obtain a second type of detection data, and then sends the second type of detection data to the data processing gateway.

[0064] In one embodiment of this application, the distance monitoring device in the target acquisition terminal can be at least one of a distance sensor, ultrasonic radar, lidar, or point cloud radar. Specifically, the distance monitoring device is appropriately installed within the corresponding target acquisition area so that it can monitor the distance within the target acquisition area and determine the presence of debris by comparing the changes in the relative distance between sub-regions within the target acquisition area. Since the distance monitoring device collects a large amount of data and the processing is complex, the target acquisition terminal uses periodic detection to acquire the second type of detection data. Specifically, the target acquisition terminal can use a periodic scheduling algorithm to handle timed tasks, such as periodically sending signal acquisition commands to the distance monitoring device. Each time the distance monitoring device receives a signal acquisition command, it monitors the target acquisition area to acquire the corresponding second type of detection data. The periodic scheduling algorithm can execute tasks at fixed time intervals, ensuring that the system responds promptly to various events.

[0065] Furthermore, the target acquisition terminal can be configured to probe the target acquisition area once every preset time interval using a distance monitoring device to obtain the second type of raw sampling data. After acquiring the second type of raw sampling data, the target acquisition terminal also needs to denoise the data to obtain the second type of denoised sampling data, ensuring the reliability of the measurement values. Specifically, a Kalman filter algorithm can be used to filter the second type of raw sampling data. Then, a sliding window algorithm is used to calculate the mean of the distance data. The sliding window algorithm has linear time complexity and can effectively handle large amounts of data, enabling real-time processing. Specifically, the second type of denoised sampling data is divided into multiple preset windows using a sliding window approach, and the mean distance of each preset window is calculated. The mean distances of all preset windows are then aggregated into the second type of detection data. To optimize the distance data calculation, a weighted moving average method can also be used when calculating the mean distance of the preset windows to ensure the accuracy of the results.

[0066] Finally, the acquired second type of detection data is packaged and sent to the data processing gateway. Table 3 shows a formatted data frame of the average distance of a single preset window in the second type of detection data.

[0067] Table 3

[0068]

[0069] In this embodiment, when the debris monitoring system based on heterogeneous data is working normally, the target acquisition terminal is also used to use a distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period after receiving high-frequency sampling information to obtain the second type of raw detection data, and then send the second type of raw detection data to the data processing gateway.

[0070] After the data processing gateway processes the received second type of detection data, it can initially determine whether there may be debris in the target acquisition area. When the data processing gateway determines that there may be debris in the target acquisition area, it will send high-frequency sampling information back to the corresponding target acquisition terminal, that is, send a command to the target acquisition terminal to continuously collect high-frequency signals, so as to further determine based on the collected second type of raw detection data. After receiving the high-frequency sampling information, the data processing gateway will continuously perform high-frequency detection on the target acquisition area using a distance monitoring device within a preset time period to obtain the second type of raw detection data. At this time, the target acquisition terminal does not need to process the second type of raw detection data; it can directly package and send it to the data processing gateway.

[0071] refer to Figure 4As shown in the description above, the target acquisition terminal needs to include a central processing unit (CPU) and a bio-monitoring device and a distance detection device connected to the CPU. The target acquisition terminal also needs to include an independent power supply module, a corresponding position debugging module, and a LoRa transmission module. The power supply module, position debugging module, and LoRa transmission module are all connected to the CPU to achieve their respective functions. The target acquisition terminal can also be designed with other structures, which are not fixedly limited here.

[0072] In this embodiment, the debris monitoring system based on heterogeneous data, when operating normally, uses a data processing gateway to determine whether there may be debris in the target acquisition area based on the second type of detection data. If so, it sends high-frequency sampling information to the target acquisition terminal.

[0073] The data processing gateway divides the target acquisition area into multiple sub-region modules, each with its corresponding regional distance threshold. After receiving the second type of detection data, the gateway compares the preset windows with the sub-region modules and sequentially determines whether the average distance of each preset window is greater than or equal to the regional distance threshold of its corresponding sub-region module. If the average distance of all preset windows in the second type of detection data is greater than the regional distance threshold of their corresponding sub-region modules, it indicates that the current target acquisition area has not undergone spatial changes compared to the initial state (i.e., there is no spatial change in the target acquisition area due to the accumulation of debris). If the average distance of one or more preset windows in the second type of detection data is not greater than the regional distance threshold of their corresponding sub-region modules, it indicates that the current target acquisition area has undergone spatial changes compared to the initial state. At this point, it can be determined that there may be debris in the current target acquisition area. Then, the data processing gateway sends high-frequency sampling information to the target acquisition terminal corresponding to the second type of detection data, so that the target acquisition terminal continuously collects the second type of raw detection data at a high frequency within a preset time period based on the high-frequency sampling information.

[0074] It should be noted that the sub-region modules within the target acquisition area can be divided based on actual conditions, and the regional distance threshold for each sub-region module is adaptively calculated based on these conditions. The regional distance threshold for each sub-region module can dynamically change based on the actual situation of its corresponding target acquisition area, thereby enabling monitoring of scenarios with significant data fluctuations. Normally, the target acquisition area remains unchanged, but in special circumstances (such as changes in spatial layout), it may alter. In such cases, the distance monitoring device in the target acquisition terminal can periodically acquire signals from the target acquisition area, and the new regional distance threshold for each sub-region module can be calculated based on the acquired signals. It should be noted that the regional distance threshold updates do not need to be excessively frequent; adjustments every six months or a year are sufficient. In practical applications, dynamically adjusting the regional distance threshold according to the actual scenario can improve the system's robustness, and this approach allows the system to effectively determine the presence of debris in the target acquisition area, achieving more accurate detection.

[0075] In this embodiment, when the debris monitoring system based on heterogeneous data is working normally, the data processing gateway is also used to perform in-depth preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If the data processing gateway determines that there may be debris in the target collection area after performing in-depth preprocessing on the second type of raw detection data, the data processing gateway needs to make further judgment based on the first type of detection data within a preset time period. If the data processing gateway determines that there is no debris in the target collection area after performing in-depth preprocessing on the second type of raw detection data, it can be directly determined that there is no debris in the target collection area. When the data processing gateway makes a judgment based on the first type of detection data within a preset time period, it needs to determine whether there may be living organisms in the target collection area. If so, it means that there are living organisms such as humans or animals in the target collection area within the preset time period. At this time, the collected data will affect the debris judgment result. Therefore, it can be directly determined that there are no debris in the target collection area within the corresponding preset time period. However, if the data processing gateway needs to determine that there are no living organisms in the target collection area based on the first type of detection data within the preset time period, it means that there are no living organisms such as humans or animals in the target collection area within the preset time period. At this time, the conclusion of deep preprocessing of the second type of raw detection data can be used as the final conclusion, that is, it is determined that there are debris in the target collection area within the preset time period.

[0076] Further, the data processing gateway performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target acquisition area. Specifically, this includes: the data processing gateway performs denoising processing on the second type of raw detection data to obtain second type of denoised raw data. The denoising method can be Kalman filtering or other reasonable methods. Then, the second type of denoised raw data is transmitted to the debris detection neural network. The debris detection neural network outputs the corresponding judgment result, which indicates that there may be debris in the target acquisition area or that there is no debris in the target acquisition area. The debris detection neural network is trained on a preset neural network based on the raw detection data training dataset. The raw detection data training dataset is the training dataset obtained by the target acquisition terminal through multiple signal acquisitions of the target acquisition area using a distance monitoring device. That is, the raw detection data training dataset includes multiple sets of second type raw sampling data and the label corresponding to each set of second type raw sampling data. The training dataset should contain a sufficient amount of second type raw sampling data, and the location and shape of the debris contained therein should be sufficiently diverse. Specifically, the label corresponding to the second type of raw sampling data is either "may have debris" or "no debris exists". In one embodiment of this application, the preset neural network can be a CNN+LSTM structure neural network.

[0077] In practical applications, the data processing gateway can receive all time periods of the first-type detection data, but it only further processes the first-type detection data within a preset time period that may contain debris after deep preprocessing of the second-type raw detection data. Specifically, the data processing gateway determines whether there may be life in the target collection area based on the first-type detection data within the corresponding preset time period. This includes: determining whether an infrared signal is detected in the infrared detection data of the first-type detection data; if so, it indicates that there may be life in the target collection area. Otherwise, it further determines whether network device information (device MAC address, etc.) is present in the sniffing detection data of the first-type detection data; if so, it determines that there may be life in the target collection area; otherwise, it determines that there is no life in the target collection area. This determination method avoids missing detections of people or animals hiding in corners or behind foreign objects in the target area, achieving a more comprehensive identification of people and animals in the target collection area, and helping to improve the intelligence and accuracy of the debris monitoring system based on heterogeneous data.

[0078] In this embodiment, the data processing gateway can also be configured with a parameter repair function, meaning it can dynamically repair any offset of the distance monitoring device. The data processing gateway periodically determines whether the distance monitoring device has shifted position based on the data it monitors. Specifically, during detection, the data acquisition terminal reports the raw data from the distance monitoring device. The gateway analyzes the raw data across multiple time dimensions to determine if any offset has occurred. If no anomalies are found, normal operation continues. If the raw data from multiple time dimensions do not match, indicating an angle anomaly, a spatial point cloud is generated to estimate the actual angle of the distance monitoring device at that time. This estimate is then sent to administrators for manual correction. Simultaneously, the gateway can automatically correct the collected data based on the estimated actual angle of the distance monitoring device at that time.

[0079] In one embodiment of this application, the debris monitoring system based on heterogeneous data further includes a cloud server that is communicatively connected to a data processing gateway.

[0080] In this embodiment, when the debris monitoring system based on heterogeneous data is working normally, the data processing gateway can also be used to send the first type of detection data to the cloud server, and send the second type of raw detection data and the determination result of whether there is debris in the target collection area within the preset time period corresponding to the second type of raw detection data to the cloud server.

[0081] Specifically, to reduce the output transmission pressure on the data processing gateway, the gateway can perform compression and other processing on the first type of detection data before transmitting it to the cloud server. The data compression technology used for processing the first type of detection data aims to reduce data transmission volume and improve efficiency. This data compression process mainly includes two aspects: deduplication and data integration. Deduplication removes duplicate device MAC addresses detected in the sniffing detection data of the first type of detection data, ensuring that the same device information appears only once in a data packet, reducing data redundancy and saving transmission bandwidth. Data integration combines data from the same device in the sniffing detection data of the first type of detection data into a single data item, further reducing the data volume. In the code implementation, these functions are achieved by maintaining a data cache, comparing newly received data, and performing integration processing. This data processing method effectively improves the efficiency and performance of data transmission.

[0082] In one embodiment of this application, the data processing gateway and the cloud server can achieve data transmission via 5G transmission technology. Specifically, the data processing gateway can package the data to be sent into JSON format data, and then transmit the packaged data to the cloud server via 5G transmission technology. Table 4 shows one type of JSON format data packet.

[0083] Table 4

[0084]

[0085] It should be noted that the data reporting cycle of the data processing gateway can be set to be the same as or different from the data reporting cycle of the data acquisition terminal; no fixed restriction is imposed here.

[0086] In this embodiment, the debris monitoring system based on heterogeneous data, during normal operation, primarily uses a cloud server to store the first type of detection data, the second type of raw detection data, and the results of determining whether debris exists in the target collection area within the corresponding preset time period, facilitating subsequent data retrieval. Simultaneously, the cloud server also displays the results of determining whether debris exists in the target collection area.

[0087] The cloud server parses the received JSON data packets, stores the first type of detection data and the second type of raw detection data, and displays the corresponding debris judgment results.

[0088] refer to Figure 5 As shown in the diagram, based on the above description, the data processing gateway includes a central processing unit (CPU) and connected to the CPU, such as a 5G transmission module, a LoRa transmission module, a data processing module, and a data parsing module. The data processing gateway can also be designed to include other structures (such as a log recording module, a serial communication module, etc.), which are not subject to fixed limitations here.

[0089] The debris monitoring system based on heterogeneous data provided in this invention, when applied to fire escape routes in buildings, can improve the timeliness and accuracy of debris obstruction alarms. The system can monitor the accumulation of debris in corridors in real time, promptly detect anomalies and issue early warning signals, avoiding the risk of blocked fire escape routes. The system performs precise analysis based on the accumulated debris data, avoiding false alarms and missed alarms, thus improving the reliability of fire alarms. This invention, through a self-organizing network data acquisition terminal and data processing gateway, can comprehensively cover the interior of high-rise buildings. Without collecting sensitive personnel information, it combines various non-intrusive data for precise analysis, improving the comprehensiveness and accuracy of monitoring. Because the data acquisition terminal is small and easy to deploy, and only one gateway device is needed to complete the self-organizing network, the manpower costs required for deployment and maintenance are greatly reduced. Compared to video acquisition system deployment, the miniaturized and automated system architecture of this invention makes the construction and management of IoT systems easier and more efficient, reducing manpower and operating costs. The small size and low overall power consumption of the terminal and gateway devices, along with the small amount of data transmitted, result in relatively low equipment costs. Furthermore, using a single gateway to complete the self-organizing network of multiple devices further reduces data communication and maintenance costs, making it suitable for a wider range of scenarios and applications. The system can achieve data-driven and model-based object recognition and business processing, which enables the system to automatically adjust parameters and strategies when facing different environments and workloads, thus improving the system's robustness and adaptability.

[0090] like Figure 6 and Figure 7 As shown, this embodiment also provides a debris monitoring method based on heterogeneous data. This method is implemented based on the debris monitoring system based on heterogeneous data in the above embodiment. The target acquisition terminal is set as any data acquisition terminal in the debris monitoring system based on heterogeneous data. The interaction between the target acquisition terminal and the data processing gateway in the debris monitoring method based on heterogeneous data specifically includes the following steps.

[0091] In step S101, the target acquisition terminal uses a biological monitoring device to detect the target acquisition area in real time to obtain the first type of detection data and sends the first type of detection data to the data processing gateway. At the same time, it uses a distance monitoring device to periodically detect the target acquisition area to obtain the second type of detection data and sends the second type of detection data to the data processing gateway.

[0092] In step S102, the data processing gateway determines whether there may be debris in the target acquisition area based on the second type of detection data. If so, it sends high-frequency sampling information to the target acquisition terminal; otherwise, it does not send information to the target acquisition terminal.

[0093] In step S103, after receiving the high-frequency sampling information, the target acquisition terminal uses a distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain the second type of raw detection data, and sends the second type of raw detection data to the data processing gateway.

[0094] Step S104: The data processing gateway performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living organisms in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it determines that there is no debris in the target collection area within the corresponding preset time period. If it determines that there are no living organisms in the target collection area based on the first type of detection data within the corresponding preset time period, it determines that there is debris in the target collection area within the preset time period.

[0095] The execution process of each step described above is consistent with the working mode of the target acquisition terminal and data processing gateway in the heterogeneous data debris monitoring system, and will not be elaborated further here. Furthermore, the data interaction methods between each data acquisition terminal and the data processing gateway are the same as described above, and will not be elaborated further here.

[0096] The debris monitoring method based on heterogeneous data provided in this invention collects biological detection data and distance detection data by setting up data acquisition terminals, and transmits the collected detection data to a data processing gateway. This allows the data processing gateway to monitor debris in a specific area based on the distance detection data and biological detection data, avoiding false alarms caused by personnel or other biological activity. Simultaneously, the data acquisition devices in the data acquisition terminals have low deployment and maintenance costs and are less likely to cause privacy issues. Multiple data acquisition terminals are installed in different data acquisition areas and self-networked with the data processing gateway to collect data from different areas. This facilitates application in various situations requiring debris monitoring, such as in corridors to monitor multiple emergency evacuation routes in a building. This helps managers achieve more comprehensive and accurate monitoring and analysis of evacuation routes, avoiding the limitations of traditional monitoring technologies in terms of monitoring range and accuracy. Real-time monitoring and early warning are achieved, greatly improving the timeliness and accuracy of debris accumulation monitoring and providing stronger support for fire prevention and control in high-rise buildings.

[0097] While the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of the invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the scope of protection of this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A debris monitoring system based on heterogeneous data, characterized in that, It includes multiple data acquisition terminals and a data processing gateway that communicates with each of the data acquisition terminals. Each data acquisition terminal has a corresponding data acquisition area. Each data acquisition terminal operates in a target acquisition terminal working mode, and the data acquisition area corresponding to the target acquisition terminal is set as the target acquisition area. The target acquisition terminal is used to detect the target acquisition area in real time using a biological monitoring device to obtain a first type of detection data and send the first type of detection data to the data processing gateway. It is also used to periodically detect the target acquisition area using a distance monitoring device to obtain a second type of detection data and send the second type of detection data to the data processing gateway. Furthermore, it is used to use the distance monitoring device to perform high-frequency detection on the target acquisition area for a preset time period after receiving high-frequency sampling information to obtain a second type of raw detection data and send the second type of raw detection data to the data processing gateway. The data processing gateway is used to determine whether there may be debris in the target collection area based on the second type of detection data. If so, it sends high-frequency sampling information to the target collection terminal and performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living organisms in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it determines that there is no debris in the target collection area within the corresponding preset time period. If it determines that there are no living organisms in the target collection area, it determines that there are debris in the target collection area within the preset time period. The target acquisition terminal periodically probes the target acquisition area using a distance monitoring device to obtain the second type of detection data, including: The target acquisition terminal uses a distance monitoring device to detect the target acquisition area once every preset time period to obtain the second type of raw sampling data. The second type of original sampling data is denoised to obtain the second type of denoised sampling data. The second type of denoised sampling data is divided into multiple preset windows by using a sliding window method, and the average distance of each preset window is calculated. The average distances of all preset windows are then combined to form the second type of detection data.

2. The system according to claim 1, characterized in that, The data processing gateway determines whether there may be debris in the target acquisition area based on the second type of detection data, including: The data processing gateway determines whether the average distance of each preset window in the second type of detection data is greater than or equal to its corresponding area distance threshold. If so, it determines that there are no debris in the current target acquisition area; otherwise, it determines that there may be debris in the current target acquisition area.

3. The system according to claim 1, characterized in that, The data processing gateway performs in-depth preprocessing on the second type of raw detection data to determine whether there may be debris in the target acquisition area, including: The data processing gateway performs denoising processing on the second type of raw detection data to obtain the second type of denoised raw data, and transmits the second type of denoised raw data to the debris detection neural network so that the debris detection neural network outputs the detection result. The debris detection neural network is obtained by training a preset neural network based on the original detection data training dataset. The original detection data training dataset includes multiple sets of second-class original sampling data and a label corresponding to each set of second-class original sampling data. The label indicates whether debris may be present or not.

4. The system according to claim 3, characterized in that, The preset neural network is a CNN+LSTM structure neural network.

5. The system according to claim 1, characterized in that, The biological monitoring device includes an infrared sensor and a network sniffer; the first type of detection data includes infrared detection data and sniffer detection data; the data processing gateway determines whether biological activity may exist in the target collection area based on the first type of detection data within a corresponding preset time period, including: If an infrared signal is detected in the infrared detection data, it is determined that there may be living organisms in the target collection area; otherwise, it is determined that there is network device information in the sniffing detection data. If there is information, it is determined that there may be living organisms in the target collection area; otherwise, it is determined that there are no living organisms in the target collection area.

6. The system according to claim 1, characterized in that, The distance monitoring device is at least one of a distance sensor, ultrasonic radar, and lidar.

7. The system according to claim 1, characterized in that, It also includes a cloud server that is communicatively connected to the data processing gateway. The data processing gateway is also used to send the first type of detection data to the cloud server and send the second type of raw detection data and the determination result of whether there are foreign objects in the target collection area within the corresponding preset time period to the cloud server. The cloud server is used to store the first type of detection data, the second type of raw detection data, and the determination results of whether there are debris in the target collection area within the corresponding preset time period, and is also used to display the determination results of whether there are debris in the target collection area.

8. The system according to claim 7, characterized in that, The data acquisition terminal and the data processing gateway achieve data transmission through LoRa transmission technology, and the data processing gateway and the cloud server achieve data transmission through 5G transmission technology.

9. A debris monitoring method based on heterogeneous data, implemented based on the debris monitoring system based on heterogeneous data according to any one of claims 1-8, characterized in that, include: The target acquisition terminal uses a biological monitoring device to detect the target acquisition area in real time to obtain a first type of detection data, and sends the first type of detection data to the data processing gateway. At the same time, it uses a distance monitoring device to periodically detect the target acquisition area to obtain a second type of detection data, and sends the second type of detection data to the data processing gateway. The data processing gateway determines whether there may be debris in the target acquisition area based on the second type of detection data. If so, it sends high-frequency sampling information to the target acquisition terminal; otherwise, it does not send information to the target acquisition terminal. After receiving high-frequency sampling information, the target acquisition terminal uses the distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain the second type of raw detection data, and sends the second type of raw detection data to the data processing gateway. The data processing gateway performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living organisms in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it determines that there is no debris in the target collection area within the corresponding preset time period. If it determines that there are no living organisms in the target collection area based on the first type of detection data within the corresponding preset time period, it determines that there is debris in the target collection area within the preset time period.

Images