Sundry monitoring system and method based on heterogeneous data

Abstract

The invention discloses a foreign matter monitoring system and method based on heterogeneous data, and the system comprises a plurality of data collection terminals and a data processing gateway, and the data collection terminals are used for carrying out the real-time detection of a data collection region through a biological monitoring device, so as to obtain a first type of detection data, and transmitting the first type of detection data to the data processing gateway. Periodically detecting a data acquisition area through a distance monitoring device to obtain second type detection data and send the second type detection data to a data processing gateway, and obtaining second type original detection data and sending the second type original detection data to the data processing gateway after high-frequency sampling information is received; and the data processing gateway respectively processes and judges the second type of detection data, the second type of original detection data and the first type of detection data so as to judge that the sundries exist in the data acquisition area within the preset time period in combination with each judgment result. False alarm of sundry monitoring caused by walking of personnel or other organisms is avoided, the maintenance cost is low, the personnel privacy problem is not likely to be caused, and the timeliness and accuracy of sundry stacking monitoring are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of area detection, and particularly to a sundry monitoring system and method based on heterogeneous data. Background Art

[0002] With the rapid development of urban economy, the number of high-rise buildings is increasing continuously. At present, there is a problem of sundry accumulation in the evacuation emergency passages of high-rise buildings, which poses a severe challenge to public safety. For example, the sundries accumulated in the evacuation emergency passages will not only hinder the evacuation of personnel, but also increase the danger level in case of emergency, bringing more hidden dangers to public safety. Therefore, improving the monitoring ability of sundry stacking in the evacuation emergency passages of high-rise buildings is an important and urgent task at present.

[0003] Most existing sundry monitoring technologies use video acquisition technology. Although the coverage range is wide, the deployment and maintenance costs are relatively high. At the same time, due to the issue of personnel privacy, it is easy to cause disputes and has certain limitations. In addition, existing monitoring technologies usually also ignore the influence of the behavior of personnel in the building evacuation emergency passages on the monitoring of sundry stacking in the building evacuation emergency passages. Therefore, there is an urgent need for a sundry monitoring system that can overcome the problems of low cost, no privacy issues of personnel, and no influence of personnel in the building evacuation emergency passages on the monitoring of sundries in the building evacuation emergency passages. Summary of the Invention

[0004] In view of the above-mentioned disadvantages of the prior art, the purpose of the present application is to provide a sundry monitoring system and method based on heterogeneous data, which are used to solve the problems existing in the existing sundry monitoring technologies, such as high deployment and maintenance costs, easy to cause privacy issues of personnel, and the behavior of personnel in the building evacuation emergency passages will affect the monitoring of sundries in the building evacuation emergency passages, resulting in false alarms.

[0005] In a first aspect, the present application provides a sundry monitoring system based on heterogeneous data, including a plurality of data acquisition terminals and a data processing gateway that communicates with all the data acquisition terminals respectively. Each data acquisition terminal has a corresponding data acquisition area, and each data acquisition terminal works in a target acquisition terminal working mode, and the data acquisition area corresponding to the target acquisition terminal is set as the target acquisition area.

[0006] The target acquisition terminal is used to detect the target acquisition area in real time through a biological monitoring device to obtain first-class detection data, and send the first-class detection data to the data processing gateway. It is also used to periodically detect the target acquisition area through a distance monitoring device to obtain second-class detection data, and send the second-class detection data to the data processing gateway. It is further used to, after receiving high-frequency sampling information, use the distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain second-class original detection data, and send the second-class original detection data to the data processing gateway;

[0007] The data processing gateway is used to determine whether there may be sundries in the target acquisition area based on the second-class detection data. If so, it sends high-frequency sampling information to the target acquisition terminal. It is also used to perform in-depth preprocessing on the second-class original detection data to determine whether there may be sundries in the target acquisition area. If so, it determines whether there may be organisms in the target acquisition area based on the first-class detection data within the corresponding preset time period. If so, it determines that the target acquisition area cannot be determined to have sundries within the corresponding preset time period. If it is determined that there are no organisms in the target acquisition area, it is determined that there are sundries in the target acquisition area within the preset time period.

[0008] In an embodiment of the present application, the target acquisition terminal periodically detects the target acquisition area through a distance monitoring device to obtain second-class detection data, including:

[0009] The target acquisition terminal detects the target acquisition area through a distance monitoring device once every preset time period to obtain second-class original sampling data;

[0010] Perform denoising processing on the second-class original sampling data to obtain second-class denoised sampling data. Divide the second-class denoised sampling data into multiple preset windows by means of a sliding window, and calculate the distance mean value of each preset window. The distance mean values of all the preset windows are aggregated into the second-class detection data.

[0011] In an embodiment of the present application, the data processing gateway determines whether there may be sundries in the target acquisition area based on the second-class detection data, including:

[0012] The data processing gateway determines whether the distance mean value of each preset window in the second-class detection data is greater than or equal to its corresponding area distance threshold. If so, it determines that there are no sundries in the current target acquisition area. Otherwise, it determines that there may be sundries in the current target acquisition area.

[0013] In an embodiment of the present application, the data processing gateway performs in-depth preprocessing on the second type of original detection data to determine whether there may be debris in the target acquisition area, including:

[0014] The data processing gateway performs denoising processing on the second type of original detection data to obtain second type of denoised original data, and transmits the second type of denoised original data to the debris determination neural network, so that the debris determination neural network outputs a determination result;

[0015] Wherein the debris determination neural network is obtained by training a preset neural network based on an original detection data training dataset, and the original detection data training dataset includes multiple groups of second type of original sampling data and the label corresponding to each group of the second type of original sampling data, and the label is that there may be debris or there is no debris.

[0016] In an embodiment of the present application, the preset neural network is a CNN+LSTM structure neural network.

[0017] In an embodiment of the present application, the biological monitoring device includes an infrared sensor and a network sniffer, and the first type of detection data includes infrared detection data and sniffing detection data; the data processing gateway determines whether there may be a biological in the target acquisition area based on the first type of detection data within a corresponding preset time period, including:

[0018] Determine whether an infrared signal is detected in the infrared detection data. If so, it is determined that there may be a biological in the target acquisition area. Otherwise, determine whether there is network device information in the sniffing detection data. If so, it is determined that there may be a biological in the target acquisition area. Otherwise, it is determined that there is no biological in the target acquisition area.

[0019] In an embodiment of the present application, the distance monitoring device is at least one of a distance sensor, an ultrasonic radar, a lidar or a point cloud radar.

[0020] In an embodiment of the present application, the debris monitoring system based on heterogeneous data further includes a cloud server communicatively connected to the data processing gateway. The data processing gateway is further configured to send the first type of detection data to the cloud server, and send the second type of original detection data and the determination result of whether there is debris in the target acquisition area within its corresponding preset time period to the cloud server;

[0021] The cloud server is configured to store the first type of detection data, the second type of original detection data, and the determination result of whether there is debris in the target acquisition area within its corresponding preset time period, and is further configured to display the determination result of whether there is debris in the target acquisition area.

[0022] In an embodiment of the present application, the data acquisition terminal and the data processing gateway implement data transmission through LoRa transmission technology, and the data processing gateway and the cloud server implement data transmission through 5G transmission technology.

[0023] In a second aspect, the present application provides a method for monitoring sundries based on heterogeneous data, which is implemented based on the system for monitoring sundries based on heterogeneous data, and includes:

[0024] The target acquisition terminal uses a biological monitoring device to perform real-time detection on the target acquisition area to obtain the first type of detection data, and sends the first type of detection data to the data processing gateway. At the same time, it uses a distance monitoring device to perform periodic detection on the target acquisition area to obtain the second type of detection data, and sends the second type of detection data to the data processing gateway;

[0025] The data processing gateway determines whether there may be sundries in the target acquisition area based on the second type of detection data. If so, it sends high-frequency sampling information to the target acquisition terminal; otherwise, it does not send information to the target acquisition terminal;

[0026] After receiving the high-frequency sampling information, the target acquisition terminal uses the distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain the second type of original detection data, and sends the second type of original detection data to the data processing gateway;

[0027] The data processing gateway performs in-depth preprocessing on the second type of original detection data to determine whether there may be sundries in the target acquisition area. If so, it determines whether there may be organisms in the target acquisition area based on the first type of detection data within the corresponding preset time period. If so, it determines that the target acquisition area cannot be determined to have sundries within the corresponding preset time period. If it is determined that there are no organisms in the target acquisition area based on the first type of detection data within the corresponding preset time period, it is determined that there are sundries in the target acquisition area within the preset time period.

[0028] Compared with the prior art, one or more of the above embodiments may have the following advantages or beneficial effects:

[0029] Applying the debris monitoring system based on heterogeneous data provided by the embodiments of the present invention, by setting data acquisition terminals to collect biological detection data and distance detection data, and transmitting the collected detection data to a data processing gateway, so that the data processing gateway can monitor debris in a specific area based on the distance detection data and biological detection data, avoiding false alarms of debris monitoring caused by the movement of personnel or other organisms; at the same time, the deployment and maintenance cost of data acquisition devices in the data acquisition terminals is low and it is not easy to cause personnel privacy problems. Install multiple data acquisition terminals in different data acquisition areas respectively, and perform self-networking of multiple data acquisition terminals with the data processing gateway to achieve data acquisition of different data acquisition areas, which is further convenient for application in various occasions that require debris monitoring. For example, it can be set in the corridor to achieve debris monitoring of multiple emergency evacuation channels in a building, helping managers to achieve more comprehensive and accurate monitoring and analysis of the emergency evacuation channels, avoiding the limitations of traditional monitoring technologies in terms of monitoring range and accuracy, realizing real-time monitoring and early warning, greatly improving the timeliness and accuracy of debris stacking monitoring, and providing stronger support for the fire prevention and control of high-rise buildings.

[0030] Other features and advantages of the present invention will be described in the following specification, and will be partially obvious from the specification, or understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in the specification, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0032] Figure 1 It shows a schematic network structure diagram of the debris monitoring system based on heterogeneous data described in the embodiments of the present application.

[0033] Figure 2 It shows an application scenario diagram of the debris monitoring system based on heterogeneous data described in the embodiments of the present application applied to high floors.

[0034] Figure 3 It shows a signal interaction schematic diagram of a data acquisition terminal and a data processing gateway in the debris monitoring system based on heterogeneous data described in the embodiments of the present application.

[0035] Figure 4 It shows a schematic structural diagram of a data acquisition terminal in the debris monitoring system based on heterogeneous data described in the embodiments of the present application.

[0036] Figure 5It shows a schematic structural diagram of the data processing gateway in the heterogeneous data-based sundry monitoring system according to the embodiments of the present application.

[0037] Figure 6 It shows a schematic flow diagram of the heterogeneous data-based sundry monitoring method according to the embodiments of the present application.

[0038] Figure 7 It shows a schematic process diagram of the heterogeneous data-based sundry monitoring method according to the embodiments of the present application. Detailed implementation manners

[0039] The following will describe in detail the implementation manners of the present invention in combination with the accompanying drawings and embodiments, so as to fully understand how the present invention uses technical means to solve technical problems and achieve the implementation process of technical effects and implement accordingly. It should be noted that as long as there is no conflict, each embodiment in the present invention and each feature in each embodiment can be combined with each other, and the formed technical solutions are all within the protection scope of the present invention.

[0040] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner. Therefore, only the components related to the present application are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in its actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0041] The sniffing technology is a wireless technology used to detect and identify Bluetooth devices, which has advantages such as low power consumption, low cost and easy deployment; for the sniffing function, people only need to carry devices with information capture functions such as mobile phones, tablets, smart bracelets or earphones and turn on the Bluetooth or WiFi function.

[0042] The following embodiments of the present application provide a heterogeneous data-based sundry monitoring system and method, which solve the problems existing in the existing sundry monitoring technology, such as high deployment and maintenance costs, easy to cause personnel privacy problems, and the personnel in the building evacuation emergency passage will affect the sundry monitoring in the building evacuation emergency passage, resulting in monitoring false alarms.

[0043] The existing contact recognition technology has problems such as being unable to identify multi-contact point signals, being unable to effectively identify contact signals, and being unable to accurately identify multi-tap events.

[0044] The following will elaborate in detail the principle and implementation manner of a heterogeneous data-based sundry monitoring system and method of this embodiment in combination with the accompanying drawings, so that those skilled in the art can understand the heterogeneous data-based sundry monitoring system and method of this embodiment without creative labor.

[0045] The debris monitoring system based on heterogeneous data in this application can be applied to multiple evacuation emergency channels in a building to monitor debris in multiple evacuation emergency channels of the building. Figure 2 This is an intended application scenario of the debris monitoring system based on heterogeneous data in this application embodiment for high-rise buildings. Applying the debris monitoring system based on heterogeneous data to multiple evacuation emergency channels in a building can help managers achieve more comprehensive and accurate monitoring and analysis of the evacuation emergency channels, avoid the limitations of traditional monitoring technologies in terms of monitoring range and accuracy, achieve real-time monitoring and early warning, greatly improve the timeliness and accuracy of debris pile monitoring, and provide stronger support for fire prevention and control in high-rise buildings. At the same time, the debris monitoring system based on heterogeneous data in this application can also be applied to other multi-region debris monitoring scenarios to monitor debris in the corresponding multi-regions. This application does not make fixed restrictions on it here.

[0046] As Figure 1 shown, this embodiment provides a debris monitoring system based on heterogeneous data, which specifically includes multiple data acquisition terminals and a data processing gateway. Among them, the multiple data acquisition terminals are respectively in communication with the data processing gateway, and each data acquisition terminal has a corresponding data acquisition area.

[0047] Among them, the data acquisition methods of each data acquisition terminal and the process of data interaction between each data acquisition terminal and the data processing gateway are the same. Therefore, in order to more clearly illustrate the working methods of the multiple data acquisition terminals, it is assumed that the data acquisition terminals all work in the working mode of the target acquisition terminal, and the data acquisition area corresponding to the target acquisition terminal is the target acquisition area.

[0048] Specifically, the target acquisition terminal is mainly used to detect the target acquisition area in real time through a biological monitoring device to obtain the first type of detection data, and send the first type of detection data to the data processing gateway, and is also used to periodically detect the target acquisition area through a distance monitoring device to obtain the second type of detection data, and send the second type of detection data to the data processing gateway. It is also used to perform high-frequency detection on the target acquisition area within a preset time period by using the distance monitoring device after receiving the high-frequency sampling information to obtain the second type of original detection data, and send the second type of original detection data to the data processing gateway.

[0049] The data processing gateway is mainly used to determine whether there may be sundries in the target acquisition area based on the second type of detection data. If so, it sends high-frequency sampling information to the target acquisition terminal, and is also used to deeply preprocess the second type of original detection data to determine whether there may be sundries in the target acquisition area. If so, it determines whether there may be organisms in the target acquisition area based on the first type of detection data within the corresponding preset time period. If so, it is determined that the sundries in the target acquisition area cannot be determined within the corresponding preset time period. If it is determined that there are no organisms in the target acquisition area, it is determined that there are sundries in the target acquisition area within the preset time period.

[0050] The sundries monitoring system based on heterogeneous data provided by the embodiments of the present invention collects biological detection data and distance detection data by setting data acquisition terminals, and transmits the collected detection data to the data processing gateway, so that the data processing gateway can monitor sundries in a specific area based on the distance detection data and biological detection data, avoiding false alarms of sundries monitoring caused by the movement of personnel or other organisms; at the same time, the deployment and maintenance cost of the data acquisition devices in the data acquisition terminals is low and it is not easy to cause personnel privacy problems.

[0051] The structures and working methods of the data acquisition terminals and the data processing gateway in the sundries monitoring system based on heterogeneous data in this embodiment are specifically described below.

[0052] Refer to Figure 1 As shown, a star network topology can be formed through communication between multiple data acquisition terminals and the data processing gateway. Specifically, data transmission between the data acquisition terminals and the data processing gateway can be realized through LoRa transmission technology. For example, the data processing gateway can preset M frequency points (M ≤ 64) through the LoRa transmission module, and N data acquisition terminals can be connected under each frequency point. At this time, the data processing gateway can support the access of M * N data acquisition terminals. The data acquisition terminal generates a random time offset within s seconds after initial power-on to avoid random access conflicts generated by devices under the same frequency point; the time offset obtained after initial power-on remains unchanged within each reporting period, thus effectively avoiding data packet loss caused by access conflicts and realizing the self-organizing network communication function. The data processing gateway, as the center of all data acquisition terminals, has the functions of data sorting and receiving and command issuing.

[0053] In practical applications, each data acquisition terminal needs to be reasonably installed in the data acquisition area. During installation, the position calibration of the biological monitoring device and the distance monitoring device needs to be carried out. By observing the terminal feedback values in real time several times, it is ensured that the positions and angles of the biological monitoring device and the distance monitoring device used are correctly installed, and calibration is carried out to eliminate system errors and reduce background noise. At the same time, the data acquisition area targeted by each data acquisition terminal is specified, the ID of the data acquisition terminal and its installation location are recorded, the data acquisition area of each data acquisition terminal is recorded (for example, the area responsible for the corridor in a high-rise building), and a mapping number is made for the unique ID of each data acquisition terminal, such as: Building_Sensor_001A, Building_Sensor_002A... A data acquisition terminal data record table is established to facilitate querying the settings of each data acquisition terminal by looking up the data acquisition terminal data record table.

[0054] Reference Figure 3 As shown, taking the target sampling terminal as an example, the working mode of the data acquisition terminal and the information interaction mode between the data acquisition terminal and the data processing gateway will be described in detail below. The target sampling terminal can be any data acquisition terminal, and the data acquisition area corresponding to the target acquisition terminal is the target acquisition area.

[0055] In the normal operation of the heterogeneous data-based sundry monitoring system of this embodiment, the target acquisition terminal can detect the target acquisition area in real time through the biological monitoring device to obtain the first type of detection data, and then send the detected first type of detection data to the data processing gateway.

[0056] In an embodiment of the present application, it can be set that the biological monitoring device in the target acquisition terminal includes an infrared sensor and a network sniffer. Among them, the infrared sensor can detect obvious organisms such as people or animals in the data acquisition area; the network sniffer can detect organisms carrying devices with Bluetooth or wireless functions such as mobile phones, tablets, bracelets, earphones, and animal locators in the data acquisition area, such as people or animals hiding in corners or behind foreign objects. In this embodiment, the infrared sensor and the network sniffer are used as the biological monitoring device in the target acquisition terminal, which can not only detect obvious people or animals in the target acquisition area, but also detect people and animals in difficult-to-identify corners such as corners or behind foreign objects in the data acquisition area through the network sniffer; realize a more comprehensive identification of people and animals in the data acquisition area, which helps to improve the intelligence and accuracy of the heterogeneous data-based sundry monitoring system.

[0057] In an embodiment of the present application, the network sniffer can be implemented by Esp32 to capture information in real time.

[0058] The target acquisition terminal uses an infrared sensor and a network sniffer to detect the target acquisition area in real time to obtain the first type of detection data, which includes infrared detection data and sniffer detection data. Then, the first type of detection data is transmitted to the data processing gateway in real time or after packing the data within a period of time. Table 1 is an example of the data frame format after packing the data within a period of time. Table 2 is a data format of the sniffer detection data in Table 1.

[0059] Table 1

[0060]

[0061] Table 2

[0062]

[0063] In this embodiment, when the sundry monitoring system based on heterogeneous data is working properly, the target acquisition terminal is also used to periodically detect the target acquisition area through a distance monitoring device to obtain the second type of detection data, and send the second type of detection data to the data processing gateway.

[0064] In an embodiment of the present application, the distance monitoring device in the target acquisition terminal can be set as at least one of a distance sensor, an ultrasonic radar, a lidar, or a point cloud radar. Specifically, the distance monitoring device is reasonably installed in the corresponding target acquisition area so that the distance monitoring device can monitor the distance in the target acquisition area, and determine whether there are sundries in the target acquisition area by comparing the change in the relative distance before and after the sub-areas in the target acquisition area. Since the quantity of data collected by the distance monitoring device is large and the processing is relatively complex, the target acquisition terminal uses a periodic detection method to obtain the second type of detection data. Specifically, the target acquisition terminal can use a periodic scheduling algorithm to process timing tasks, such as periodically sending a signal acquisition instruction to the distance monitoring device. Each time the distance monitoring device receives the signal acquisition instruction, it monitors the target acquisition area once to obtain the corresponding second type of detection data. The periodic scheduling algorithm can execute tasks at fixed time intervals to ensure that the system can respond to various events in a timely manner.

[0065] Furthermore, it can be set that the target acquisition terminal performs a detection on the target acquisition area through the distance monitoring device every preset time period to obtain the second type of original sampling data. After the target acquisition terminal obtains the second type of original sampling data, it is also necessary to perform denoising processing on the second type of original sampling data to obtain the second type of denoised sampling data to ensure the reliability of the measurement value. Specifically, the Kalman filter algorithm can be used to implement the filtering of the second type of original sampling data. Then, the sliding window algorithm is used to calculate the mean value of the distance data. The sliding window algorithm has a linear time complexity and can effectively process a large amount of data to achieve real-time processing. Specifically, the second type of denoised sampling data is divided into multiple preset windows in a sliding window manner, and the distance mean value of each preset window is calculated. The distance mean values of all preset windows are aggregated into the second type of detection data. And to optimize the calculation of the distance data, a weighted moving average method can also be used when calculating the distance mean value of the preset window to ensure the accuracy of the result.

[0066] Finally, the obtained second type of detection data is packaged and sent to the data processing gateway. Among them, Table 3 is a format data frame of the distance mean value of a single preset window in the second type of detection data.

[0067] Table 3

[0068]

[0069] When the debris monitoring system based on heterogeneous data in this embodiment is working properly, the target acquisition terminal is also used to perform high-frequency detection on the target acquisition area through the distance monitoring device in a preset time period after receiving the high-frequency sampling information to obtain the second type of original detection data, and send the second type of original detection data to the data processing gateway.

[0070] After the data processing gateway processes the received second type of detection data, it can initially judge whether there may be debris in the target acquisition area. When the data processing gateway determines that there may be debris in the target acquisition area, it will feedback the high-frequency sampling information to the corresponding target acquisition terminal, that is, send an instruction to continuously send a high-frequency acquisition signal to the target acquisition terminal for subsequent further judgment based on the collected second type of original detection data. When the data processing gateway receives the high-frequency sampling information, it will continuously perform high-frequency detection on the target acquisition area through the distance monitoring device in a preset time period to obtain the second type of original detection data. At this time, the target acquisition terminal does not need to process the second type of original detection data and directly packages and sends it to the data processing gateway.

[0071] Reference Figure 4As shown in the figure, based on the above description, it can be known that the target acquisition terminal needs to include a central processor, a biological monitoring device and a distance detection device respectively connected to the central processor. At the same time, the target acquisition terminal also needs to include an independent power supply module, a corresponding position debugging module and a LoRa transmission module. Among them, the power supply module, the position debugging module and the LoRa transmission module are also respectively connected to the central processor to implement corresponding functions. The target acquisition terminal can also be designed to include other structures, and no fixed restrictions are imposed on them here.

[0072] When the debris monitoring system based on heterogeneous data in this embodiment is working normally, the data processing gateway is used to judge whether there may be debris in the target acquisition area based on the second type of detection data. If so, high-frequency sampling information is sent to the target acquisition terminal.

[0073] Specifically, the data processing gateway will divide the target acquisition area into multiple sub-area modules, and each sub-area module has its corresponding area distance threshold. After receiving the second type of detection data, the data processing gateway will compare the preset window with the sub-area module, and sequentially judge whether the distance mean value of each preset window is greater than or equal to the area distance threshold of its corresponding sub-area module. If the distance mean values of all preset windows in the second type of detection data are greater than the area distance threshold of their corresponding sub-area modules, it means that the current target acquisition area has not changed spatially compared with the initial state (that is, there is no change in the space of the target acquisition area caused by debris accumulation). If there is one or more preset windows in the second type of detection data whose distance mean values are not greater than the area distance threshold of their corresponding sub-area modules, it means that the current target acquisition area has changed spatially compared with the initial state. At this time, it can be determined that there may be debris in the current target acquisition area, and then the data processing gateway sends high-frequency sampling information to the target acquisition terminal corresponding to the above second type of detection data, so that the target acquisition terminal continuously and highly frequently acquires the second type of original detection data based on the high-frequency sampling information within a preset time period.

[0074] It should be noted that the sub-region modules in the target acquisition area can be divided based on the actual situation, and the area distance threshold corresponding to each sub-region module is calculated adaptively based on the actual situation. The area distance threshold corresponding to each sub-region module can change dynamically based on the actual situation of the corresponding target acquisition area, so as to realize the monitoring of scenarios with large fluctuations in monitoring data. Usually, the target acquisition area does not change, but in special cases (such as renovating the spatial pattern), the target acquisition area will change to a certain extent. At this time, the distance monitoring device in the target acquisition terminal can periodically collect signals from the target acquisition area, and calculate the new area distance threshold corresponding to each sub-region module through the collected signals. It should be noted that the update of the area distance threshold does not need to be too frequent, and it can be adjusted once every six months or one year. Dynamically adjusting the area distance threshold according to the actual scenario in practical applications can improve the robustness of the system, and this method enables the system to effectively judge whether there are sundries in the target acquisition area and achieve a more accurate detection function.

[0075] When the sundries monitoring system based on heterogeneous data in this embodiment is working normally, the data processing gateway is also used to perform in-depth preprocessing on the second type of original detection data to judge whether there may be sundries in the target acquisition area. If the data processing gateway judges that there may be sundries in the target acquisition area after in-depth preprocessing of the second type of original detection data, the data processing gateway needs to make a further judgment based on the first type of detection data within a preset time period. If the data processing gateway judges that there are no sundries in the target acquisition area after in-depth preprocessing of the second type of original detection data, it can directly determine that there are no sundries in the target acquisition area. When the data processing gateway makes a judgment based on the first type of detection data within a preset time period, it needs to judge whether there may be organisms in the target acquisition area. If so, it means that there are organisms such as people or animals in the target acquisition area during the preset time period, and the data collected at this time will affect the sundries judgment result. Therefore, it can be directly determined that the sundries in the target acquisition area cannot be judged during the corresponding preset time period; if the data processing gateway needs to judge that there are no organisms in the target acquisition area based on the first type of detection data within a preset time period, it means that there are no organisms such as people or animals in the target acquisition area during the preset time period. At this time, the conclusion of the in-depth preprocessing of the second type of original detection data can be used as the final conclusion, that is, it is determined that there are sundries in the target acquisition area during the preset time period.

[0076] Further, the data processing gateway performs in-depth preprocessing on the second type of original detection data to determine whether there may be debris in the target acquisition area, which specifically includes: the data processing gateway performs denoising processing on the second type of original detection data to obtain the second type of denoised original data, and the denoising method can adopt the Kalman filter denoising method or other reasonable methods. Then, the second type of denoised original data is transmitted to the debris determination neural network, and the debris determination neural network outputs the corresponding determination result, where the determination result is that there may be debris in the target acquisition area or there is no debris in the target acquisition area. Among them, the debris determination neural network is obtained by training a preset neural network based on the original detection data training dataset. The original detection data training dataset is a training dataset obtained by the target acquisition terminal through the distance monitoring device to perform multiple signal acquisitions on the target acquisition area. That is, the original detection data training dataset includes multiple groups of the second type of original sampling data and the labels corresponding to each group of the second type of original sampling data. The second type of original sampling data in this training dataset should be sufficient, and the position and shape of the debris included should be rich enough. Specifically, the labels corresponding to the second type of original sampling data are two types: there may be debris or there is no debris. In an embodiment of the present application, the preset neural network can select a CNN + LSTM structure neural network.

[0077] Meanwhile, in practical applications, the data processing gateway can actually receive the first type of detection data for all time periods, but will only further process the first type of detection data in the preset time period when there may be debris after in-depth preprocessing of the second type of original detection data. Specifically, the data processing gateway determines whether there may be organisms in the target acquisition area based on the first type of detection data in the corresponding preset time period, which specifically includes: determining whether an infrared signal is detected in the infrared detection data of the first type of detection data. If so, it indicates that there may be organisms in the target acquisition area. Otherwise, it further determines whether there is network device information (such as device Mac address, etc.) in the sniffing detection data of the first type of detection data. If so, it is determined that there may be organisms in the target acquisition area. Otherwise, it is determined that there are no organisms in the target acquisition area. This determination method can avoid missed detection of organisms such as people or animals hiding in corners or behind foreign objects in the target area, realize more comprehensive identification of organisms such as people and animals in the target acquisition area, and help improve the intelligence and accuracy of the debris monitoring system based on heterogeneous data.

[0078] The data processing gateway in this embodiment can also be set to have a parameter repair function, that is, the data processing gateway is set to have the function of monitoring whether the distance monitoring device has shifted and performing dynamic parameter repair. The data processing gateway regularly determines whether there is a position shift of the distance monitoring device based on the data monitored by the distance monitoring device. Specifically, during detection, the data acquisition terminal reports the original data of the distance monitoring device, and the gateway determines whether there is an offset of the distance monitoring device by analyzing the original data in multiple time dimensions. If no abnormality occurs, it continues to work normally. If the original data of the distance monitoring device in multiple time dimensions does not match and an angle abnormality occurs, then a spatial point cloud is generated to obtain an estimate of the actual corresponding angle of the distance monitoring device at that time, and it is sent to the management personnel so that they can manually correct the distance monitoring device. At the same time, the collected data can also be automatically corrected based on the estimated actual corresponding angle of the distance monitoring device at that time.

[0079] In an embodiment of the present application, the sundry monitoring system based on heterogeneous data further includes a cloud server communicatively connected to the data processing gateway.

[0080] When the sundry monitoring system based on heterogeneous data in this embodiment is working normally, the data processing gateway can also be used to send the first type of detection data to the cloud server, and send the second type of original detection data and the determination result of whether there are sundries in the target acquisition area within the preset time period corresponding to the second type of original detection data to the cloud server.

[0081] Specifically, in order to reduce the output transmission pressure of the data processing gateway, before the data processing gateway transmits the first type of detection data to the cloud server, it can perform processing such as compressing the first type of detection data. Specifically, for the data compression technology used in the processing of the first type of detection data, the purpose is to reduce the data transmission volume and improve efficiency. This kind of data compression processing mainly includes two aspects: deduplication and data integration; the deduplication processing removes duplicates of the device Mac addresses detected in the sniffing detection data of the first type of detection data, that is, removes the device information repeatedly detected in the data frame to ensure that the same device information only appears once in a data packet, reduces data redundancy, and saves transmission bandwidth; while the data integration processing integrates the data of the same device in the sniffing detection data of the first type of detection data into one data item, further reducing the data volume. In code implementation, these functions are realized by maintaining data caches, comparing newly received data, and integrating processing. Such a data processing method effectively improves the efficiency and performance of data transmission.

[0082] In an embodiment of the present application, the data processing gateway and the cloud server can realize data transmission through 5G transmission technology. That is, the data processing gateway can package the data to be sent into Json format data, and then transmit the packaged data to the cloud server through 5G transmission technology. Table 4 is a kind of Json format data packet.

[0083] Table 4

[0084]

[0085] It should be noted that the data reporting period of the data processing gateway can be set to be the same as that of the data acquisition terminal, or can be set to be different from that of the data acquisition terminal, and no fixed limit is imposed here.

[0086] When the heterogeneous data-based sundry monitoring system in this embodiment is working properly, the cloud server is mainly used to store the first type of detection data, the second type of original detection data, and the determination result of whether there are sundries in the target acquisition area within the corresponding preset time period for subsequent data query. At the same time, the cloud server is also used to display the determination result of whether there are sundries in the target acquisition area.

[0087] Specifically, the cloud server parses the received Json data packet, stores the first type of detection data and the second type of original detection data therein, and at the same time presents the corresponding sundry determination result.

[0088] Reference Figure 5 As shown, based on the above description, it can be known that the data processing gateway includes a central processing unit and a 5G transmission module, a LoRa transmission module, a data processing module, a data parsing module, etc. connected to the central processing unit. The data processing gateway can also be designed to include other structures (such as a log recording module, a serial communication module, etc.), and no fixed limit is imposed here.

[0089] When the debris monitoring system based on heterogeneous data provided by the embodiments of the present invention is applied to the building fire evacuation passage, it can improve the timeliness and accuracy of debris occupancy alarms in the fire evacuation passage: The system can monitor the debris accumulation in the corridor in real time, detect abnormalities in time and issue warning signals, avoiding the risk of the fire evacuation passage being blocked. The system conducts precise analysis based on the piled-up object data to avoid false alarms and missed alarms, and improve the reliability of fire alarms. Through the self-organizing network data collection terminal and the data processing gateway, the present invention can comprehensively cover the interior of high-rise buildings. Without collecting sensitive information of personnel, it combines a variety of non-intrusive data for precise analysis, improving the comprehensiveness and accuracy of monitoring. Since the data collection terminal is small in size, easy to deploy, and only one gateway device is required to complete the self-organizing network, it greatly reduces the labor costs required for deployment and maintenance. Compared with the deployment of video collection systems, the miniaturized and automated system architecture of the present invention makes the construction and management of the Internet of Things system easier and more efficient, reducing labor costs and operating costs. The terminal and gateway devices are small in size, low in overall power consumption, and small in data transmission volume, making the cost of the devices themselves relatively low. And using a single gateway to complete the self-organizing network of multiple devices further reduces data communication and maintenance costs, and is applicable to a wider range of scenarios and applications. The system can realize digital and model-based debris identification and business processing, which enables the system to automatically adjust parameters and strategies when facing different environments and workloads, improving the robustness and adaptability of the system.

[0090] As Figure 6 and Figure 7 shown, this embodiment also provides a debris monitoring method based on heterogeneous data. This method is implemented based on the debris monitoring system based on heterogeneous data in the above embodiment. Set the target collection terminal as any data collection terminal in the debris monitoring system based on heterogeneous data. Then the interaction mode between the target collection terminal and the data processing gateway in the debris monitoring method based on heterogeneous data specifically includes the following steps.

[0091] Step S101, the target collection terminal conducts real-time detection on the target collection area through the biological monitoring device to obtain the first type of detection data, and sends the first type of detection data to the data processing gateway. At the same time, it conducts periodic detection on the target collection area through the distance monitoring device to obtain the second type of detection data, and sends the second type of detection data to the data processing gateway.

[0092] Step S102, the data processing gateway judges whether there may be debris in the target collection area based on the second type of detection data. If so, it sends high-frequency sampling information to the target collection terminal, otherwise it does not send information to the target collection terminal.

[0093] Step S103: After receiving the high-frequency sampling information, the target acquisition terminal uses the distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain the second type of original detection data, and sends the second type of original detection data to the data processing gateway.

[0094] Step S104: The data processing gateway performs in-depth preprocessing on the second type of original detection data to determine whether there may be sundries in the target acquisition area. If so, it determines whether there may be organisms in the target acquisition area based on the first type of detection data within the corresponding preset time period. If so, it is determined that the sundries in the target acquisition area cannot be determined within the corresponding preset time period. If it is determined based on the first type of detection data within the corresponding preset time period that there are no organisms in the target acquisition area, it is determined that there are sundries in the target acquisition area within the preset time period.

[0095] The execution processes of the above steps are all the same as the working modes of the target acquisition terminal and the data processing gateway in the sundries monitoring system for heterogeneous data described above, and will not be elaborated here. And the data interaction methods between each data acquisition terminal and the data processing gateway are all the same as the above process, and will not be elaborated here.

[0096] The sundries monitoring method based on heterogeneous data provided by the embodiments of the present invention collects biological detection data and distance detection data by setting up data acquisition terminals, and transmits the collected detection data to the data processing gateway, so that the data processing gateway can monitor sundries in a specific area based on the distance detection data and biological detection data, avoiding false alarms of sundries monitoring caused by the movement of personnel or other organisms; at the same time, the deployment and maintenance cost of the data acquisition devices in the data acquisition terminal is low and it is not easy to cause problems of personnel privacy. A plurality of data acquisition terminals are respectively installed in different data acquisition areas, and a plurality of data acquisition terminals and the data processing gateway are self-organized into a network to realize data acquisition of different data acquisition areas, which is further convenient for application in various occasions that require sundries monitoring. For example, it can be set in the corridor to realize sundries monitoring of multiple evacuation channels in a building, helping managers to achieve more comprehensive and accurate monitoring and analysis of the evacuation channels, avoiding the limitations of traditional monitoring technologies in monitoring range and accuracy, realizing real-time monitoring and early warning, greatly improving the timeliness and accuracy of sundries stacking monitoring, and providing stronger support for the fire prevention and control of high-rise buildings.

[0097] Although the embodiments disclosed in the present invention are as above, the content described above is only an embodiment adopted for the convenience of understanding the present invention, and is not used to limit the present invention. Any person skilled in the art within the technical field to which the present invention belongs can make any modifications and changes in the form of implementation and details without departing from the spirit and scope disclosed by the present invention. However, the protection scope of the present invention shall still be subject to the scope defined by the appended claims.

Claims

1. A debris monitoring system based on heterogeneous data, characterized in that: It includes a plurality of data acquisition terminals and a data processing gateway that respectively communicates with all the data acquisition terminals, each of the data acquisition terminals has a corresponding data acquisition area, each of the data acquisition terminals works in a target acquisition terminal working mode, and the data acquisition area corresponding to the target acquisition terminal is set as the target acquisition area; The target acquisition terminal is used to perform real-time detection of the target acquisition area through a biological monitoring device to obtain first-type detection data, and send the first-type detection data to the data processing gateway, and to perform periodic detection of the target acquisition area through a distance monitoring device to obtain second-type detection data, and send the second-type detection data to the data processing gateway, and is also used to perform high-frequency detection of the target acquisition area in a preset time period using the distance monitoring device after receiving high-frequency sampling information to obtain second-type original detection data, and send the second-type original detection data to the data processing gateway; The data processing gateway is used to determine whether there may be debris in the target collection area based on the second type of detection data. If so, high-frequency sampling information is sent to the target collection terminal, and deep preprocessing is performed on the second type of original detection data to determine whether there may be debris in the target collection area. If so, based on the first type of detection data within the corresponding preset time period, it is determined whether there may be living things in the target collection area. If so, it is determined that no debris can be determined in the target collection area within the corresponding preset time period. If it is determined that there are no living things in the target collection area, it is determined that there are debris in the target collection area within the preset time period.

2. The system according to claim 1, characterized in that The target acquisition terminal periodically detects the target acquisition area through a distance monitoring device to obtain the second type of detection data, including: The target acquisition terminal detects the target acquisition area once every preset time period through a distance monitoring device to obtain the second type of original sampling data; The second category of original sampling data is denoised to obtain second category denoised sampling data, the second category of denoised sampling data is divided into multiple preset windows by a sliding window method, and the distance mean of each preset window is calculated, and the distance mean values ​​of all the preset windows are combined into the second category of detection data.

3. The system according to claim 2, characterized in that The data processing gateway determines whether there may be debris in the target collection area based on the second type of detection data, including: The data processing gateway determines whether the distance mean of each preset window in the second type of detection data is greater than or equal to the distance threshold of its corresponding area. If so, it is determined that there is no debris in the current target collection area; otherwise, it is determined that there may be debris in the current target collection area.

4. The system according to claim 1, characterized in that The data processing gateway performs deep preprocessing on the second type of raw detection data to determine whether there may be debris in the target collection area, including: The data processing gateway performs denoising on the second type of original detection data to obtain second type of denoised original data, and transmits the second type of denoised original data to the debris determination neural network, so that the debris determination neural network outputs a determination result; The debris determination neural network is obtained by training a preset neural network based on an original detection data training data set, wherein the original detection data training data set includes multiple groups of second-category original sampling data and labels corresponding to each group of second-category original sampling data, and the labels are the possible presence of debris or the absence of debris.

5. The system according to claim 4, characterized in that The preset neural network is a CNN+LSTM structure neural network.

6. The system according to claim 1, characterized in that The biological monitoring device includes an infrared sensor and a network sniffer, and the first type of detection data includes infrared detection data and sniffing detection data; the data processing gateway determines whether there may be biological organisms in the target collection area based on the first type of detection data within a corresponding preset time period, including: Determine whether an infrared signal is detected in the infrared detection data. If so, determine that there may be living things in the target collection area. Otherwise, determine whether there is network device information in the sniffing detection data. If so, determine that there may be living things in the target collection area. Otherwise, determine that there are no living things in the target collection area.

7. The system according to claim 1, characterized in that The distance monitoring device is at least one of a distance sensor, an ultrasonic radar, a laser radar or a point cloud radar.

8. The system according to claim 1, characterized in that Also included is a cloud server in communication with the data processing gateway, the data processing gateway is further used to send the first type of detection data to the cloud server, and send the second type of original detection data and the corresponding determination result of whether there are debris in the target collection area within the preset time period to the cloud server; The cloud server is used to store the first type of detection data, the second type of original detection data, and the corresponding results of whether there are debris in the target collection area within a preset time period, and is also used to display the results of whether there are debris in the target collection area.

9. The system according to claim 8, characterized in that The data acquisition terminal and the data processing gateway realize data transmission through LoRa transmission technology, and the data processing gateway and the cloud server realize data transmission through 5G transmission technology.

10. A method for monitoring debris based on heterogeneous data, implemented based on the system for monitoring debris based on heterogeneous data as claimed in any one of claims 1 to 9, characterized in that: include: The target acquisition terminal detects the target acquisition area in real time through a biological monitoring device to obtain first-type detection data, and sends the first-type detection data to the data processing gateway, and at the same time periodically detects the target acquisition area through a distance monitoring device to obtain second-type detection data, and sends the second-type detection data to the data processing gateway; The data processing gateway determines whether there may be debris in the target acquisition area based on the second type of detection data, and if so, sends high-frequency sampling information to the target acquisition terminal, otherwise, does not send information to the target acquisition terminal; After receiving the high-frequency sampling information, the target acquisition terminal uses the distance monitoring device to perform high-frequency detection on the target acquisition area within a preset time period to obtain the second type of original detection data, and sends the second type of original detection data to the data processing gateway; The data processing gateway performs deep preprocessing on the second type of original detection data to determine whether there may be debris in the target collection area. If so, it determines whether there may be living things in the target collection area based on the first type of detection data within the corresponding preset time period. If so, it is determined that debris cannot be determined in the target collection area within the corresponding preset time period. If it is determined that there are no living things in the target collection area based on the first type of detection data within the corresponding preset time period, it is determined that there are debris in the target collection area within the preset time period.

Images