Signal fusion sensing system and method

Abstract

The embodiment of the invention provides a signal fusion sensing system and method, and the system comprises a master gateway and a slave gateway, the master gateway comprises a first optical fiber sensing sub-module which is used for collecting a first backscattering light signal through a branch optical fiber and extracting a first feature of the first backscattering light signal, the first wireless signal sensing sub-module is used for collecting a first wireless signal of communication between a main gateway and a terminal and extracting a second feature of the first wireless signal, and the first feature fusion algorithm sub-module is used for performing feature fusion analysis on the first feature and the second feature to realize sensing of an external environment; and the slave gateway is connected with the master gateway through the branch optical fiber. Therefore, according to the embodiment of the invention, the problem that the limitation exists in the aspects of detecting the state change of a fixed target and identifying the behavior of a moving target when a traditional single environment sensing technology is adopted in an FTTR (Fiber To The Room) system in the existing environment sensing technology can be solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of communication technologies, and more particularly, to a signal fusion perception system and method. Background Art

[0002] Falls are the leading cause of injury-related deaths among the elderly over 65 years old, accounting for 42% of the total number of injury-related deaths. Performing cardiopulmonary resuscitation (CPR), which is the rescue for cardiac arrest within 4 - 6 minutes after a fall, can significantly improve the survival rate. Therefore, quickly detecting abnormal situations and promptly calling for rescue have become the key to ensuring the safety of the elderly.

[0003] Currently, video surveillance technology is widely used in public places to monitor security conditions. However, the video image quality is easily affected by lighting and weather conditions, and its use in private spaces such as personal residences may raise privacy issues and thus cannot be widely applied. Although environmental perception technology avoids the above deficiencies, a single environmental perception technology often uses fiber optic sensing or wireless signal analysis, each having limitations. Fiber optic sensing technology can efficiently detect minute vibrations and displacements of fixed targets (such as walls and floors), but has relatively weak perception ability for moving targets (such as people or animals). On the contrary, wireless signal analysis technology can capture the behavioral characteristics of moving targets (such as human postures and movement trajectories), but has low sensitivity in detecting vibrations of fixed targets (such as walls and floors), and the signal quality is unstable in complex environments, resulting in a decrease in the accuracy of external environment perception.

[0004] Based on this, no effective solution has been proposed in the related art. Summary of the Invention

[0005] The embodiments of the present application provide a signal fusion perception system and method to at least solve the problem that existing environmental perception technologies have limitations in detecting state changes of fixed targets and behavior recognition of moving targets in a fiber to the room (FTTR) system.

[0006] According to an embodiment of the present application, a signal fusion perception system is provided, including: a main gateway and a slave gateway; the main gateway includes a first service module and a first perception module, the first service module is used to implement the operation of communication services; the first perception module includes a first optical fiber perception sub-module, a first wireless signal perception sub-module and a first feature fusion algorithm sub-module; wherein, the first optical fiber perception sub-module is used to collect a first backward scattering optical signal through a branched optical fiber and extract a first feature of the first backward scattering optical signal, the first wireless signal perception sub-module is used to collect a first wireless signal of the communication between the main gateway and the terminal and extract a second feature of the first wireless signal, and the first feature fusion algorithm sub-module is used to perform feature fusion analysis on the first feature and the second feature to realize the perception of the external environment; the slave gateway is connected to the main gateway through a branched optical fiber.

[0007] According to another embodiment of the present application, a signal fusion perception method is provided, which is applied to the main gateway and includes: collecting a backward scattering optical signal and a wireless signal; respectively extracting a first feature of the backward scattering optical signal and a second feature of the wireless signal; performing feature fusion analysis on the first feature and the second feature to realize the perception of the external environment.

[0008] According to another embodiment of the present application, a signal fusion perception method is provided, which is applied to the slave gateway and includes: collecting a second backward scattering optical signal and a second wireless signal of the communication between the slave gateway and the terminal; respectively extracting a third feature of the second backward scattering optical signal and a fourth feature of the second wireless signal; performing feature fusion analysis on the third feature and the fourth feature to realize the perception of the external environment.

[0009] According to still another embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored, and wherein the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0010] According to still another embodiment of the present application, an electronic device is further provided, including a memory and a processor, a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in the above method embodiments.

[0011] According to still another embodiment of the present application, a computer program product is further provided, including a computer program, and the computer program realizes the steps in the above method embodiments when executed by a processor.

[0012] Through the above embodiments of the present application, a signal fusion perception system is provided. The first optical fiber perception sub-module collects the first backscattered light signal through a branched optical fiber and extracts the first feature of the first backscattered light signal. The first wireless signal perception sub-module collects the first wireless signal of the communication between the main gateway and the terminal and extracts the second feature of the first wireless signal. The first feature fusion algorithm sub-module performs feature fusion analysis on the first feature and the second feature to achieve the perception of the external environment, that is, the main gateway collects the backscattered light signal and the wireless signal, extracts the first feature and the second feature from the backscattered light signal and the wireless signal respectively, and performs feature fusion analysis on the first feature and the second feature. Through the multi-modal feature fusion algorithm, the perception of the external environment can be realized, such as human fall, object tipping, human posture, etc., improving the accuracy of the perception of the external environment. Therefore, through the embodiments of the present application, the problem that the existing environmental perception technology has limitations in detecting the state change of fixed targets and the behavior recognition of moving targets in the fiber to the room (FTTR) system can be solved, and further the effect of reducing the false alarm rate and improving the accuracy and reliability of the perception of the external environment is achieved. Description of the Drawings

[0013] Figure 1 is the system architecture diagram of the FTTR system according to the embodiment of the present application;

[0014] Figure 2 is the system architecture diagram of the signal fusion perception system according to the embodiment of the present application;

[0015] Figure 3 is the flowchart of the signal fusion perception method applied to the main gateway according to the embodiment of the present application;

[0016] Figure 4 is the flowchart of the signal fusion perception method applied to the slave gateway according to the embodiment of the present application;

[0017] Figure 5 is the schematic diagram of the optical fiber environment perception according to the embodiment of the present application;

[0018] Figure 6 is the schematic diagram of the main gateway making a fusion perception decision according to the embodiment of the present application;

[0019] Figure 7 is the flowchart of the centralized multi-modal feature fusion algorithm according to the embodiment of the present application;

[0020] Figure 8 is the schematic diagram of the signal being preprocessed and then reported to the main gateway for fusion perception decision according to the embodiment of the present application;

[0021] Figure 9It is a schematic diagram of the fusion perception decision-making of the main gateway and the slave gateway after deploying the three-dimensional light perception device according to the embodiments of the present application;

[0022] Figure 10 It is a flowchart of the distributed multi-modal feature fusion algorithm according to the embodiments of the present application. Detailed implementation manners

[0023] In the following, the embodiments of the present application will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0024] It should be noted that the terms "first", "second", etc. in the specification, claims and above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence.

[0025] Currently, falls are the leading cause of injury-related deaths among the elderly over 65 years old, accounting for 42% of the total injury deaths. The golden rescue time for cardiac arrest after a fall is 4-6 minutes. Performing cardiopulmonary resuscitation within this time window can significantly improve the survival rate. For every 1-minute delay in rescue, the survival rate will decrease by 10%. Therefore, quickly detecting abnormalities and calling for rescue is the key to saving lives.

[0026] Although most public places have achieved video surveillance coverage, the video image effects are easily affected by factors such as light and climate and may cause privacy problems, which are not suitable for widespread application. Environmental perception technology can avoid the above deficiencies. However, single environmental perception technology often uses fiber optic sensing or wireless signal analysis. The former can detect the vibration of fixed targets, but has weak perception ability for moving targets (such as human behaviors). The latter can detect the behavioral characteristics of moving targets, but has low sensitivity to the vibration of fixed targets. How to achieve all-weather environmental perception effectively and with privacy protection without being affected by light and climate conditions, especially the accurate recognition and timely warning of abnormal events such as human falls and object displacements, is an urgent problem to be solved at present.

[0027] In view of the above problems, an intelligent environment perception and abnormal event warning method based on multimodal fusion of an FTTR (Fiber To The Room) system is proposed in an embodiment of the present application. The networking method of the master and slave devices of the FTTR system is more convenient for dynamically perceiving environmental changes, realizing abnormal event detection and warning. Specifically, an embodiment of the present application proposes a multimodal fusion perception method combining fiber optic sensing and wireless signal analysis, which uses optical fibers and Wifi nodes to collect optical signals and wireless signals in real time, respectively preprocess the optical signals and wireless signals such as signal filtering and signal denoising, extract features from the preprocessed optical signals and wireless signals respectively, fuse the features to obtain environmental information, and further analyze the environmental information using a feature fusion algorithm to complete the identification of abnormal events of moving targets and fixed targets, and send out a distress signal when matching dangerous situations such as human falls or trips and object vibrations. Therefore, the embodiment of the present application can monitor the state changes (such as vibrations and displacements) of fixed targets (such as walls and floors) and the behavior characteristics (such as human postures and movement trajectories) of moving targets (such as people or animals) in real time to realize the identification and warning of abnormal events.

[0028] Figure 1 is the system architecture diagram of the FTTR system according to an embodiment of the present application, as Figure 1 shown, the FTTR system includes an MFU (Master Fiber Unit, main gateway) and one or more SFUs (Secondary Fiber Unit, slave gateways). Figure 1 Taking two SFUs as an example, such as SFU1 and SFU2, the MFU is connected to SFU1 and SFU2 through optical fibers to jointly form an FTTR system, and the user terminal or cloud server is used to receive environmental perception information and warning information.

[0029] During the rate evolution of PON (Passive Opitcal Network), the FTTR system will also face challenges such as signal quality, network latency, and device compatibility. When the traditional PON network improves the rate, it may not be able to balance the stability and integrity of the signal, resulting in a decline in network performance. Based on this, the PON rate evolution and multi-mode coexistence method based on the FTTR system proposed in an embodiment of the present application includes three parts, namely fiber optic environment perception, wireless environment perception, and multimodal feature fusion, as follows:

[0030] (1) Fiber optic environment perception

[0031] When the synchronous optical pulse propagates along the optical fiber, the backward scattered optical signal of Rayleigh scattering will return to the incident end along the optical fiber, forming an echo optical signal. When the optical path is relatively stable without external disturbances, the phase and optical intensity of the backward scattered optical signal relative to the transmitted optical signal are fixed. When the optical fiber is subjected to external disturbances (such as vibration, displacement, etc.), the phase and optical intensity of the echo signal will change briefly. After the external disturbance is eliminated, it will return to the stable state. Based on the above principle, when the FTTR system operates in the 2.5G symmetric / asymmetric mode, the optical module at the receiving direction of the lower link port of the main gateway usually only receives the optical signal of 1310nm. When the FTTR system operates in the 10G symmetric / asymmetric mode, the optical module at the receiving direction of the lower link port of the main gateway usually only receives the optical signal of 1270nm. To analyze the backward scattered optical signal, a new channel is introduced in the receiving direction of the optical module at the lower link port of the main gateway / the optical module of the slave gateway, which is independent of the upstream service channel.

[0032] Among them, the new channel does not carry specific services, but only serves for the acquisition of the phase and optical intensity of the backward scattered optical signal. The collected phase and optical intensity parameters are analyzed by the main chip and the environmental perception result is output.

[0033] (2)Wireless Environment Perception

[0034] In wireless communication, a certain frequency band is generally divided into multiple subcarriers, and each subcarrier is responsible for transmitting a part of the data. By analyzing the phase and amplitude information on each subcarrier, the propagation path of the wireless signal in space can be accurately inferred. The specific applications are as follows:

[0035] The phase and amplitude differences on multiple subcarriers can show information such as the propagation path, reflection, scattering, and attenuation of the wireless signal. By analyzing the above information, it can be judged the path that the wireless signal passes through, whether it has experienced multipath propagation, etc., so as to have a more comprehensive understanding of the propagation of the wireless signal in space.

[0036] The current single Customer Premises Equipment (CPE) can collect and analyze Channel State Information (CSI) signals. However, in the face of complex scenarios such as wall blockages, the signal coverage of a single CPE device will inevitably have blind spots, and the subcarrier information in the blind spots cannot be effectively collected, so the link state cannot be judged. Based on this situation, in the FTTR system, CSI collection and analysis sub-modules are arranged for both the main gateway and each slave gateway. Each slave gateway independently collects CSI information (the embodiments of this application are not limited to CSI information, and can also collect millimeter-wave signals, XingShan raw signals, etc. The embodiments of this application take the collection of CSI information as an example), and reports the collection results to the main gateway through the Wireless Management in the Control Plane Interface (WMCI) channel (the embodiments of this application are not limited to the WMCI channel, and can also be an optical management message channel, an Ethernet channel, etc. The embodiments of this application take the WMCI channel as an example), and the main gateway makes a judgment by comprehensively analyzing the CSI information of each slave gateway.

[0037] (3) Multi-modal feature fusion algorithm

[0038] Since fiber optic environment perception can detect the slight vibrations and displacements of fixed targets (such as walls and floors), but has a weak ability to perceive moving targets (such as people or animals); on the contrary, wireless environment perception can capture the behavioral characteristics of moving targets (such as people or animals), but has a low sensitivity to the vibrations and displacements of fixed targets.

[0039] Therefore, in order to simultaneously detect slow events such as vibrations of fixed targets and relatively fast events such as moving targets, the embodiments of this application deploy fiber optic sensing modules at the main gateway and slave gateways respectively, collect information such as the optical path vibrations along the way of multiple sub-gateways through multiple branch optical fibers, and by optimizing the laying method of the branch optical fibers, the effect of full coverage of a specific building area can be achieved; wireless sensing modules are deployed at the main gateway and slave gateways respectively, CSI information is collected through the wireless sensing modules, the main gateway and slave gateways share the collected information or the slave gateways report the collected CSI information to the main gateway through the WMCI channel, and then the main gateway or slave gateway analyzes the collected CSI information or the shared CSI information in real time through the feature fusion algorithm, matches the features in the event library, and outputs abnormal events such as human falls, item tipping, and object vibrations, and uploads them to the user terminal or cloud server to send out warning information.

[0040] Figure 2 is the system architecture diagram of the signal fusion perception system according to the embodiments of this application, as Figure 2As shown, the signal fusion perception system includes: a main gateway and a slave gateway; the main gateway includes a first service module and a first perception module, and the first service module is used to implement the operation of communication services; the first perception module includes a first optical fiber perception sub-module, a first wireless signal perception sub-module, and a first feature fusion algorithm sub-module; wherein, the first optical fiber perception sub-module is used to collect a first backscattered optical signal through a branched optical fiber and extract a first feature of the first backscattered optical signal, the first wireless signal perception sub-module is used to collect a first wireless signal of the communication between the main gateway and the terminal and extract a second feature of the first wireless signal, and the first feature fusion algorithm sub-module is used to perform feature fusion analysis on the first feature and the second feature to realize the perception of the external environment; the slave gateway is connected to the main gateway through a branched optical fiber.

[0041] In this embodiment, the signal fusion perception system is composed of two major parts: a main gateway and a slave gateway, where the slave gateway includes one or more.

[0042] The main gateway includes a first service module and a first perception module. The main gateway integrates a module for conventional service operation (the first service module) and a module for environmental perception (the first perception module). Therefore, while processing communication services, the main gateway can also perform environmental monitoring and event detection.

[0043] The first service module included in the main gateway can ensure that the main gateway processes conventional communication services such as data transmission, voice calls, video streams, etc., so that the daily network usage of users will not be interfered when the signal fusion perception system performs intelligent environmental perception.

[0044] The first perception module included in the main gateway is further divided into a first optical fiber perception sub-module, a first wireless signal perception sub-module, and a first feature fusion algorithm sub-module.

[0045] First optical fiber perception sub-module: Through a branched optical fiber, it collects backscattered optical signals (such as the first backscattered optical signal), and extracts features related to the environment (the first feature) from them, such as changes in light intensity, frequency, phase, etc., for detecting events such as fixed target vibration and displacement.

[0046] First wireless signal perception sub-module: It is responsible for collecting wireless signals (such as the first wireless signal) between the main gateway and the terminal devices within its coverage area, and extracting features related to moving targets (the second feature) from the wireless signals, such as the power of CSI, scattering changes, etc., for capturing information such as the posture and movement trajectory of moving targets.

[0047] The first feature fusion algorithm sub-module: It fuses the different types of signal features extracted by the above-mentioned first optical fiber sensing sub-module and the first wireless signal sensing sub-module, and analyzes the fused first feature and second feature through an algorithm to achieve a comprehensive perception of the external environment and identify potential abnormal events.

[0048] If the slave gateway is connected to the master gateway through a branch optical fiber, then the slave gateway can also perform optical fiber environment perception, and at the same time can collect and process wireless signals, and then report the collected wireless signals to the master gateway through the WMCI channel, or the slave gateway reports to the master gateway after preliminary analysis, which can improve the response speed to abnormal events without infringing on user privacy.

[0049] In one embodiment, the slave gateway is configured with a service module for implementing the operation of communication services; or, the slave gateway is configured with a service module and a sensing module for implementing the operation of communication services and the perception of the environment.

[0050] In this embodiment, when the slave gateway is configured with a service module, the slave gateway includes a second service module for implementing the operation of communication services; when the slave gateway is configured with a service module and a sensing module, the slave gateway includes a second service module and a second sensing module; the second service module is used to implement the operation of communication services, and the second sensing module includes a second optical fiber sensing sub-module, a second wireless signal sensing sub-module and a second feature fusion algorithm sub-module; wherein, the second optical fiber sensing sub-module is used to collect a second backscattered light signal through a branch optical fiber and extract a third feature of the second backscattered light signal; the second wireless signal sensing sub-module is used to collect a second wireless signal of the communication between the slave gateway and the terminal and extract a fourth feature of the second wireless signal; the second feature fusion algorithm sub-module is used to perform feature fusion analysis on the third feature and the fourth feature to achieve the perception of the external environment; or, the slave gateway includes a second service module and a third sensing module; the second service module is used to implement the operation of communication services, and the third sensing module includes a third wireless signal sensing sub-module for collecting a third wireless signal of the communication between the slave gateway and the terminal.

[0051] In this embodiment, the slave gateway includes three implementation manners. Specifically, the first implementation manner: the slave gateway includes a second service module for implementing the operation of communication services;

[0052] The second implementation manner: the slave gateway includes a second service module and a second sensing module;

[0053] The second service module, which is used to implement the operation of communication services, ensures the integrity and stability of the signal fusion perception system, so that even if an interruption occurs in environmental perception or abnormal event detection, the communication service can still operate.

[0054] The second sensing module includes a second optical fiber sensing sub-module, a second wireless signal sensing sub-module, and a second feature fusion algorithm sub-module; the second optical fiber sensing sub-module: collects the backscattered light signal (the second backscattered light signal) through a branched optical fiber, and extracts features (the third feature) reflecting events such as the vibration and displacement of a fixed target for monitoring the state change of a fixed object; the second wireless signal sensing sub-module: collects the wireless signal (the second wireless signal) between the gateway and the terminal devices within its coverage area, and extracts features (the fourth feature) reflecting a moving target (such as a person or an animal) for capturing the movement trajectory and behavior posture of the moving target; the second feature fusion algorithm sub-module: fuses and analyzes the above-mentioned third feature and fourth feature, and further improves the accuracy and comprehensiveness of environmental perception by comprehensively analyzing the feature information of the optical fiber and the wireless signal to identify potential abnormal events.

[0055] The third implementation mode: the slave gateway includes a second service module and a third sensing module;

[0056] The second service module is used to implement the operation of the communication service, ensuring the integrity and stability of the signal fusion sensing system, so that even if an interruption occurs during environmental perception or abnormal event detection, the communication service can still operate.

[0057] The third sensing module includes a third wireless signal sensing sub-module. The slave gateway only has the ability to sense wireless signals and does not have an optical fiber sensing sub-module. The third wireless signal sensing sub-module in the third sensing module is responsible for collecting the wireless signal (the third wireless signal) between the gateway and the terminal device and sending the third wireless signal to the master gateway for monitoring the behavior of the moving target. This configuration can be applied to scenarios where optical fiber environmental perception is not required or not convenient, such as in an environment with good wireless signal coverage and inapplicable optical fiber sensing.

[0058] In one embodiment, the first wireless signal includes at least one of the following: the first channel state information CSI, the first millimeter wave signal, and the first satellite flash original signal; the second wireless signal includes at least one of the following: the second channel state information CSI, the second millimeter wave signal, and the second satellite flash original signal; the third wireless signal includes at least one of the following: the third channel state information CSI, the third millimeter wave signal, and the third satellite flash original signal.

[0059] In this embodiment, when the slave gateway includes a sensing module, the wireless signal sensing sub-modules of the master gateway and the slave gateway can each collect the wireless signal communicating with the terminal. The wireless signal includes but is not limited to CSI information, millimeter wave signals, and satellite flash original signals.

[0060] In one embodiment, when the slave gateway includes a second service module and a second sensing module, the slave gateway includes a three-dimensional optical sensing device for collecting a spatial image, analyzing the spatial image to obtain a three-dimensional optical sensing result, and sending the three-dimensional optical sensing result to the master gateway.

[0061] In this embodiment, a three-dimensional optical sensing device (3D optical sensing device, such as lidar) is deployed in the slave gateway. The slave gateway can collect CSI information of the terminal, millimeter-wave signals, star flash raw signals, and spatial images (also called laser images), and run a laser image recognition algorithm to output a three-dimensional optical sensing result, and send the three-dimensional optical sensing result to the master gateway.

[0062] In one embodiment, the terminal is used to receive the parsing result after feature fusion parsing, and send an alarm message according to the parsing result.

[0063] In this embodiment, if the master gateway or the slave gateway determines that there is an abnormal event based on the parsing result after feature fusion parsing, the master gateway or the slave gateway will report the parsing result to the user terminal or the cloud server in real time, and send an alarm message to notify the user in real time.

[0064] Through the above embodiments of the present application, a signal fusion sensing system is provided. The first optical fiber sensing sub-module collects the first backscattered optical signal through a branched optical fiber and extracts the first feature of the first backscattered optical signal. The first wireless signal sensing sub-module collects the first wireless signal of the communication between the master gateway and the terminal and extracts the second feature of the first wireless signal. The first feature fusion algorithm sub-module performs feature fusion parsing on the first feature and the second feature to realize the perception of the external environment, that is, the master gateway collects the backscattered optical signal and the wireless signal, extracts the first feature and the second feature from the backscattered optical signal and the wireless signal respectively, and performs feature fusion parsing on the first feature and the second feature. Through the multi-modal feature fusion algorithm, the perception of the external environment can be realized, such as human falls, object tipping, human postures, etc., improving the accuracy of the perception of the external environment. Therefore, through the embodiments of the present application, the problem that the existing environmental perception technology has limitations in detecting the state changes of fixed targets and the behavior recognition of moving targets in the fiber to the room (FTTR) system can be solved, and further the effect of reducing the false alarm rate and improving the accuracy and reliability of the perception of the external environment is achieved.

[0065] The embodiments of the present application also propose a signal fusion sensing method running on the above signal fusion sensing system. Figure 3 It is a flowchart of the signal fusion sensing method applied to the master gateway according to the embodiments of the present application. As Figure 3 shown, the method may specifically include the following steps:

[0066] Step S302: Collect the backscattered optical signal and the wireless signal;

[0067] In this embodiment, the slave gateway is configured with a service module; alternatively, the slave gateway is configured with a service module and a sensing module. In the case where the slave gateway is configured with a service module, collect the first backscattered optical signal and the first wireless signal for communication between the master gateway and the terminal; in the case where the slave gateway is configured with a service module and a sensing module, it includes: in the case where the slave gateway is configured with a second sensing module and collects the second backscattered optical signal and the second wireless signal, collect the first backscattered optical signal and the first wireless signal and receive the second backscattered optical signal and the second wireless signal sent by the slave gateway; or, in the case where the slave gateway is configured with a third sensing module and collects the third wireless signal, collect the first backscattered optical signal and the first wireless signal and receive the third wireless signal sent by the slave gateway.

[0068] In this embodiment, the master gateway can collect the first backscattered optical signal and the first wireless signal, or can also receive the second backscattered optical signal and the second wireless signal or the third wireless signal reported by the slave gateway.

[0069] In this step, the backscattered optical signal is the reflected optical signal returned when the external environment of the optical fiber changes after the master gateway emits a laser pulse to the slave gateway.

[0070] In this embodiment, the master gateway and the slave gateway are connected through a branched optical fiber, and the slave gateway is one or more.

[0071] In one embodiment, in the case where the slave gateway is configured with a second sensing module and collects the second backscattered optical signal and the second wireless signal, it further includes: in the case where a three-dimensional optical sensing device is deployed at the slave gateway, receive the three-dimensional optical sensing result after the slave gateway analyzes the collected spatial image.

[0072] In this embodiment, a three-dimensional optical sensing device (3D optical sensing device, such as lidar) is deployed in the slave gateway. The slave gateway can collect the CSI information of the terminal, millimeter-wave signals, star flash original signals, and spatial images (also called laser images), and run a laser image recognition algorithm to output a three-dimensional optical sensing result. The master gateway receives the three-dimensional optical sensing result sent by the slave gateway to run a feature fusion sensing algorithm to determine whether an abnormal event occurs.

[0073] In one embodiment, in the case where the slave gateway is configured with a second sensing module, after collecting the first backscattered optical signal and the first wireless signal, the method further includes: sending the first backscattered optical signal and the first wireless signal to the slave gateway.

[0074] In this embodiment, when a second sensing module is configured in the slave gateway, the slave gateway can run the feature fusion sensing algorithm. The master gateway shares the collected first backscattered optical signal and first wireless signal to the slave gateway through the WMCI channel, so that the slave gateway can perform feature fusion analysis.

[0075] Step S304: Extract the first feature of the backscattered optical signal and the second feature of the wireless signal respectively;

[0076] In one embodiment, before extracting the first feature of the backscattered optical signal and the second feature of the wireless signal respectively, the method further includes: preprocessing the backscattered optical signal and the wireless signal respectively, where the preprocessing includes at least one of the following: signal calibration, signal filtering, and signal denoising.

[0077] Step S306: Perform feature fusion analysis on the first feature and the second feature to realize the perception of the external environment.

[0078] In one embodiment, after performing feature fusion analysis on the first feature and the second feature, it further includes: sending the first analysis result after feature fusion analysis to the terminal, so that the terminal issues an alarm message according to the first analysis result, where the first analysis result includes at least one of the following: object dumping, object vibration, and human body falling.

[0079] In this embodiment, if the master gateway determines that there is an abnormal event based on the first analysis result after feature fusion analysis, the master gateway will report the first analysis result to the user terminal or the cloud server in real time, and issue an alarm message to notify the user in real time. For example, when a human body falls, the master gateway will report the human body falling to the user terminal or the cloud server in real time, and issue an alarm message to notify the user.

[0080] Through the above embodiments of the present application, a signal fusion sensing method is provided, which is applied to the master gateway. By collecting the backscattered optical signal and the wireless signal, extracting the first feature of the backscattered optical signal and the second feature of the wireless signal respectively, and performing feature fusion analysis on the first feature and the second feature, the perception of the external environment is realized, that is, through the multi-modal feature fusion algorithm, abnormal events such as human body falling and object dumping can be effectively identified and distinguished, the discovery cycle of abnormal events such as people falling can be effectively shortened, and the treatment success rate can be improved.

[0081] Figure 4 It is a flowchart of a signal fusion sensing method applied to a slave gateway according to an embodiment of the present application. As Figure 4 shown, the method may specifically include the following steps:

[0082] Step S402: Collect the second backscattered optical signal and the second wireless signal for communication between the slave gateway and the terminal;

[0083] In this embodiment, collecting the second backscattered optical signal and the second wireless signal includes: when the second sensing module is configured in the slave gateway, collecting the second backscattered optical signal and the second wireless signal.

[0084] In one embodiment, when the second sensing module is configured in the slave gateway, it further includes: when a three-dimensional optical sensing device is deployed in the slave gateway, collecting a spatial image and analyzing the spatial image to obtain a three-dimensional optical sensing result, and sending the three-dimensional optical sensing result to the master gateway.

[0085] In this embodiment, a three-dimensional optical sensing device (3D optical sensing device, such as a lidar) is deployed in the slave gateway. The slave gateway can collect the CSI information of the terminal, millimeter-wave signals, star flash raw signals, and spatial images (also called laser images), and run a laser image recognition algorithm to output a three-dimensional optical sensing result. The slave gateway sends the three-dimensional optical sensing result to the master gateway.

[0086] In one embodiment, after collecting the second backscattered optical signal and the second wireless signal, the method further includes: sending the second backscattered optical signal and the second wireless signal to the master gateway.

[0087] In this embodiment, when the second sensing module is configured in the slave gateway, the second sensing module of the slave gateway can collect the second backscattered optical signal and the second wireless signal, and share them to the master gateway through the WMCI channel for the master gateway to perform feature fusion analysis.

[0088] Step S404: Extract the third feature of the second backscattered optical signal and the fourth feature of the second wireless signal respectively;

[0089] In one embodiment, before extracting the third feature of the second backscattered optical signal and the fourth feature of the second wireless signal respectively, the method further includes: preprocessing the second backscattered optical signal and the second wireless signal respectively, where the preprocessing includes at least one of the following: signal calibration, signal filtering, and signal denoising.

[0090] Step S406: Perform feature fusion analysis on the third feature and the fourth feature to realize the perception of the external environment.

[0091] In one embodiment, after performing feature fusion analysis on the third feature and the fourth feature, the method further includes: sending the second analysis result after feature fusion analysis to a terminal, so that the terminal issues an alarm message according to the second analysis result, where the second analysis result includes at least one of the following: object toppling, object vibration, human body falling.

[0092] In this embodiment, if it is determined from the second analysis result after feature fusion analysis by the gateway that there is an abnormal event currently, the second analysis result is reported in real time from the gateway to the user terminal or the cloud server to issue an alarm message to notify the user in real time. For example, when a human body falls, the fact that the human body has fallen is reported in real time from the gateway to the user terminal or the cloud server to issue an alarm message to notify the user.

[0093] In one embodiment, the third wireless signal of the communication between the gateway and the terminal is collected from the gateway and sent to the main gateway.

[0094] In this embodiment, the gateway only has the ability to sense wireless signals and does not have an optical fiber sensing sub-module. The third wireless signal sensing sub-module in the third sensing module is responsible for collecting the wireless signal (the third wireless signal) between the gateway and the terminal device and sending the third wireless signal to the main gateway for monitoring the behavior of moving targets. This configuration is applicable to scenarios where optical fiber environment sensing is not required or not convenient, such as in an environment with good wireless signal coverage and where optical fiber sensing is not applicable.

[0095] Through the above embodiments of the present application, a signal fusion sensing method is provided, which is applied to the gateway. By collecting the second backscattered optical signal and the second wireless signal of the communication between the gateway and the terminal, the third feature of the second backscattered optical signal and the fourth feature of the second wireless signal are respectively extracted, and the third feature and the fourth feature are subjected to feature fusion analysis to realize the perception of the external environment. That is, through the multi-modal feature fusion algorithm, abnormal events such as human body falling and object toppling can be effectively identified and distinguished, the discovery cycle of abnormal events such as people falling can be effectively shortened, and the success rate of treatment can be improved.

[0096] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0097] Example 1

[0098] Figure 5 is a schematic diagram of optical fiber environment perception according to an embodiment of the present application. As Figure 5 shown, the main gateway MFU is connected to the slave gateways SFU1 and SFU2 through two branch optical fibers respectively. The perception module in the main gateway further includes an optical fiber perception sub-module, a wireless signal perception sub-module, and a feature fusion algorithm sub-module. In this Example 1, the case where the perception module includes an optical fiber perception sub-module is taken as an example for description.

[0099] (1) When the communication service is running:

[0100] Step S201: The service modules of the slave gateways SFU1 and SFU2 send upstream packets to the main gateway MFU;

[0101] Step S202: The service module of the main gateway MFU receives the upstream packets and conducts service communication.

[0102] (2) When an abnormal vibration occurs in a certain position of the branch optical fiber 1, the following process is executed:

[0103] Step S201: The service module of the slave gateway SFU1 sends an upstream packet to the main gateway MFU;

[0104] Step S202: The service module of the main gateway MFU receives the upstream packet and conducts service communication;

[0105] Step S203: The light intensity and phase of the Rayleigh scattering backscattered optical signal of the optical fiber at the ☆ position of the branch optical fiber 1 change and are transmitted back to the perception module of the main gateway MFU through the optical fiber.

[0106] Step S204: The perception module of the main gateway MFU analyzes the light intensity and phase of the Rayleigh scattering backscattered optical signal to judge the possible vibration points on the branch optical fiber 1.

[0107] (3) When an abnormal vibration occurs in a certain position of the branch optical fiber 2, the following process is executed:

[0108] Step S201: The service module of the slave gateway SFU2 sends an upstream packet to the main gateway MFU;

[0109] Step S202: The service module of the main gateway MFU receives the upstream packet and conducts service communication;

[0110] Step S205: The light intensity and phase of the Rayleigh scattering backscattered optical signal of the optical fiber at the △ position of the branch optical fiber 2 change and are transmitted back to the perception module of the main gateway MFU and the perception module of the slave gateway SFU2 through the optical fiber;

[0111] Step S206: The sensing modules of the main gateway MFU and the slave gateway SFU2 analyze the optical intensity and phase of the Rayleigh scattering backscattered optical signal to determine the possible vibration points on the optical fiber 2.

[0112] Embodiment 2

[0113] Figure 6 is a schematic diagram of the main gateway for fusion perception decision-making according to the embodiment of the present application. As Figure 6 shown, the slave gateways SFU1 and SFU2 collect and report CSI information, and the main gateway MFU makes a decision. The operating mechanism is as follows:

[0114] Steps S601 / S602 / S603: The main gateway MFU continuously collects CSI information by itself; the slave gateways SFU1 and SFU2 continuously collect CSI information and continuously report it to the main gateway MFU through the WMCI channel. When Figure 6 a person moves in the space, the collected CSI information will change;

[0115] Step S604: The main gateway MFU continuously receives the CSI information reported by the slave gateways SFU1 and SFU2 through the WMCI channel;

[0116] Step S605: The feature fusion algorithm deployed by the main gateway MFU can analyze the CSI information collected by itself and received in real time;

[0117] Step S606: The feature fusion algorithm deployed by the main gateway MFU analyzes that there is an abnormal event currently, and reports the analysis result to the user terminal APP (Application Software) or the cloud server in real time to send an alarm message to notify the user.

[0118] Embodiment 3

[0119] Figure 7 is a flowchart of the centralized multi-modal feature fusion algorithm according to the embodiment of the present application. As Figure 7 shown, in this centralized multi-modal feature fusion algorithm, the slave gateways SFU1 and SFU2 are only responsible for collecting CSI information, and the main gateway MFU performs feature fusion perception algorithm analysis, which specifically includes the following steps:

[0120] Step S701: The slave gateways SFU1 and SFU2 continuously collect CSI information and report it to the main gateway MFU, and the main gateway MFU collects the backscattered optical signals of the optical fibers of the slave gateways SFU1 and SFU2;

[0121] Step S702: The main gateway MFU continuously collects CSI information by itself;

[0122] Step S703: The feature fusion perception algorithm deployed by the main gateway MFU analyzes the CSI information and the backscattered optical signal;

[0123] Step S704: The main gateway MFU determines whether an abnormal event has occurred and the specific event type;

[0124] Step S705: If an abnormal event occurs, the main gateway MFU reports the abnormal event and the specific event type to the user terminal APP or the cloud server, and sends an alarm message to notify the user.

[0125] Embodiment 4

[0126] Figure 8 It is a schematic diagram of reporting to the main gateway for fusion perception decision after signal preprocessing according to the embodiment of the present application. As Figure 8 shown, the CSI information is collected from the slave gateways SFU1 and SFU2, and after preprocessing the CSI information, it is reported to the main gateway MFU. The decision-making schematic diagram of the main gateway MFU is as follows:

[0127] Step S801 / S802 / S803: The main gateway MFU continuously collects CSI information by itself; the slave gateways SFU1 and SFU2 continuously collect CSI information, and after preprocessing the CSI information, the slave gateways SFU1 and SFU2 report it to the main gateway MFU through the WMCI channel;

[0128] Step S804: The main gateway MFU continuously receives the preprocessed CSI information from the slave gateways SFU1 and SFU2 through the WMCI channel;

[0129] Step S805: The feature fusion algorithm deployed by the main gateway MFU can analyze the CSI information collected by itself and received after preprocessing in real time;

[0130] Step S806: The feature fusion algorithm deployed by the main gateway MFU analyzes that there is an abnormal event currently, and reports the analysis result to the user terminal APP (Application Software) or the cloud server in real time, and sends an alarm message to notify the user.

[0131] Embodiment 5

[0132] Figure 9 It is a schematic diagram of the main gateway and the slave gateway making fusion perception decisions after deploying the three-dimensional optical perception device according to the embodiment of the present application. As Figure 9 shown, in this embodiment, a slave gateway SFU is deployed, and a 3D optical perception device (lidar) is deployed in the slave gateway SFU. The main gateway MFU and the slave gateway SFU are respectively deployed with feature fusion perception algorithms. The operation mechanism is as follows:

[0133] Step S901 / S902 / S903: The main gateway MFU continuously collects CSI information by itself; the slave gateway SFU continuously collects CSI information and laser images, and runs a laser image recognition algorithm to output 3D light perception results;

[0134] Step S904: The main gateway MFU runs a feature fusion perception algorithm to determine whether an abnormal event occurs and the specific event type;

[0135] Step S905: The main gateway MFU shares the CSI information continuously collected by itself with the slave gateway SFU through the WMCI channel. The slave gateway SFU can also share the 3D light perception results and report the CSI information with the main gateway MFU through the WMCI channel. The slave gateway runs a feature fusion perception algorithm to determine whether an abnormal event occurs and the specific event type;

[0136] Step S906 / S907: If an abnormal event occurs, the main gateway MFU and the slave gateway SFU respectively report the abnormal event and the specific event type to the user terminal APP or the cloud server, and send an alarm message to notify the user.

[0137] Embodiment 6

[0138] Figure 10 is a flowchart of a distributed multi-modal feature fusion algorithm according to an embodiment of the present application. As Figure 10 shown, in this distributed multi-modal feature fusion algorithm, the slave gateway SFU and the main gateway MFU are respectively deployed with a feature fusion perception algorithm, and the feature fusion perception algorithm can be flexibly selected to be implemented by the slave gateway SFU or the main gateway MFU according to factors such as the load of the terminal device CPU (Central Processing Unit) and the fusion perception method supported by the terminal device. The operation mechanism is as follows:

[0139] Step S1001: The slave gateway SFU continuously collects CSI information;

[0140] Step S1002: The slave gateway SFU continuously emits laser beams and generates and collects corresponding laser images;

[0141] Step S1003: The slave gateway SFU continuously collects the backscattered light signals of the distributed optical fiber and reports the backscattered light signals to the main gateway MFU;

[0142] Step S1004: The slave gateway SFU reports the collected CSI information to the main gateway MFU;

[0143] Step S1005: The main gateway MFU sends the CSI information collected by itself to the slave gateway SFU;

[0144] Step S1006: Run a feature fusion algorithm from the gateway SFU to analyze CSI information, backscattered optical signals, and laser images;

[0145] Step S1007: Determine from the gateway SFU whether an abnormal event has occurred and the specific event type;

[0146] Step S1008: The main gateway MFU receives the backscattered optical signals and CSI information of the sub-fibers reported by the gateway SFU, as well as the CSI information collected by the main network optical itself, and runs a fusion algorithm to analyze the CSI information and the backscattered optical signals;

[0147] Step S1009: The main gateway MFU determines whether an abnormal event has occurred and the specific event type;

[0148] Step S1010: If an abnormal event occurs, the main gateway MFU and the gateway SFU respectively report the abnormal event and the specific event type to the user terminal APP or the cloud server, and send an alarm message to notify the user.

[0149] It should be noted that in the above embodiments, taking the collection of CSI information as an example, millimeter wave signals, star flash original signals, etc. can also be collected for signal fusion perception analysis; in the above embodiments, taking the WMCI channel as an example, it can also be through an optical management message channel, an Ethernet channel, etc. to share information.

[0150] In the embodiments of the present application, the multimodal fusion of optical signals and wireless signals makes the system not easily affected by external factors such as climate and ambient light, can maintain stable performance under various conditions, and uses the existing networking method of the FTTR master-slave gateway, without the need to deploy additional sensors, reducing the system deployment and maintenance costs, so that the environmental perception and abnormal event warning functions can be more widely applied.

[0151] The embodiments of the present application can be applied to the home or enterprise networking of the FTTR system to realize the monitoring and reporting of abnormal situations such as human falls.

[0152] It should be noted that the above-mentioned modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to this: the above-mentioned modules are all located in the same processor; or, the above-mentioned modules are respectively located in different processors in any combination form.

[0153] The embodiments of the present application also provide a computer-readable storage medium, in which a computer program is stored, and the computer program is set to execute the steps in any one of the above method embodiments when running.

[0154] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disks, magnetic disks, or optical discs.

[0155] An embodiment of the present application further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0156] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0157] According to another embodiment of the present disclosure, there is also provided a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the methods described in various embodiments of the present disclosure.

[0158] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be repeated here.

[0159] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.

[0160] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included in the protection scope of the present application.

Claims

1. A signal fusion perception system, characterized in that: include: Master gateway, slave gateway; The main gateway includes a first business module and a first perception module, wherein the first business module is used to implement the operation of the communication business; the first perception module includes a first optical fiber perception submodule, a first wireless signal perception submodule and a first feature fusion algorithm submodule; wherein the first optical fiber perception submodule is used to collect the first backscattered light signal through the optical fiber and extract the first feature of the first backscattered light signal, the first wireless signal perception submodule is used to collect the first wireless signal communicated between the main gateway and the terminal and extract the second feature of the first wireless signal, and the first feature fusion algorithm submodule is used to perform feature fusion analysis on the first feature and the second feature to realize the perception of the external environment; The slave gateway is connected to the master gateway via a branch optical fiber.

2. The system according to claim 1, characterized in that in, The slave gateway is configured with a service module for implementing the operation of the communication service; or, The slave gateway is configured with a service module and a perception module for implementing the operation of communication services and the perception of the environment.

3. The system according to claim 2, characterized in that In the case where the slave gateway is configured with a service module, the slave gateway includes a second service module for implementing the operation of the communication service; In the case where the slave gateway is configured with a service module and a perception module, the slave gateway includes a second service module and a second perception module; the second service module is used to implement the operation of the communication service, and the second perception module includes a second optical fiber perception submodule, a second wireless signal perception submodule and a second feature fusion algorithm submodule; wherein the second optical fiber perception submodule is used to collect the second backscattered light signal through the optical fiber and extract the third feature of the second backscattered light signal; the second wireless signal perception submodule is used to collect the second wireless signal communicated between the slave gateway and the terminal and extract the fourth feature of the second wireless signal; the second feature fusion algorithm submodule is used to perform feature fusion analysis on the third feature and the fourth feature to realize the perception of the external environment; or, The slave gateway includes a second service module and a third perception module; the second service module is used to implement the operation of the communication service, and the third perception module includes a third wireless signal perception submodule, which is used to collect the third wireless signal communicated between the slave gateway and the terminal.

4. The system according to any one of claims 1 to 3, characterized in that: in, The slave gateway includes one or more.

5. The system according to claim 3, characterized in that in, The first wireless signal includes at least one of the following: first channel state information CSI, a first millimeter wave signal, and a first star flash original signal; the second wireless signal includes at least one of the following: second channel state information CSI, a second millimeter wave signal, and a second star flash original signal; the third wireless signal includes at least one of the following: third channel state information CSI, a third millimeter wave signal, and a third star flash original signal.

6. The system according to claim 3, characterized in that in, In the case where the slave gateway includes a second business module and a second perception module, the slave gateway includes a three-dimensional light perception device for collecting spatial images and analyzing the spatial images to obtain three-dimensional light perception results, and sending the three-dimensional light perception results to the master gateway.

7. The system according to claim 1, characterized in that in, The terminal is used to receive the analysis result after feature fusion analysis, and issue an alarm message according to the analysis result.

8. A signal fusion perception method, characterized in that: Applied to the main gateway, including: Collect backscattered light signals and wireless signals; extracting a first feature of the backscattered light signal and a second feature of the wireless signal respectively; The first feature and the second feature are subjected to feature fusion analysis to achieve perception of the external environment.

9. The method according to claim 8, characterized in that in, The backscattered light signal is a reflected light signal returned when the external environment of the optical fiber changes after the master gateway transmits a laser pulse to the slave gateway.

10. The method according to claim 9, characterized in that in, The master gateway is connected to the slave gateway via a branch optical fiber, and there are one or more slave gateways.

11. The method according to claim 10, characterized in that in, The slave gateway is configured with a service module; or, the slave gateway is configured with a service module and a perception module.

12. The method according to claim 11, characterized in that The collecting of backscattered light signals and wireless signals comprises: In the case where the slave gateway is configured with a service module, collecting a first backscattered light signal and a first wireless signal communicated between the master gateway and the terminal; In the case where the slave gateway is configured with a service module and a perception module, it includes: In the case where the slave gateway is configured with a second sensing module and collects a second backscattered light signal and a second wireless signal, collecting the first backscattered light signal and the first wireless signal and receiving the second backscattered light signal and the second wireless signal sent by the slave gateway; or, In the case where the slave gateway is configured with a third sensing module and collects a third wireless signal, the first backscattered light signal and the first wireless signal are collected and the third wireless signal sent by the slave gateway is received.

13. The method according to claim 12, characterized in that in, In the case where the slave gateway is configured with a second sensing module and collects a second backscattered light signal and a second wireless signal, the method further includes: In the case where the slave gateway is deployed with a three-dimensional light sensing device, the slave gateway receives a three-dimensional light sensing result after parsing the acquired spatial image.

14. The method according to claim 12, characterized in that In the case where the slave gateway is configured with a second sensing module, after collecting the first backscattered light signal and the first wireless signal, the method further includes: The first backscattered light signal and the first wireless signal are sent to the slave gateway.

15. The method according to claim 8, characterized in that After performing feature fusion analysis on the first feature and the second feature, the method further includes: The first analysis result after feature fusion analysis is sent to the terminal, so that the terminal issues an alarm message according to the first analysis result, wherein the first analysis result includes at least one of the following: object tipping, object vibration, and human body falling.

16. A signal fusion perception method, characterized in that: Applied to slave gateways, including: Collecting a second backscattered light signal and a second wireless signal communicated between the gateway and the terminal; extracting a third feature of the second backscattered light signal and a fourth feature of the second wireless signal respectively; The third feature and the fourth feature are subjected to feature fusion analysis to achieve perception of the external environment.

17. The method according to claim 16, characterized in that After collecting the second backscattered light signal and the second wireless signal communicated between the gateway and the terminal, the method further includes: The second backscattered light signal and the second wireless signal are sent to a main gateway.

18. The method according to claim 16, characterized in that The collecting of the second backscattered light signal and the second wireless signal communicated between the gateway and the terminal includes: In the case where the slave gateway is configured with a second sensing module, a second backscattered light signal and a second wireless signal communicated between the slave gateway and the terminal are collected.

19. The method according to claim 18, characterized in that In the case where the slave gateway is configured with a second perception module, the second sensing module further includes: In the case where the slave gateway is deployed with a three-dimensional light sensing device, a spatial image is collected and analyzed to obtain a three-dimensional light sensing result, and the three-dimensional light sensing result is sent to the master gateway.

20. The method according to claim 16, characterized in that After performing feature fusion analysis on the third feature and the fourth feature, the method further includes: The second analysis result after feature fusion analysis is sent to the terminal, so that the terminal issues an alarm message according to the second analysis result, wherein the second analysis result includes at least one of the following: object tipping, object vibration, and human body falling.

21. The method according to claim 16, characterized in that include: A third wireless signal communicated between the slave gateway and the terminal is collected, and the third wireless signal is sent to the master gateway.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 8 to 15 or 16 to 21.

23. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 8 to 15 or 16 to 21 are implemented.

24. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 8 to 15 or 16 to 21 are implemented.

Images