Old people falling monitoring and multi-terminal linkage emergency response system

Abstract

The invention discloses an old people fall monitoring and multi-terminal linkage emergency response system, and relates to the technical field of medical treatment, and the system comprises the steps: carrying out the non-inductive collection of human body dynamic information in a weak light visual blind area environment; the voice interaction verification judgment module comprises a far-field voice excitation unit; and setting a priority queue management mechanism, and confirming a behavior perception output tumble instruction. According to the invention, closed-loop management of fall monitoring and emergency treatment of the old people is realized, and the system has remarkable beneficial effects; the privacy of the user is effectively protected, and the false alarm rate and the invalid scheduling of nursing resources are obviously reduced; the event discrimination accuracy and the individual suitability are improved, and it is ensured that high-risk events are not missed; the timeliness and accuracy of emergency response are improved, and safe, intelligent and humanized technical support is provided for intelligent old-age care and home health monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical treatment, in particular to an old people fall monitoring and multi-terminal linkage emergency response system. BACKGROUND

[0002] At present, with the continuous deepening of population aging in China, the number of the elderly population is growing rapidly, and the safety problem in home-based care and institutional care is increasingly prominent. As one of the most common accidental injuries among the elderly, falls not only can cause fractures, brain damage and other serious consequences, but also can endanger life if not discovered and rescued in time.

[0003] At present, most of the old people fall monitoring systems remain in the passive mode of one-way detection and alarm, lack effective intelligent interaction mechanism, and once the system triggers an alarm, it cannot actively inquire through voice whether the fallen old people need help, can they get up by themselves, etc. Therefore, it is difficult to distinguish between real falls, false alarm events or user states, resulting in that nursing staff can only blindly respond to the alarm signal, and the rescue opportunity of the real emergency situation may be delayed due to the lack of communication. At the same time, due to the lack of voice interaction module, the system cannot establish a real-time communication channel between the old people and their families, medical staff in emergency situations, making it difficult to achieve multi-party collaborative disposal, and seriously weakening the timeliness and accuracy of emergency response, restricting the development of the intelligent elderly care system in the direction of humanization and closed loop. SUMMARY

[0004] In view of the problems existing in the prior art of old people fall monitoring and multi-terminal linkage emergency response system, the present application is proposed.

[0005] Therefore, the problem to be solved by the present application is how to realize accurate identification and state discrimination of fall events under the premise of protecting the privacy of the elderly, and to build a multi-terminal collaborative closed-loop emergency response mechanism through intelligent voice interaction to verify the authenticity of the event, in order to overcome the problems of high false alarm rate, lack of interaction ability and response lag of the existing system.

[0006] To solve the above technical problems, the present application provides the following technical solutions: In a first aspect, the present application provides an old people fall monitoring and multi-terminal linkage emergency response system, which comprises a penetrating behavior perception module for using an electromagnetic wave radar sensing array to non-invasively collect human body dynamic information in a weak light visual blind area environment; A voice interaction verification and decision module is connected to the event trigger interface of the penetrating behavior perception module, and the voice interaction verification and decision module comprises a far-field voice excitation unit. A heterogeneous terminal multi-protocol linkage scheduling module is used to set a priority queue management mechanism to confirm the behavior perception output fall instruction.

[0007] As a preferred scheme of the old people fall monitoring and multi-terminal linkage emergency response system, the penetrating behavior perception module comprises: The radar signal acquisition unit is configured in the frequency band of the monitoring area, transmits electromagnetic waves in a modulation mode, receives the echo reflected by the target body in the frequency band of the monitoring area, and outputs the original intermediate frequency signal. The preprocessing suppression unit is used to obtain the human cloud data stream of the original intermediate frequency signal, combine the static modeling and dynamic threshold segmentation algorithm, and extract the human dynamic cloud data stream or human static cloud data stream. The behavior sequence analysis output unit is used to construct a spatial motion point set according to the dynamic cloud data stream. The posture feature reconstruction unit is used to construct a spatial motion point set based on the dynamic cloud data stream, identify the action mode of the human fall process, determine whether to trigger a suspected fall event in combination with the electromagnetic wave radar sensing array, generate a behavior perception output fall instruction with a timestamp, and input the behavior perception output fall instruction into the voice interaction verification decision module.

[0008] As a preferred scheme of the old people fall monitoring and multi-terminal linkage emergency response system, the voice interaction verification decision module further comprises verifying the behavior perception output fall instruction with a timestamp through the voice interaction verification decision module, and the verification through the voice interaction verification decision module comprises an interactive verification process with context perception, calling a far-field voice excitation unit to generate an inquiry sentence, and processing the inquiry sentence through the electromagnetic wave radar sensing array. The processing of the inquiry sentence comprises judging whether the content of the inquiry sentence belongs to the self-recovery, help needed, pain unable to move, or no response state category, and calculating a comprehensive confidence score of the state category through a Bayesian fusion algorithm. When the comprehensive confidence score is higher than a first threshold , and the content of the inquiry sentence is determined as a help signal, it is marked as a first-level emergency event. When the comprehensive confidence score is between a first threshold and a second threshold , there is a fuzzy or partial response situation of the inquiry sentence, and it is marked as a second-level to-be-confirmed event. When the comprehensive confidence score is lower than the second threshold , and there is no abnormal expression in the inquiry sentence, it is determined as a false alarm confirmation event, and the process is terminated.

[0009] As a preferred scheme of the old people fall monitoring and multi-terminal linkage emergency response system, the calculation formula of the comprehensive confidence score is: wherein, represents the probability that the voice semantic recognition outputs the corresponding help category under the premise that the event is a real emergency, represents the posture feature under the premise that the event is a real emergency, represents the prior probability of the event, represents the marginal probability, represents the comprehensive confidence score.

[0010] As a preferred scheme of the old people fall monitoring and multi-terminal linkage emergency response system, the first threshold value and the second threshold value The decision threshold values of the first threshold value and the second threshold value are solved by a numerical optimization method, and the specific calculation formula is: wherein, represents the decision threshold value, represents the candidate threshold value, represents the false alarm cost coefficient, represents the negative class cumulative distribution function value, represents the positive class cumulative distribution function value.

[0011] As a preferred scheme of the old people fall monitoring and multi-terminal linkage emergency response system, the first threshold value and the second threshold value include calculating by using the decision threshold value , and the specific calculation formula is: wherein, represents the correction gain coefficient, represents the secondary threshold scaling factor, represents the decision threshold value, represents the first threshold value, represents the second threshold value; represents the threshold correction amount.

[0012] As a preferred scheme of the old people fall monitoring and multi-terminal linkage emergency response system, the heterogeneous terminal multi-protocol linkage scheduling module further includes an event level label output according to the audio interaction verification decision, and a priority queue management mechanism is set; For a first-level emergency event, a safety alarm message containing a behavior perception output fall instruction with a timestamp, a spatial motion point set and a posture feature reconstruction is packaged, a priority transmission channel is allocated by a scheduling strategy, the message is pushed to a nursing station system terminal through a link, is transmitted to a family mobile terminal and a community smart elderly care cloud platform through network encryption, and an interface linkage of an emergency system is triggered. For a second-level to-be-confirmed event, a delay observation mechanism is started, posture changes and voice inputs are continuously monitored within a set period, if no deterioration trend appears, the event is automatically downgraded and filed as a false alarm confirmation event, otherwise, the event is upgraded to a first-level emergency event and is determined as a help signal.

[0013] In a second aspect, an embodiment of the present application provides an elderly fall monitoring and multi-terminal linkage emergency response method, which comprises: using an electromagnetic wave radar sensing array to non-invasively collect human body dynamic information in a weak light visual blind area environment; An event triggering interface connected to the penetrating behavior perception module, and the voice interaction verification decision module comprises a far-field voice excitation unit. A priority queue management mechanism is set to confirm the behavior perception output fall instruction. A frequency band configured in a monitoring area is used to emit electromagnetic waves and receive reflected echoes of the target body in the frequency band of the monitoring area, and output original intermediate frequency signals. Human cloud data streams of the original intermediate frequency signals are obtained, and static modeling and dynamic threshold segmentation algorithms are combined to extract human dynamic cloud data streams or human static cloud data streams. A spatial motion point set is constructed according to the dynamic cloud data streams. The spatial motion point set is constructed based on the dynamic cloud data streams, the action mode of the human body fall process is identified, whether a suspected fall event is triggered is determined in combination with the electromagnetic wave radar sensing array, a behavior perception output fall instruction with a timestamp is generated, and input to the voice interaction verification decision module.

[0014] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory and a processor, and the memory stores a computer program, wherein: the processor implements any step of the above-mentioned elderly fall monitoring and multi-terminal linkage emergency response system when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, wherein: the computer program is executed by a processor to implement any step of the above-mentioned elderly fall monitoring and multi-terminal linkage emergency response system.

[0016] The application has the beneficial effects that: the application realizes the closed-loop management of the fall monitoring and emergency disposal of the elderly by constructing the intelligent response architecture integrated with perception, interaction, decision-making and linkage, and has significant beneficial effects; first, the high-penetration non-visual perception technology such as millimeter wave radar is adopted, which can realize non-susceptible monitoring in private or weak light environments such as bathrooms and bedrooms, effectively protect user privacy, and overcome the ethical obstacles of the camera scheme and the poor compliance of wearable devices; second, after detecting a suspected fall, the system actively triggers voice interaction verification, judges the real state of the elderly by combining semantic understanding and emotion recognition technology, and significantly reduces the false positive rate and invalid scheduling of nursing resources; third, through multi-modal information fusion and adaptive threshold decision mechanism, the accuracy and individual adaptability of event discrimination are improved, and high-risk events are not missed; finally, the system supports multi-protocol linkage response based on event level, pushes the alarm information to the nursing station, family terminal and emergency platform in real time, forms a cross-terminal and cross-system collaborative disposal link, greatly improves the timeliness and accuracy of emergency response, and provides safe, intelligent and humanized technical support for smart elderly care and home health monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor. Among them: Figure 1 A flowchart of an elderly fall monitoring and multi-terminal linkage emergency response system provided by the embodiment of the application.

[0018] Figure 2 A method schematic diagram of an elderly fall monitoring and multi-terminal linkage emergency response system provided by the embodiment of the application.

[0019] Figure 3 A structure schematic diagram of a medium of an elderly fall monitoring and multi-terminal linkage emergency response system provided by the embodiment of the application.

[0020] Figure 4 A structure schematic diagram of a computing device of a method of an elderly fall monitoring and multi-terminal linkage emergency response system provided by the embodiment of the application. DETAILED DESCRIPTION

[0021] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present application.

[0022] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application, however, can be practiced in a variety of ways beyond the specific details set forth herein without departing from the scope of the present application. It can be appreciated by those skilled in the art that the present application can be practiced without such specific details.

[0023] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. The "in one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments.

[0024] The present application is described in detail in conjunction with the schematic diagram. In the detailed description of the embodiments of the present application, the cross-sectional view of the device structure is locally enlarged without the general proportion for the convenience of description, and the schematic diagram is only an example, which should not limit the scope of protection of the present application herein. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in actual manufacture.

[0025] Meanwhile, in the description of the present application, it should be noted that the terms "upper, lower, inner and outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first, second or third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0026] Unless otherwise specifically defined and limited in the present application, the terms "mounting, connecting, connection" should be understood broadly, for example: it can be fixed connection, detachable connection or integral connection; it can also be mechanical connection, electrical connection or direct connection, it can also be indirectly connected through intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0027] Example 1 Reference Figures 1-4For the first embodiment of the application, the embodiment provides an old people fall monitoring and multi-terminal linkage emergency response system, comprising: S1: a penetrating behavior perception module, configured to use an electromagnetic wave radar sensing array to perform non-invasive collection of human body dynamic information in a weak light visual blind area environment.

[0028] The penetrating behavior perception module comprises: A radar signal collection unit, configured in a frequency band of a monitoring area, uses a modulation method to emit electromagnetic waves and receive target body reflection echoes in the frequency band of the monitoring area, and outputs original intermediate frequency signals. A preprocessing suppression unit, configured to obtain human body cloud data streams of the original intermediate frequency signals, and extract human body dynamic cloud data streams or human body static cloud data streams by combining static modeling and dynamic threshold segmentation algorithms. A behavior sequence analysis output unit, configured to construct a spatial motion point set according to the dynamic cloud data streams. A posture feature reconstruction unit, configured to construct a spatial motion point set based on the dynamic cloud data streams, identify the action mode of the human body falling process, determine whether to trigger a suspected falling event in combination with the electromagnetic wave radar sensing array, generate a behavior perception output falling instruction with a timestamp, and input the behavior perception output falling instruction to a voice interaction verification decision module.

[0029] S2: a voice interaction verification decision module, configured to be connected to an event trigger interface of the penetrating behavior perception module, and the voice interaction verification decision module comprises a far-field voice excitation unit.

[0030] The voice interaction verification decision module further comprises verifying the behavior perception output falling instruction with a timestamp through the voice interaction verification decision module, and the verification through the voice interaction verification decision module comprises an interactive verification process with context perception, calling the far-field voice excitation unit to generate an inquiry sentence, and processing the inquiry sentence through the electromagnetic wave radar sensing array. Processing the inquiry sentence comprises judging whether the content of the inquiry sentence belongs to a self-raise, help needed, pain unable to move, or no response state category, and calculating a comprehensive confidence score of the state category by using a Bayesian fusion algorithm. When the comprehensive confidence score is higher than a first threshold , and the content of the inquiry sentence is determined as a help signal, it is marked as a first-level emergency event. When the comprehensive confidence score is between the first threshold and a second threshold , and there is an inquiry sentence ambiguity or partial response, it is marked as a second-level to-be-confirmed event. When the comprehensive confidence score is lower than the second threshold , and there is no abnormal expression in the inquiry sentence, it is determined as a false alarm confirmation event, and the process is terminated.

[0031] Further, processing the inquiry sentence includes judging the content of the inquiry sentence belongs to the state category of autonomous getting up, needing help, pain unable to move or no response, etc. The comprehensive confidence score of the state category is calculated by the Bayesian fusion algorithm. When the comprehensive confidence score is higher than the first threshold value 0.85, and the voice recognition result contains the keywords such as save me, can't get up, pain, etc., the tone fluctuation amplitude is more than ± 15dB, and the response delay is less than 8 seconds, it is judged as an explicit help signal, and marked as a first-level emergency event. When the comprehensive confidence score is between 0.60 and 0.85, or the voice recognition confidence is lower than 0.70, the semantic is fuzzy (such as unstructured response such as hmm, ah, etc.), the voiceprint is incomplete, and the response delay is more than 12 seconds, it is marked as a second-level event to be confirmed. When the comprehensive confidence score is lower than 0.60, and the voice content recognition is not a help expression such as nothing, can get up, false touch, etc., combined with the continuous getting up action (center of gravity rising speed ≥ 0.3m / s, attitude angle change rate ≥ 45° / s) perceived by the radar, it is judged as a false alarm confirmation event, and the processing flow is terminated.

[0032] S2.1: The calculation formula of the comprehensive confidence score is: Among them, represents the probability of the voice semantic recognition output corresponding to the help category under the premise that the event is a real emergency situation, represents the posture feature under the premise that the event is a real emergency situation, represents the prior probability of the event, represents the marginal probability, represents the comprehensive confidence score.

[0033] S2.2: The first threshold value and the second threshold value The decision threshold values of the first threshold value and the second threshold value are solved by a numerical optimization method, and the specific calculation formula is: Among them, represents the decision threshold value, represents the candidate threshold value, represents the false alarm cost coefficient, represents the negative class cumulative distribution function value, represents the positive class cumulative distribution function value.

[0034] S2.3: The first threshold value and the second threshold value include calculating by using the decision threshold value , and the specific calculation formula is: wherein, denotes a correction gain coefficient, denotes a secondary threshold scaling factor, denotes a decision threshold, denotes a first threshold, denotes a second threshold; denotes a threshold correction amount.

[0035] S3: a heterogeneous terminal multi-protocol linkage scheduling module, configured to set a priority queue management mechanism and confirm the behavior perception output fall instruction.

[0036] The heterogeneous terminal multi-protocol linkage scheduling module further comprises an event level label output according to an audio interaction verification decision, and a priority queue management mechanism is set; For a first-level emergency event, a safety alarm message containing the behavior perception output fall instruction, a set of spatial motion points and a posture feature reconstruction with a time stamp is encapsulated, a priority transmission channel is allocated through a scheduling strategy, the message is pushed to a nursing station system terminal through a link, and is transmitted to a family mobile terminal and a community smart elderly care cloud platform through network encryption, so that an interface linkage of an emergency system is triggered; For a second-level to-be-confirmed event, a delay observation mechanism is started, posture changes and voice inputs are continuously monitored within a set period, if no deterioration trend appears, the event is automatically downgraded and filed as a false alarm confirmation event, otherwise, the event is upgraded to a first-level emergency event and is determined as a help signal.

[0037] In one preferred embodiment, an elderly fall monitoring and multi-terminal linkage emergency response method, the system comprises using an electromagnetic wave radar sensing array to non-invasively collect human body dynamic information in a weak light visual blind area environment; The event trigger interface is connected to the penetrating behavior perception module, and the voice interaction verification decision module comprises a far-field voice excitation unit; A priority queue management mechanism is set to confirm the behavior perception output fall instruction; The frequency band configured in the monitoring area adopts a modulation mode to emit electromagnetic waves and receive the reflected echoes of the target body in the frequency band of the monitoring area, and outputs the original intermediate frequency signal; The human body cloud data stream of the original intermediate frequency signal is acquired, static modeling and dynamic threshold segmentation algorithms are combined, and human body dynamic cloud data stream or human body static cloud data stream is extracted; The set of spatial motion points is constructed according to the dynamic cloud data stream; Based on dynamic cloud data flow, a spatial motion point set is constructed, an action mode of a human body falling process is identified, whether a suspected falling event is triggered is determined by combining an electromagnetic wave radar sensing array, a behavior perception output falling instruction with a time stamp is generated, and the falling instruction is input into a voice interaction verification judgment module.

[0038] The above-mentioned unit modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in the computer device in software form, so as to call and execute the operations corresponding to the above-mentioned modules by the processor.

[0039] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved by WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or can be a key, trackball or touchpad arranged on the shell of the computer device, or can be an external keyboard, touchpad or mouse, etc.

[0040] In summary, the present application realizes the closed-loop management of the fall monitoring and emergency disposal of the elderly by constructing the intelligent response architecture integrating perception, interaction, decision and linkage, and has significant beneficial effects. First, the high-penetration non-visual perception technology such as millimeter wave radar is used, which can realize non-invasive monitoring in private or weak light environments such as bathrooms and bedrooms, effectively protect user privacy, and overcome the ethical obstacles of the camera solution and the poor compliance of wearable devices. Second, the system actively triggers voice interaction verification after detecting a suspected fall, judges the real state of the elderly by combining semantic understanding and emotion recognition technology, and significantly reduces the false positive rate and invalid scheduling of nursing resources. Third, through multi-modal information fusion and adaptive threshold decision mechanism, the accuracy and individual adaptability of event discrimination are improved, and high-risk events are not missed. Finally, the system supports multi-protocol linkage response based on event level, pushes alarm information to nursing stations, family terminals and emergency platforms in real time, forms a cross-terminal and cross-system collaborative disposal link, greatly improves the timeliness and accuracy of emergency response, and provides safe, intelligent and humanized technical support for smart elderly care and home health monitoring.

[0041] Embodiment 2 Reference Figures 1-4For the second embodiment of the application, the embodiment provides an elderly fall monitoring and multi-terminal linkage emergency response system. In order to verify the beneficial effects of the application, scientific demonstration is carried out through simulation experiment.

[0042] The traditional fall monitoring system relies on wearable devices and lacks interactive ability, resulting in a wearing rate of less than 60% for the elderly, a false positive rate of up to 35%, an average response time of more than 6 minutes for nursing staff, and an inability to determine the authenticity of the event. After introducing the elderly fall monitoring and multi-terminal linkage emergency response system of the application, the 77GHz millimeter wave radar deployed in 20 key areas such as bathrooms and bedrooms realizes non-invasive monitoring, penetrates clothing and thin walls, covers weak light and visual blind areas, and has a privacy protection rate of 100%. After monitoring the abnormal posture, the system automatically triggers voice inquiry within 3 seconds: Do you need help, and combines voiceprint recognition and semantic analysis to determine the response of the elderly.

[0043] In an actual event, the system detected that the center of gravity acceleration of Mr. Zhang from standing to falling was 4.2m / s 2 , the attitude inclination angle changed by 78°, and the initial judgment was suspected fall. The voice interaction was initiated immediately, the old man replied that he could not get up or his knees hurt, the voice semantic confidence was 0.82, the tone fluctuation was +18dB, the system fused the radar motion characteristics and voice evidence, and the comprehensive confidence score was 0.91, which exceeded the first level threshold 0.85, and was immediately marked as a first level emergency event. The alarm information was synchronized and pushed to the nursing station HIS system, family mobile phone APP and community first aid platform through Wi-Fi 6 and NB-IoT dual link within 1.2 seconds, and the nursing staff arrived at the scene to implement rescue within 2 minutes and 18 seconds, which was 63% shorter than the original system response time. Three months of operation data show that the overall false positive rate of the system is reduced to 8.7%, the true event recognition accuracy is 96.4%, and the voice interaction verification effectively reduces the invalid police by 42%, significantly improving the care efficiency and the safety of the elderly.

[0044] The comparison between the application and the prior art is shown in the following table 1: Table 1 Comparison table of the application and the prior art Comparison item Prior art Technical solutions of the present application Perception mode Multi-reliance camera or wearable device Adopting non-contact perception of millimeter wave radar, strong penetration, suitable for private spaces such as bathroom and bedroom Privacy protection Camera is easy to leak privacy, and users strongly resist No need for visual collection, full-process non-invasive monitoring, privacy protection rate reaches 100% Environmental adaptability Camera is greatly affected by light, and fails in dark light Not affected by light and obstruction, can work stably in weak light and complex environment Fall recognition accuracy Single sensor false alarm rate is high, generally > 30% Fusion posture evolution analysis, recognition accuracy is above 96%, false alarm rate is reduced to 8.7% Interaction capability Only alarm prompt, no voice interaction Support intelligent voice inquiry and response, can judge the state of "whether can get up" and "whether in pain" Event discrimination mechanism One-way triggering, unable to distinguish real falls from false alarms Through voice + posture multi-modal fusion scoring, automatically classified as first-level / second-level / false alarm event Emergency response efficiency Nursing staff blindly goes out, average response time > 6 minutes Multi-end linkage real-time push, average response time is shortened to 2 minutes and 18 seconds System intelligence level Passive alarm, lack of closed-loop management Realize "perception-interaction-decision-linkage" full-link closed loop, support personalized threshold adjustment Table 1 shows that compared with the prior art, it realizes a comprehensive breakthrough in sensing mode, privacy protection, environmental adaptability, identification accuracy and response intelligence. By using non-contact millimeter wave radar sensing technology, the privacy leakage and use compliance problems of traditional cameras and wearable devices in private scenes are effectively solved; combined with intelligent voice interaction verification and multi-modal information fusion discrimination mechanism, the false positive rate is significantly reduced, and the accuracy of event recognition is improved; and through the multi-terminal linkage grading response strategy, a closed-loop disposal process from detection, confirmation to rescue is constructed, the emergency response time is greatly shortened, and the intelligentization, humanization and high efficiency of the elderly fall monitoring system are truly realized.

[0045] Having generally described the exemplary embodiments of the method and system of the present application, reference will now be made to the Figure 3 The computer-readable storage medium of the exemplary embodiments of the present application is described with reference to Figure 3 The computer-readable storage medium shown is an optical disc 30, which stores a computer program (i.e., a program product) thereon, the computer program, when executed by a processor, implements each step described in the above method embodiments, for example, a penetrating behavior perception module for collecting human dynamic information in a weak light visual blind area environment using an electromagnetic wave radar sensor array; a voice interaction verification and decision module connected to an event trigger interface of the penetrating behavior perception module, the voice interaction verification and decision module including a far-field voice excitation unit; and a heterogeneous terminal multi-protocol linkage scheduling module for setting a priority queue management mechanism to confirm a behavior perception output fall instruction, the specific implementation of each step is not repeated here.

[0046] It should be noted that examples of the computer-readable storage medium can also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other optical, magnetic storage media, which are not repeated here.

[0047] Having generally described the exemplary embodiments of the method and medium of the present application, reference will now be made to the Figure 4 The elderly fall monitoring and multi-terminal linkage emergency response device of the exemplary embodiments of the present application.

[0048] Figure 4 A block diagram of an exemplary computing device 40 suitable for implementing exemplary embodiments of the present application is shown, which can be a computer system or a server. Figure 4 The computing device 40 shown is merely one example and should not be construed as limiting the scope of the present embodiments.

[0049] As shown in Figure 4 The components of the computing device 40 can include, but are not limited to, one or more processors or processing units 401, a system memory 402, and a bus 403 that couples various system components including the system memory 402 and the processing unit 401.

[0050] The computing device 40 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by the computing device 40 and includes both volatile and non-volatile media, removable and non-removable media.

[0051] System memory 402 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 4021 and / or cache memory 4022. Computing device 40 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, ROM 4023 can be used for Figure 4 storage of non-removable, non-volatile media (e.g., read only memory (ROM)) 4024. A basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computing device 40, such as during start-up, can typically be stored in ROM 4023. RAM 4021 can also include a Figure 4 transitory medium upon which one or more program modules can be stored and executed. By way of example, and not limitation, a RAM 4021, such as system memory 402, is described herein to store information, such as computer program, using a transitory medium. The transitory medium can also be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device.

[0052] Program / utility 4025, having a set of programs / modules 4024, can be stored in, for example, system memory 402 and implemented by such programs / modules 4024. Such programs / modules 4024 include, but are not limited to, an operating system, one or more applications, other program modules, and program data, each of which or a combination thereof, can include implementation of a network environment. Program modules 4024 generally carry out the functions and / or methodologies described in embodiments of the application.

[0053] Computing device 40 can also communicate with one or more external devices 404 such as a keyboard or a pointing device, through I / O interface 405. Additionally, computing device 40 can communicate with one or more networks such as a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet, through network adapter 406. As Figure 4 illustrated, network adapter 406 can communicate with the other components of computing device 40, through bus 403. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with computing device 40. Examples, include, but are not limited to, microcode, device drivers, redundant processing units, and external disk drive arrays, to name a few. Figure 4

[0054] ​The processing unit 401 executes various function applications and data processing by running programs stored in the system memory 402, for example, using the electromagnetic wave radar sensor array to non-invasively collect human dynamic information in a weak light visual blind area environment; the event trigger interface connected to the penetration behavior perception module, the voice interaction verification decision module includes a far-field voice excitation unit; a priority queue management mechanism is set to confirm the behavior perception output fall instruction; the frequency band configured in the monitoring area uses a modulation method to transmit electromagnetic waves and receive the target body reflection echo of the monitoring area frequency band, and outputs the original intermediate frequency signal; the human cloud data stream of the original intermediate frequency signal is obtained, combined with the static modeling and dynamic threshold segmentation algorithm, the human dynamic cloud data stream or the human static cloud data stream is extracted; the spatial motion point set is constructed according to the dynamic cloud data stream; the spatial motion point set is constructed based on the dynamic cloud data stream, the action mode of the human fall process is identified, whether the suspected fall event is triggered is determined in combination with the electromagnetic wave radar sensor array, the behavior perception output fall instruction with a time stamp is generated, and input to the voice interaction verification decision module.

[0055] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0056] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. The device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, which can be electrical, mechanical or other forms.

[0057] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0058] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0059] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the prior art that essentially contributes or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0060] Finally, it should be noted that the above embodiments are only specific implementations of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0061] In addition, although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all of the shown operations must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, combined into one step, and / or divided into multiple steps.

[0062] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application, and they should be covered within the scope of the claims of the present application.

Claims

1. A fall monitoring and multi-port linked emergency response system for the elderly, characterized by: comprising, a penetrating behavior perception module for collecting human body dynamic information in a weak light visual blind area environment using an electromagnetic wave radar sensor array; a voice interaction verification decision module connected to an event trigger interface of the penetrating behavior perception module, the voice interaction verification decision module comprising a far-field voice incentive unit; a heterogeneous terminal multi-protocol linkage scheduling module for setting a priority queue management mechanism to confirm a behavior perception output fall instruction.

2. The old person fall monitoring and multi-terminal linkage emergency response system according to claim 1, wherein: the penetrating behavior perception module comprises: a radar signal collection unit configured in a frequency band of a monitoring area, which transmits electromagnetic waves in a modulation mode and receives target body reflection echoes in the frequency band of the monitoring area to output original intermediate frequency signals; a preprocessing suppression unit for obtaining human body cloud data streams of the original intermediate frequency signals, combining static modeling and dynamic threshold segmentation algorithms, and extracting human body dynamic cloud data streams or human body static cloud data streams; a behavior sequence analysis output unit for constructing a spatial motion point set according to the dynamic cloud data streams; a posture feature reconstruction unit for constructing a spatial motion point set based on the dynamic cloud data streams, identifying a motion mode of a human body fall process, determining whether to trigger a suspected fall event in combination with the electromagnetic wave radar sensor array, generating a behavior perception output fall instruction with a time stamp, and inputting the behavior perception output fall instruction to the voice interaction verification decision module.

3. The fall monitoring and multi-port linked emergency response system for the elderly of claim 2, wherein: The voice interaction verification decision module further comprises verifying the behavior perception output fall instruction with a time stamp through the voice interaction verification decision module, and the verification through the voice interaction verification decision module comprises an interactive verification process with context awareness, calling the far-field voice incentive unit to generate an inquiry sentence, and processing the inquiry sentence through the electromagnetic wave radar sensor array; the processing of the inquiry sentence comprises judging whether the content of the inquiry sentence belongs to a self-raise, help needed, pain unable to move, or no response state category, and calculating a comprehensive confidence score of the state category through a Bayesian fusion algorithm; When the integrated confidence score is higher than a first threshold and the content of the inquiry statement is determined to be a help signal, the incident is flagged as a level one emergency. When the combined confidence score is between the first threshold and the second threshold , there is an ambiguous or partial response situation, the event is flagged as a secondary pending event; When the composite confidence score is below a second threshold When the composite confidence score is below a second threshold and the query statement is free of anomalous expressions, a false positive confirmation event is determined and the process is terminated.

4. The fall monitoring and multi-port linked emergency response system for the elderly of claim 3, wherein: the calculation formula of the comprehensive confidence score is: wherein, represents the probability that the voice semantic recognition outputs the corresponding help category on the premise that the event is a real emergency, represents the gesture feature on the premise that the event is a real emergency, represents the prior probability of the event, represents the marginal probability, represents the comprehensive confidence score.

5. The fall monitoring and multi-port linked emergency response system for the elderly of claim 4, wherein: the first threshold value and the second threshold value the first threshold value and the second threshold value the decision threshold value is calculated by a numerical optimization method, and the specific formula is: wherein, denotes a decision threshold, denotes a candidate threshold, denotes a false alarm cost coefficient, denotes a negative class cumulative distribution function value, denotes a positive class cumulative distribution function value.

6. The fall monitoring and multi-port linked emergency response system for the elderly of claim 5, wherein: the first threshold value and the second threshold value comprises using a decision threshold a calculation, specifically the following formula: wherein denotes a correction gain coefficient, denotes a secondary threshold scaling factor, denotes a decision threshold, denotes a first threshold, denotes a second threshold; denotes a threshold correction amount.

7. The fall monitoring and multi-port linked emergency response system for the elderly of claim 6, wherein: The heterogeneous terminal multi-protocol linkage scheduling module further comprises setting a priority queue management mechanism according to an event level label output by the voice interaction verification decision; for a first-level emergency event, a safety alarm message containing the behavior perception output fall instruction with a time stamp, the spatial motion point set, and the posture feature reconstruction is encapsulated, a priority transmission channel is allocated through a scheduling strategy, the message is pushed to a nursing station system terminal through a link, and the message is transmitted to a family mobile terminal and a community smart elderly care cloud platform through network encryption, thereby triggering interface linkage of an emergency system; for a second-level to-be-confirmed event, a delay observation mechanism is started, posture changes and voice inputs are continuously monitored within a set period, if no deterioration trend appears, the event is automatically downgraded and archived as a false alarm confirmation event, otherwise, the event is upgraded to a first-level emergency event and determined as a help signal.

8. A method for fall detection and multi-terminal linkage emergency response for the elderly, based on the system for fall detection and multi-terminal linkage emergency response for the elderly according to any one of claims 1 to 7, characterized in that: comprising, collecting human body dynamic information in a weak light visual blind area environment using an electromagnetic wave radar sensor array; connecting to an event trigger interface of a penetrating behavior perception module, the voice interaction verification decision module comprising a far-field voice incentive unit; A priority queue management mechanism is set to confirm the behavior perception output fall instruction; The frequency band configured in the monitoring area transmits electromagnetic waves in a modulation mode and receives the echo reflected by the target body in the frequency band of the monitoring area, and outputs the original intermediate frequency signal; The human body cloud data stream of the original intermediate frequency signal is acquired, and a static modeling and dynamic threshold segmentation algorithm is combined to extract the human body dynamic cloud data stream or the human body static cloud data stream; A spatial motion point set is constructed according to the dynamic cloud data stream; Based on the spatial motion point set constructed according to the dynamic cloud data stream, the action mode of the human body falling process is identified, whether a suspected fall event is triggered is determined in combination with the electromagnetic wave radar sensing array, a behavior perception output fall instruction with a time stamp is generated, and is input into a voice interaction verification judgment module. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the fall monitoring and multi-terminal linkage emergency response system for the elderly in any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the fall monitoring and multi-terminal linkage emergency response system for the elderly in any one of claims 1-7.

Images