Fall recognition method and device

Abstract

The invention discloses a tumble identification method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: acquiring a reflection energy change condition of a to-be-monitored object; wherein the reflection energy change condition is obtained by sending a radar detection signal to an area where the object is located; determining suspected tumble of the object in response to the fact that the change condition of the reflected energy exceeds a preset attitude fluctuation amplitude, and determining a target historical time period according to current time and a preset step length; and obtaining target video stream data of the target historical time period from the local, determining that the object falls according to the target video stream data, and triggering an alarm. According to the embodiment, only tumble pre-judgment is performed daily, and only when the pre-judged object is suspected to tumble, the video of the related target historical time period is called from the local for analysis, so that real-time analysis is avoided, and the problems of privacy disclosure risk and power consumption increase are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a fall recognition method and device. Background Art

[0002] Current fall monitoring technology mainly uses millimeter wave radar or camera combined with AI (Artificial Intelligence) algorithm for real-time posture analysis. Recently, two-in-one monitoring products have emerged that combine the advantages of both, improving detection accuracy. However, this dual-mode monitoring method has the risk of privacy leakage and increased power consumption, especially for battery-powered devices, which will significantly shorten their working time. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide a fall recognition method and device, which can at least solve the problem of privacy leakage risk and increased power consumption in the dual-mode monitoring method in the prior art.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, a fall recognition method is provided, comprising:

[0005] Obtaining changes in reflected energy of the object to be monitored; wherein the changes in reflected energy are obtained by sending a radar detection signal to the area where the object is located;

[0006] In response to the reflected energy change exceeding a preset posture fluctuation amplitude, determining that the object is suspected of falling, and determining a target historical time period according to a current time and a preset step length;

[0007] Target video stream data of the target historical time period is locally acquired, and in response to determining that the subject has fallen according to the target video stream data, an alarm is triggered.

[0008] Optionally, the method further comprises: collecting postures of the objects in the area, obtaining video stream data and storing the data locally.

[0009] Optionally, in response to the reflected energy change exceeding a preset posture fluctuation amplitude, determining that the object is suspected of falling includes:

[0010] calculating an energy difference between current reflected energy and last reflected energy, and in response to the energy difference being greater than or equal to a preset energy threshold, determining that the subject is suspected of falling;

[0011] calculating an energy difference and a time difference between current reflected energy and last reflected energy, and determining that the subject is suspected of falling in response to a quotient of the energy difference and the time difference being greater than or equal to a preset rate threshold;

[0012] The energy difference and time difference between the current reflected energy and the last reflected energy are calculated, and in response to the energy difference being greater than or equal to a preset energy threshold, and the quotient of the energy difference and the time difference being greater than or equal to a preset speed threshold, it is determined that the object is suspected of falling.

[0013] Optionally, the method further comprises:

[0014] In response to the start-up operation of the fall recognition device, the radar monitoring module and the visual module are turned on, and the area where the object is located is determined according to the information collected by the radar monitoring module or the visual module;

[0015] wherein, the object and the area where it is located are determined according to the reflected energy transmitted by the radar monitoring module;

[0016] Receive the image within the field of view captured by the visual module, perform edge calculation on the image, identify the outline of the object, and then determine the object and the area where it is located.

[0017] Optionally, in response to determining according to the target video stream data that the object has fallen, triggering an alarm includes one of the following situations:

[0018] Sending the target video stream data to a local analysis module, so that the local analysis module calculates the probability of falling, and in response to the probability of falling being greater than or equal to a preset probability of falling threshold, determining that the object has fallen, and triggering an alarm; in response to the probability of falling being less than the preset probability of falling threshold, sending the target video stream data to a server-side analysis module; receiving an analysis result returned by the server-side analysis module, and in response to the analysis result indicating that the object has fallen, triggering an alarm;

[0019] Sending the target video stream data to a local analysis module, so that the local analysis module calculates the fall probability, and in response to the fall probability being greater than or equal to a preset fall probability threshold, determining that the subject has fallen and triggering an alarm; in response to the fall probability being less than the preset fall probability threshold, confirming that the subject has not fallen;

[0020] The target video stream data is sent to a server-side analysis module, an analysis result returned by the server-side analysis module is received, and in response to the analysis result that the object has fallen, an alarm is triggered.

[0021] To achieve the above object, according to another aspect of an embodiment of the present invention, there is provided a fall recognition device, including a radar monitoring module, a visual module, a video storage module and a control module;

[0022] The radar monitoring module is used to send a radar detection signal to the area where the monitored object is located, obtain an echo signal, determine the reflected energy according to the echo signal, and then obtain the reflected energy change. In response to the reflected energy change exceeding the preset posture fluctuation amplitude, it is determined that the object is suspected of falling;

[0023] The visual module is used to collect the posture of the object, obtain video stream data, and send the video stream data to the video storage module;

[0024] The video storage module is used to locally store the video stream data;

[0025] The control module is used to receive the suspected fall information of the object, determine the target historical time period according to the current time and the preset step length; obtain the target video stream data of the target historical time period from the video storage module, and trigger an alarm in response to determining that the object has fallen according to the target video stream data.

[0026] Optionally, the device further comprises an analysis module, which is used to analyze the target video stream data to determine whether the object falls, obtain the analysis result and return it to the control module.

[0027] Optionally, the analysis module includes one or more of a local analysis module and a server analysis module, and the analyzing the target video stream data to determine whether the object falls includes the following situations:

[0028] Analyze the target video stream data through the local analysis module to obtain the fall probability, and in response to the fall probability being greater than or equal to the preset fall probability threshold, determine that the object has fallen; in response to the fall probability being less than the preset fall probability threshold, send the target video stream data to the server analysis module; analyze the target video stream data through the server analysis module to obtain an analysis result;

[0029] Analyzing the target video stream data through a local analysis module to obtain a fall probability, and in response to the fall probability being greater than or equal to a preset fall probability threshold, determining that the object has fallen; and in response to the fall probability being less than the preset fall probability threshold, confirming that the object has not fallen;

[0030] The target video stream data is analyzed by a server-side analysis module to obtain an analysis result.

[0031] Optionally, the analysis module is further used for:

[0032] Determine the object and the area where it is located according to the reflected energy transmitted by the radar monitoring module;

[0033] Receive an image within the field of view acquired by the visual module, perform edge calculation on the image, identify the outline of the object, and then determine the object and the area where it is located;

[0034] In which, in response to the start-up operation of the fall recognition device, the radar monitoring module and the visual module are turned on, and the information collected by the radar monitoring module or the visual module is forwarded to the analysis module through the control module.

[0035] To achieve the above objective, according to another aspect of an embodiment of the present invention, a fall recognition electronic device is provided.

[0036] The electronic device of an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned fall recognition methods.

[0037] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, any of the above fall recognition methods is implemented.

[0038] To achieve the above object, according to another aspect of an embodiment of the present invention, a computer program product is provided. A computer program product of an embodiment of the present invention includes a computer program, and when the program is executed by a processor, the fall recognition method provided by the embodiment of the present invention is implemented.

[0039] According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: falls are predicted by millimeter-wave radar, and only when the predicted object is suspected of falling, the video of the relevant target historical time period is retrieved from the local for AI analysis, thereby overcoming the two main shortcomings of the prior art: 1) Usually only millimeter-wave radar or control module is used for fall perception, and video stream data is not uploaded to the cloud, effectively protecting daily privacy; 2) Only when the predicted object is suspected of falling, the video stream data of this segment is retrieved for analysis, avoiding real-time analysis and greatly reducing power consumption.

[0040] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0042] Figure 1 is a schematic diagram of main modules of a fall recognition device according to an embodiment of the present invention;

[0043] Figure 2is a schematic diagram of the main process of a fall recognition method according to an embodiment of the present invention;

[0044] Figure 3 is a flowchart of a specific fall recognition method according to an embodiment of the present invention;

[0045] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0046] Figure 5 It is a schematic diagram of the structure of a computer system of a mobile device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0048] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0049] Fall monitoring is a very important and valuable task in many scenarios, especially for elderly people living alone. At present, the human fall monitoring for home use on the market is mainly through an accessory fixed in the room, such as a millimeter-wave human posture detector, or a camera that collects human posture videos in real time and uses AI algorithms to judge the results. In recent years, some smart hardware for fall monitoring has emerged, and two-in-one monitoring products that use millimeter-wave radar and visual AI to monitor falls. The two sensors perform fall monitoring and analysis at the same time, which can improve the accuracy of fall detection. It is also the latest fall detection technology. Among them, millimeter-wave radar: a radar that uses electromagnetic waves with a wavelength in the millimeter range to achieve target detection.

[0050] However, this dual detection method has obvious problems: first, the visual algorithm needs to analyze the human body posture in real time, which requires the video stream data to be transmitted to the cloud for AI analysis in real time, which brings the risk of daily privacy leakage. Second, turning on two detection methods at the same time will greatly increase power consumption, which will lead to frequent charging problems for battery-powered devices (such as household robots).

[0051] See also Figure 1 , shows a schematic diagram of the main modules of a fall recognition device provided by an embodiment of the present invention, including a radar monitoring module 1, a visual module 2, a video storage module 3, an analysis module 4, and a control module 5:

[0052] 1. Radar monitoring module 1, such as a millimeter wave radar monitoring module, preferably one. The radar echo signal can detect the body motion information of the object to be monitored within the specified detection range. The object to be monitored is preferably a human body. In addition, other target objects that need to be monitored can also be considered.

[0053] Radar can determine reflected energy based on echo signals. When the radar transmits signals, these signals encounter the object to be monitored and reflect back. The echo signals received by the radar receiver contain information about the object to be monitored, including distance, speed, and reflected energy. The strength of the reflected energy is usually related to factors such as the size, shape, material, and distance of the object to be monitored from the radar. By analyzing these echo signals, the radar system can estimate the characteristics of the object to be monitored. Among them, echo signal: the signal reflected by the target after the radar sends the detection signal.

[0054] Radar can determine changes in reflected energy. By continuously transmitting detection signals and receiving echo signals, radar can monitor changes in the intensity of reflected signals. These changes may be caused by a variety of factors, such as the movement of the monitored object, changes in surface characteristics, changes in environmental conditions (such as rain, snow, fog, etc.), and changes in the relative position between the monitored object and the radar. By analyzing these changes in reflected energy, radar can achieve target tracking, identification, classification and other functions. For example, in meteorological radars, changes in reflected energy can be used to monitor meteorological phenomena such as precipitation intensity and wind speed. In military radars, changes in reflected energy can help identify and track enemy targets. In this scheme, the main monitoring is whether the monitored object falls.

[0055] 2. The visual module 2, preferably one in number, can collect the motion posture of the object to be monitored within the field of view, obtain video stream data, and send the video stream data to the video storage module 3. Here, the visual module 2 is usually provided with a camera component, such as a camera, through which the motion posture of the object to be monitored is collected.

[0056] 3. Video storage module 3 is located locally and is used to store video stream data locally. A variety of strategies can be used for storage here. For example, if the local storage space is limited, assuming that the video stream data is stored once every 1 minute, 5 video stream data can be stored, that is, up to the latest 5 segments can be stored, and they can be overwritten when full. It can also be a complete overwrite storage, storing only one segment and overwriting it cyclically. However, if the local storage space is large, the maximum storage capacity can be set according to the frequency of video stream data acquisition by the visual module 2.

[0057] 4. Analysis module 4, preferably an AI analysis module, can be a local analysis module or a server-side analysis module, and can also include a local analysis module and a server-side analysis module, that is, local and server-side interaction is required. The local analysis module here can perform local analysis on the video stream data to determine whether there is a fall, and the server-side analysis module can perform cloud analysis on the video stream data to determine whether there is a fall. For the combination of the two, local analysis can be performed first. If the local analysis result is a confirmed fall, there is no need to send it to the cloud for processing. If the analysis result is no fall, it needs to be sent to the cloud for secondary confirmation.

[0058] 5. Control module 5 needs a master control to execute the fall recognition logic. Specifically, the radar monitoring module 1 sends the reflected energy change of the monitored object to the control module 5, and the control module 5 determines whether the reflected energy change exceeds the preset posture fluctuation amplitude. If it meets the requirements, it is determined that the object is suspected of falling, and the target historical time period is determined according to the current time and the preset step length. For example, if the current time is 10:50 and the step length is 5 minutes, the target historical time period is 10:45-10:50; the target video stream data from 10:45 to 10:50 is obtained from the video storage module 3, and the target video stream data is sent to the analysis module 4. In response to the result returned by the analysis module 4 that confirms the fall, an alarm is triggered.

[0059] In the above-mentioned implementation, the control module 5 determines whether the change of the reflected energy exceeds the preset posture fluctuation range, and if so, determines that the object is suspected of falling. In actual operation, the radar monitoring module 1 can also autonomously determine whether the change of the reflected energy exceeds the preset posture fluctuation range, and if so, determines that the object is suspected of falling, and sends a video call instruction or information about the suspected fall of the object to the control module 5, so that the control module 5 confirms the target historical time period, obtains the target video stream data of the target historical time period from the video storage module 3, sends the target video stream data to the analysis module 4, and responds to the result returned by the analysis module 4 that the fall is confirmed, triggering an alarm. In this solution, the radar monitoring module 1 autonomously determines whether the change of the reflected energy exceeds the preset posture fluctuation range to achieve fall prediction.

[0060] See also Figure 2, which shows a main flow chart of a fall recognition method provided by an embodiment of the present invention, comprising the following steps:

[0061] S201: Obtaining a change in reflected energy of an object to be monitored; wherein the change in reflected energy is obtained by sending a radar detection signal to an area where the object is located;

[0062] S202: In response to the reflected energy change exceeding a preset posture fluctuation amplitude, determining that the object is suspected of falling, and determining a target historical time period according to the current time and a preset step length;

[0063] S203: Acquire the target video stream data of the target historical time period locally, and trigger an alarm in response to determining that the subject has fallen according to the target video stream data.

[0064] In the above embodiment, the change of reflected energy of the object to be monitored is obtained through the radar monitoring module, such as transmitting a radar detection signal to the detection range (i.e., the area where the object to be monitored is located) in real time to receive an echo signal in real time, and then determining the reflected energy based on the echo signal, and generating a reflected energy change based on the reflected energy at different times, which reflects the change of the reflected energy.

[0065] The radar monitoring module or control module determines whether the change in the reflected energy exceeds the preset posture fluctuation amplitude. If it meets the requirements, it is determined that the object is suspected of falling. Specifically, if the change in reflected energy exceeds a certain threshold, it is considered that a suspected fall has occurred. However, considering that falling and lying down are similar, in order to distinguish between the two, it is also possible to consider that the rate of change exceeds a certain threshold, and it is considered that a suspected fall has occurred. Falling usually happens in an instant, so the energy difference and time difference between the current reflected energy and the last reflected energy can be calculated, and based on one or more of the comparison of the energy difference and the preset energy threshold, the comparison of the quotient of the energy difference and the time difference with the preset rate threshold, confirm whether the monitored object is suspected of falling.

[0066] When it is determined that the monitored object is suspected of falling, the control module determines the target historical time period according to the current time and the preset step length. For example, if the current time is 10:50 and the step length is 5 minutes, the target historical time period is 10:45-10:50; the target video stream data from 10:45 to 10:50 is obtained from the video storage module. The target video stream data is sent to the analysis module, which analyzes the target video stream data to determine whether the object has fallen and obtains the final analysis result. In response to the result returned by the analysis module to confirm the fall, the control module triggers an alarm reminder, such as obtaining the geographical location where the fall occurred, and sending the fall alarm information and the geographical location of the fall to the guardian or the monitoring center.

[0067] The analysis module of this solution includes one or more of a local analysis module and a server analysis module. When the analysis module includes the local analysis module and the server analysis module, the specific operation of confirming whether the object falls is as follows:

[0068] Step 1: Send the target video stream data to the local analysis module, the local analysis module calculates the probability of falling, and in response to the probability of falling being greater than or equal to the preset threshold value of the probability of falling (such as 70%), determines that the object has fallen and transmits it to the control module, so that the control module triggers an alarm. For example, it is recognized that the elderly suddenly loses balance and falls to the ground quickly. Based on the action features of this video stream data, the probability of falling is 85% at this time, which exceeds the set threshold of 70%, so the alarm is triggered.

[0069] Step 2: In response to the fall probability being less than the preset fall probability threshold, the target video stream data is sent to the server-side analysis module. That is, when the calculated fall probability is lower than the preset threshold (such as 70%), it indicates that the action in the current video stream is not sufficient to confirm whether a fall has occurred. At this time, the analysis module will automatically forward the video stream data to the server-side analysis module in the cloud for more accurate analysis. For example, in the above scenario, if the elderly person only squats down to pick up something without actually falling, then the probability of falling at this time may be only 30%, which is lower than the threshold, so the video stream data is sent to the cloud for further judgment.

[0070] Step 3: Receive the analysis result returned by the server-side analysis module, which can be directly sent to the control module. The control module triggers an alarm in response to the analysis result that the object has fallen. After receiving the video stream data forwarded from the local, the server-side analysis module uses more powerful computing resources and advanced algorithm models to conduct in-depth analysis of the video content and finally gives a more accurate analysis result. This is because the server-side analysis module may be continuously updated, while the local analysis module is more complicated and inconvenient to update.

[0071] If the server also confirms that there is a fall in the video, it will feed back this result to the control module. For example, if the cloud analysis confirms that the elderly person has indeed experienced a fall, even if the probability of falling is low, the control module will be notified in time to take action. Based on the exact analysis results returned by the server analysis module, the control module makes the final judgment. If the result on the server also points to a "fall", the corresponding emergency response mechanism will be immediately activated, such as calling an emergency rescue number or notifying caregivers to go to the scene to check the situation. This ensures that even those situations that were initially misjudged as non-fall events can be handled promptly and effectively.

[0072] The method provided in the above embodiment is applied to a hardware system that has both radar and camera for monitoring falls. In daily situations, only the change in the reflected energy of the object is obtained through the radar, and the camera collects the posture of the object and stores it locally in a loop. No real-time analysis is required. Only after the suspected fall of the object is predicted, the backtracked video clip is called and sent to AI for analysis, which greatly reduces power consumption and solves the existing problems of privacy leakage risk and increased power consumption.

[0073] See also Figure 3 , shows a schematic flow chart of a specific fall recognition method according to an embodiment of the present invention. After the process starts, the millimeter wave radar monitoring module and the camera recording in the visual module are turned on at the same time. At this time, the camera starts recording, but does not transmit the video stream data to the cloud, but stores it periodically in the local video storage module. The storage period can be customized as needed.

[0074] Considering that the location of the object to be monitored is not determined when the fall recognition device is just turned on, the first step is to determine the object and the area where it is located. This analysis operation still relies on the analysis module, but the information collected by the radar monitoring module or the vision module needs to be forwarded to the analysis module through the control module. Whether using the radar monitoring module or the vision module, the analysis module can accurately determine the location of the object to be monitored, thereby providing basic information for subsequent fall detection. It should be noted that this operation is completed locally and does not need to be uploaded to the cloud for processing.

[0075] Embodiment 1: Radar monitoring module.

[0076] The radar monitoring module scans the surrounding environment, transmits detection signals and receives reflected echo signals. The reflected energy is determined based on the echo signal, and the reflected energy can contain reflection intensity information at different directions and distances. The control module packages the reflected energy data collected by the radar monitoring module and sends it to the analysis module to ensure the efficiency and accuracy of data transmission. After receiving the reflected energy data, the analysis module begins to process the reflected energy information and calculates the position and shape of different objects through algorithms. Assuming that the radar monitoring module detects a large reflected energy area in the corner of the room, the analysis module can determine that there is a large object here, which may be a person or other object. After further analysis, combined with the known room layout information, the analysis module determines that this reflected energy area corresponds to a standing person and accurately locates the specific location in the room, such as near the bed.

[0077] Embodiment 2: Vision module.

[0078] The visual module captures the video stream data in the room through the camera, and the collected images are forwarded to the analysis module through the control module. For example, in the nursing home room, the camera starts recording the room, including continuous image frames. The control module ensures that the image can be quickly and losslessly transmitted to the analysis module for subsequent processing. After receiving the image, the analysis module performs edge computing processing on each frame of the image to extract the key features in the image. By performing edge detection on the image, the analysis module identifies the outline of the object in the room. For example, the camera captures a person standing in the center of the room, and the analysis module identifies the outline of the person through the edge detection algorithm. After identifying the outline of the person, the analysis module further analyzes the specific location of the person. Combined with the layout information of the room, the analysis module can accurately locate the area where the person is located. For example, the analysis module finds that the outline of the person appears in the lower right corner of the video screen. Combined with the room layout diagram, it can be determined that the person is located in the lower right corner of the room, near the window.

[0079] The radar monitoring module sends a radar detection signal to the area where the monitored object is located in real time, obtains an echo signal, determines the reflected energy based on the echo signal, and then obtains the change in reflected energy. The radar monitoring module or control module determines whether the monitored object is suspected of falling based on the change in reflected energy. For example, if the change in reflected energy exceeds a certain threshold, it is considered that a suspected person has fallen. At this time, a large posture fluctuation is sensed within the millimeter wave monitoring range. Standing and falling have different reflection cross-sections for radar energy. The larger the area, the more energy is reflected. The radar monitoring module can sense the changes in reflected energy of the two postures of standing and falling, and then predict whether a person has fallen.

[0080] When it is determined that the monitored object is suspected of falling, the target historical time period is determined according to the current time and the preset step size, and the target video stream data of the target historical time period is obtained from the video storage module, that is, a short video before the current time. The preset step size in this scheme can be different from the frequency at which the visual module generates video stream data, but usually the preset step size needs to be less than or equal to the acquisition frequency, such as 5 minutes ≤ 6 minutes. This is because a fall is often judged to have already occurred, so the video needs to be traced back for a period of time. The target video stream data is sent to the analysis module for analysis to obtain the analysis result (local analysis result or server-side analysis result). If the analysis result is a fall, an alarm is triggered.

[0081] The method provided in the above embodiment predicts falls through millimeter-wave radar, and only when the predicted object is suspected of falling, the video of the relevant target historical time period is retrieved from the local for AI analysis, thereby overcoming the two main shortcomings of the prior art: usually only millimeter-wave radar or control module is used for fall perception, and video stream data is not uploaded to the cloud, effectively protecting daily privacy; only when the predicted object is suspected of falling, the video stream data is retrieved for analysis, avoiding real-time analysis and greatly reducing power consumption.

[0082] Figure 4 An exemplary system architecture 400 to which embodiments of the present invention may be applied is shown, including terminal devices 401 , 402 , 403 , a network 404 and a server 405 (only an example).

[0083] Terminal devices 401, 402, 403 may be various electronic devices with display screens and supporting web browsing, and may be installed with various communication client applications. Users may use terminal devices 401, 402, 403 to interact with server 405 via network 404 to receive or send messages, etc.

[0084] The network 404 is used to provide a medium for communication links between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0085] The server 405 may be a server that provides various services, such as a backend management server that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403 (for example only). The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - for example only) to the terminal device. It should be noted that the method provided in the embodiment of the present invention is generally executed by the server 405, and accordingly, the device is generally set in the server 405.

[0086] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0087] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0088] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0089] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0090] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0091] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0092] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be arranged in a processor. For example, they may be described as follows: a processor includes a radar monitoring module, a visual module, a video storage module, and a control module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the control module may also be described as a "controller".

[0094] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device executes any of the above fall recognition methods.

[0095] The computer program product of the present invention comprises a computer program, and when the computer program is executed by a processor, the fall recognition method in the embodiment of the present invention is implemented.

[0096] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A fall recognition method, characterized in that: include: Obtaining changes in reflected energy of the object to be monitored; wherein the changes in reflected energy are obtained by sending a radar detection signal to the area where the object is located; In response to the reflected energy change exceeding a preset posture fluctuation amplitude, determining that the object is suspected of falling, and determining a target historical time period according to a current time and a preset step length; Target video stream data of the target historical time period is locally acquired, and in response to determining that the subject has fallen according to the target video stream data, an alarm is triggered.

2. The method according to claim 1, characterized in that The method further includes: collecting postures of the objects in the area, obtaining video stream data and storing the data locally.

3. The method according to claim 1, characterized in that In response to the reflected energy change exceeding a preset posture fluctuation amplitude, determining that the object is suspected of falling includes: calculating an energy difference between current reflected energy and last reflected energy, and in response to the energy difference being greater than or equal to a preset energy threshold, determining that the subject is suspected of falling; calculating an energy difference and a time difference between current reflected energy and last reflected energy, and determining that the subject is suspected of falling in response to a quotient of the energy difference and the time difference being greater than or equal to a preset rate threshold; The energy difference and time difference between the current reflected energy and the last reflected energy are calculated, and in response to the energy difference being greater than or equal to a preset energy threshold, and the quotient of the energy difference and the time difference being greater than or equal to a preset speed threshold, it is determined that the object is suspected of falling.

4. The method according to claim 1, characterized in that: The method further comprises: In response to the start-up operation of the fall recognition device, the radar monitoring module and the visual module are turned on, and the area where the object is located is determined according to the information collected by the radar monitoring module or the visual module; wherein, the object and the area where it is located are determined according to the reflected energy transmitted by the radar monitoring module; Receive the image within the field of view captured by the visual module, perform edge calculation on the image, identify the outline of the object, and then determine the object and the area where it is located.

5. The method according to claim 4, characterized in that The step of triggering an alarm in response to determining that the object has fallen according to the target video stream data includes one of the following situations: Sending the target video stream data to a local analysis module, so that the local analysis module calculates the probability of falling, and in response to the probability of falling being greater than or equal to a preset probability of falling threshold, determining that the object has fallen, and triggering an alarm; in response to the probability of falling being less than the preset probability of falling threshold, sending the target video stream data to a server-side analysis module; receiving an analysis result returned by the server-side analysis module, and in response to the analysis result indicating that the object has fallen, triggering an alarm; Sending the target video stream data to a local analysis module, so that the local analysis module calculates the fall probability, and in response to the fall probability being greater than or equal to a preset fall probability threshold, determining that the subject has fallen and triggering an alarm; in response to the fall probability being less than the preset fall probability threshold, confirming that the subject has not fallen; The target video stream data is sent to a server-side analysis module, an analysis result returned by the server-side analysis module is received, and in response to the analysis result that the object has fallen, an alarm is triggered.

6. A fall recognition device, characterized in that: It includes radar monitoring module, vision module, video storage module and control module; The radar monitoring module is used to send a radar detection signal to the area where the monitored object is located, obtain an echo signal, determine the reflected energy according to the echo signal, and then obtain the reflected energy change. In response to the reflected energy change exceeding the preset posture fluctuation amplitude, it is determined that the object is suspected of falling; The visual module is used to collect the posture of the object, obtain video stream data, and send the video stream data to the video storage module; The video storage module is used to locally store the video stream data; The control module is used to receive the suspected fall information of the object, determine the target historical time period according to the current time and the preset step length; obtain the target video stream data of the target historical time period from the video storage module, and trigger an alarm in response to determining that the object has fallen according to the target video stream data.

7. The device according to claim 6, characterized in that The device also includes an analysis module, which is used to analyze the target video stream data to determine whether the object has fallen, obtain the analysis result and return it to the control module.

8. The device according to claim 7, characterized in that The analysis module includes one or more of a local analysis module and a server analysis module, and the analysis of the target video stream data to determine whether the object falls includes the following situations: Analyze the target video stream data through the local analysis module to obtain the fall probability, and in response to the fall probability being greater than or equal to the preset fall probability threshold, determine that the object has fallen; in response to the fall probability being less than the preset fall probability threshold, send the target video stream data to the server analysis module; analyze the target video stream data through the server analysis module to obtain an analysis result; Analyzing the target video stream data through a local analysis module to obtain a fall probability, and in response to the fall probability being greater than or equal to a preset fall probability threshold, determining that the object has fallen; and in response to the fall probability being less than the preset fall probability threshold, confirming that the object has not fallen; The target video stream data is analyzed by a server-side analysis module to obtain an analysis result.

9. The device according to claim 8, characterized in that The analysis module is also used for: Determine the object and the area where it is located according to the reflected energy transmitted by the radar monitoring module; Receive an image within the field of view acquired by the visual module, perform edge calculation on the image, identify the outline of the object, and then determine the object and the area where it is located; In which, in response to the start-up operation of the fall recognition device, the radar monitoring module and the visual module are turned on, and the information collected by the radar monitoring module or the visual module is forwarded to the analysis module through the control module.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images