Object abnormal behavior prompting method and device, wearable equipment and medium

Abstract

Embodiments of the invention disclose an object abnormal behavior prompting method and apparatus, a wearable device and a medium. The method comprises the steps of collecting and obtaining device motion change data of the wearable device based on a first frequency; when it is detected that the equipment movement change data meet the risk action detection condition, environment perception data are obtained, and the equipment movement change data and object physiological change data of the equipment wearing object are collected based on a second frequency; if any abnormal behavior detection condition is satisfied, determining that the equipment wearing object has an abnormal behavior state; and performing cooperative detection on the multi-modal sensor data, determining abnormal level information corresponding to the abnormal behavior state, and prompting a corresponding target user based on a target prompting mode corresponding to the abnormal level information, the problem of response lag caused by the fact that a traditional wearable device depends on manual alarm information triggering and single-detection-dimension anomaly detection is effectively solved, and the accuracy of object risk behavior judgment and the timeliness of emergency response are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wearable devices, and particularly relates to an object abnormal behavior prompting method and device, a wearable device and a medium. BACKGROUND

[0002] As an important component of smart wearable devices, phone watches have been widely popularized in recent years due to their positioning, communication and health monitoring functions. Such devices usually integrate GPS, cellular network, Bluetooth and other technologies, aiming to provide safety protection for the wearer and help guardians to grasp the location and health status in real time. With the advancement of technology, the functions of phone watches have gradually become richer, but their core design still revolves around basic needs, such as location tracking, voice calls and exercise data recording. However, the active help-seeking function in emergency situations has not been fully developed, making it difficult to meet the immediate protection needs in sudden danger.

[0003] Currently, the emergency help function of phone watches mainly relies on manual triggering, such as long-pressing a specific button or sending a help signal through a pre-set shortcut operation. Some devices also support the geo-fencing function, which sends an alert to the guardian when the wearer exceeds the set range.

[0004] However, the main problem of the existing technology is the passivity and limitation of the emergency help function. The manual triggering mechanism may not be effective in operating when in panic, injured or coerced, leading to failed help-seeking. At the same time, the lack of intelligent detection algorithms makes it difficult for the device to automatically identify emergency situations, such as falling, getting lost or being taken away by strangers, causing response delays or false alarms. Based on this, the existing wearable devices have the problems of passive triggering unreliability, insufficient intelligent recognition capability and high false alarm rate, leading to response failure or trust crisis in dangerous situations. SUMMARY

[0005] The present application provides an object abnormal behavior prompting method, device, wearable device and medium to overcome the response lag problem caused by the dependence of traditional wearable devices on manual triggering of alarm information and single detection dimension abnormal detection, and to improve the accuracy of object risk behavior judgment and the timeliness of emergency response.

[0006] In a first aspect, an embodiment of the present application provides an object abnormal behavior prompting method applied to a wearable device, which comprises:

[0007] acquiring and obtaining device motion change data of the wearable device based on a first frequency; wherein the device motion change data includes at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope;

[0008] acquire environment perception data, and collect the device motion change data and object physiological change data of the device wearing object based on a second frequency when it is detected that the device motion change data meets a risk action detection condition; wherein the object physiological change data includes at least one of heart rate collected by a heart rate sensor and skin conductance change rate collected by a skin conductance response sensor, and the environment perception data includes environment audio data collected by an audio acquisition unit and environment image data collected by a shooting unit;

[0009] If any of the following abnormal behavior detection conditions is met, it is determined that the device wearing object has an abnormal behavior state; the abnormal behavior detection conditions include: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s 2 ; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10μS; the environment audio data detects an abnormal audio segment; and the environment image data detects an abnormal image frame;

[0010] The multi-modal sensor data is cooperatively detected to determine abnormal level information corresponding to the abnormal behavior state, so as to prompt a target user in a target prompting mode corresponding to the abnormal level information;

[0011] The threshold values in the risk action detection condition and the abnormal behavior detection condition are periodically updated based on historical device motion change data and historical object physiological change data of the device wearing object.

[0012] In a second aspect, the embodiments of the present application also provide an object abnormal behavior prompting device applied to a wearable device, which comprises:

[0013] A first data acquisition module is configured to collect and acquire device motion change data of the wearable device based on a first frequency; wherein the device motion change data includes at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope;

[0014] A second data acquisition module is configured to acquire environment perception data and collect the device motion change data and object physiological change data of the device wearing object based on a second frequency when it is detected that the device motion change data meets a risk action detection condition; wherein the object physiological change data includes at least one of heart rate collected by a heart rate sensor and skin conductance change rate collected by a skin conductance response sensor, and the environment perception data includes environment audio data collected by an audio acquisition unit and environment image data collected by a shooting unit;

[0015] An abnormal behavior detection module is configured to determine that the device wearing object is in an abnormal behavior state if any of the following abnormal behavior detection conditions is met: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10 μS; the environmental audio data detects an abnormal audio segment; and the environmental image data detects an abnormal image frame. 2

[0016] An abnormal behavior prompting module is configured to perform collaborative detection on the multi-modal sensor data, determine abnormal level information corresponding to the abnormal behavior state, and prompt a target user based on a target prompting manner corresponding to the abnormal level information.

[0017] The threshold values in the risk action detection condition and the abnormal behavior detection condition are periodically updated based on historical device motion change data and historical object physiological change data of the device wearing object.

[0018] In a third aspect, an electronic device is provided, and the electronic device includes:

[0019] one or more processors;

[0020] a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the object abnormal behavior prompting method according to any of the embodiments of the present application.

[0021] In a fourth aspect, a storage medium containing computer executable instructions is provided, when the computer executable instructions are executed by a computer processor, the computer processor is caused to implement the object abnormal behavior prompting method according to any of the embodiments of the present application.

[0022] ​The technical scheme of the embodiment of the present application is based on a first frequency to collect and acquire device motion change data of a wearable device, the device motion change data including at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope; thus, when it is detected that the device motion change data meets a risk action detection condition, environmental perception data is acquired, and device motion change data and object physiological change data of a device wearing object are collected based on a second frequency; wherein the object physiological change data includes at least one of a heart rate collected by a heart rate sensor and a skin conductance change rate collected by a skin conductance sensor, and the environmental perception data includes environmental audio data collected by an audio collection unit and environmental image data collected by a shooting unit; further, if any of the following abnormal behavior detection conditions is met, it is determined that the device wearing object has an abnormal behavior state; the abnormal behavior detection conditions include: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10μS; an abnormal audio segment is detected in the environmental audio data; and an abnormal image frame is detected in the environmental image data; finally, multi-modal sensor data is cooperatively detected to determine abnormal level information corresponding to the abnormal behavior state, and a target user is prompted in a target prompt mode corresponding to the abnormal level information. The technical scheme provided in the embodiment has a detection accuracy of emergency help reaching 95% or more, a false alarm rate of less than 5%, and a system response time of less than 2 seconds, ensuring that guardians quickly learn about emergency situations; the scheme realizes active identification of abnormal states of the human body based on dynamic monitoring of acceleration and angular velocity motion parameters combined with physiological index analysis of heart rates and skin conductance; at the same time, cross verification of environmental audio and image data significantly reduces the false alarm rate, and a determination strategy including a grading warning mechanism is adopted, i.e., a hierarchical response according to the degree of emergency, low-risk recording only, medium-risk notification pushing, high-risk automatic alarm and phone call, which significantly improves the warning efficiency, effectively overcomes the response lag problem caused by manual trigger alarm information and single detection dimension abnormal detection of traditional wearable devices, and improves the accuracy of object risk behavior determination and the timeliness of emergency response. 2 BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical scheme of the exemplary embodiments of the present application, the drawings needed in the description of the embodiments are briefly introduced as follows. Obviously, the drawings introduced are only a part of the drawings of the embodiments to be described by the present application, and not all the drawings. Those skilled in the art can obtain other drawings from these drawings without creating any creative labor.

[0024] Figure 1 A flowchart of an object abnormal behavior prompting method provided by the embodiment of the present application; ​

[0025] Figure 2 A flowchart of a method for prompting abnormal behavior of an object according to an embodiment of the present application is shown in FIG. 1. The method can be applied to any situation where real-time detection of abnormal behavior of a wearer based on multi-modal data and triggering of intelligent warning are required. The method can be executed by an object abnormal behavior prompting device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device such as a mobile terminal, a PC terminal, or a server, etc.

[0026] Figure 3 A structural diagram of an object abnormal behavior prompting device according to an embodiment of the present application is shown in FIG. 2. The device can be implemented in the form of software and / or hardware. The hardware can be an electronic device such as a mobile terminal, a PC terminal, or a server, etc.

[0027] Figure 4 A structural diagram of an electronic device according to an embodiment of the present application is shown in FIG. 3. The device can be implemented in the form of software and / or hardware. The hardware can be an electronic device such as a mobile terminal, a PC terminal, or a server, etc. DETAILED DESCRIPTION

[0028] The present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application, but not to limit the present application. In addition, it should be noted that only the parts related to the present application are shown in the accompanying drawings for the purpose of description.

[0029] Embodiment 1

[0030] Figure 1 A flowchart of a method for prompting abnormal behavior of an object according to an embodiment of the present application is shown in FIG. 1. The method can be applied to any situation where real-time detection of abnormal behavior of a wearer based on multi-modal data and triggering of intelligent warning are required. The method can be executed by an object abnormal behavior prompting device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device such as a mobile terminal, a PC terminal, or a server, etc.

[0031] The technical solution provided by the embodiments of the present application can be applied to a wearable device. The wearable device can be a child safety watch, a smart watch, a smart bracelet, smart glasses, a smart helmet, smart clothing, smart shoes, a health monitoring device, a motion tracking device, an AR / VR head-mounted display, or any other wearable electronic device with motion sensing and environmental sensing capabilities. For the purpose of clear description, the wearable device in the following description is taken as a child safety watch. Specifically, the technical solution provided by the embodiments of the present application can be deployed in the processor of the wearable device, so as to implement the method for prompting abnormal behavior of an object.

[0032] The hardware structure of the wearable device can include: motion sensors (acceleration sensors and gyroscopes), physiological sensors (heart rate sensors and galvanic skin response (GSR) sensors), audio acquisition units, shooting units, microprocessors, memories, wireless communication modules, positioning modules, and power management modules. Among them, the acceleration sensor can collect acceleration data in real time; the gyroscope can collect angular velocity data in real time; the heart rate sensor can collect heart rate data of the device wearer in real time; the galvanic skin response (GSR) sensor can collect heart rate data of the device wearer in real time to measure skin conductivity data; the audio acquisition unit can collect environmental audio data; the shooting unit can be activated in abnormal situations to shoot videos or photos. The processor receives and processes the data of all sensors and data acquisition models, and executes the object abnormal behavior prompting method provided in the embodiment; the memory stores historical data and event logs for subsequent analysis. The wireless communication module sends a distress signal to the guardian of the device wearer through a wireless network or a cellular network. The positioning module can provide real-time location information of the device wearer. The power management module optimizes power consumption to ensure long-term operation of the device.

[0033] On this basis, as shown in Figure 1 The object abnormal behavior prompting method comprises:

[0034] S110, collecting and acquiring device motion change data of the wearable device based on a first frequency.

[0035] The first frequency refers to the data sampling rate or data update rate used by the wearable device when initially collecting device motion change data, that is, the number of times of data acquisition of motion parameters such as acceleration change rate and angular velocity change rate per unit time. This frequency serves as a basic monitoring threshold, determining the sensitivity of the device to the wearer's regular movements. Its setting needs to take into account both energy efficiency and basic motion capture requirements, and is usually lower than the second frequency after triggering a risky action, to ensure low power consumption when no abnormalities are detected, and to switch to a more detailed monitoring mode at a higher frequency when potential risks are detected.

[0036] The device motion change data refers to dynamic physical quantity data reflecting the change in the motion state of the device wearer, collected in real time by the acceleration sensor and gyroscope built into the wearable device. The device motion change data can include acceleration change rate collected by the acceleration sensor, and / or angular velocity change rate collected by the gyroscope. The acceleration change rate can reflect the degree of motion intensity and the mutation characteristics of direction; the angular velocity change rate can reflect the dynamic characteristics of limb rotation or posture adjustment.

[0037] Specifically, in the normal monitoring state, the wearable device continuously collects device motion change data through the acceleration sensor and the gyroscope at a preset basic sampling rate, i.e., a first frequency. The first frequency serves as a default baseline sampling rate and is mainly used for basic monitoring of the regular movements of the device wearer in a low-power state. The core purpose of setting the first frequency is to optimize energy consumption performance while ensuring basic motion feature capture capability, thereby providing initial data support for subsequent risk action judgment.

[0038] For example, when a child wearing a safety watch walks normally, the device records the acceleration change rate through the acceleration sensor at a first frequency of 10 Hz, and the gyroscope records the angular velocity change rate at the same frequency. The acceleration change rate can record the slight vibration generated by the gait, such as detecting regular acceleration fluctuations with an amplitude of 0.5-2 m / s 2 The angular velocity change rate can record the angular velocity change formed by the wrist swing, such as periodic data within ±30° / s. This low-frequency continuous sampling can capture basic action features such as hand waving and jogging, and avoid rapid power consumption caused by high-frequency sampling.

[0039] In S120, when it is detected that the device motion change data meets the risk action detection condition, environmental perception data is obtained, and device motion change data and object physiological change data of the device wearer are collected based on a second frequency.

[0040] The risk action detection condition refers to a pre-set dynamic motion feature threshold or pattern rule for judging whether the device wearer is in a dangerous state.

[0041] The environmental perception data refers to a set of objective information collected by the environmental sensors of the wearable device, such as the audio acquisition unit and the shooting unit, which reflects the surrounding scene of the wearer. It mainly includes environmental audio data obtained by the audio acquisition unit and environmental image data captured by the shooting unit.

[0042] The second frequency refers to a higher data acquisition rate switched to by the wearable device when the risk action detection condition is triggered, which is used for fine monitoring of device motion change data and object physiological change data. The frequency, as an enhanced sampling rate in an abnormal state, is significantly higher than the first frequency in the basic monitoring stage. Its core role is to improve the spatio-temporal resolution of data to meet the accuracy requirements of abnormal behavior analysis. The device wearer refers to the direct user of the wearable device, i.e., a biological individual who forms a stable coupling relationship with the device through physical contact methods such as wearing, wearing, etc.

[0043] The object physiological change data refers to a physiological parameter data set reflecting the state of the autonomic nervous system and the cardiovascular function of the device wearer, which is collected in real time by the biological sensor built in the wearable device. The object physiological change data can include heart rate collected by a heart rate sensor, and / or skin conductance change rate collected by a galvanic skin response sensor. The heart rate can represent the rhythm change characteristics of the heart beat. The skin conductance change rate can represent the conductivity fluctuation caused by sweat gland activity, which is a key quantitative indicator for measuring the degree of autonomic nervous excitation in this scheme, and can capture the unconscious physiological changes of the human body in abnormal states such as fear and tension. By detecting the slight fluctuation of the skin surface conductivity caused by sweat gland activity, objective physiological evidence of the emotional stress response of the device wearer is provided.

[0044] In this embodiment, when the device motion change data breaks through the amplitude threshold, change gradient or specific waveform characteristics set by the condition in the time domain or frequency domain, the following two response mechanisms are simultaneously started: first, the audio acquisition unit and the shooting unit are activated to collect the environmental perception data of the surrounding scene; at the same time, the sampling frequency of the motion sensor and the physiological sensor is increased from the basic first frequency to the higher second frequency. In particular, the second frequencies corresponding to the motion sensor and the physiological sensor can be the same or different. Thus, fine synchronization monitoring of the device acceleration rate / angular velocity change rate and physiological indicators such as the heart rate / skin conductance of the device wearing object is achieved. This process essentially builds an “environment-motion-physiology” triple data fusion abnormality detection system, which provides a higher spatiotemporal resolution and more complete scene information analysis basis for subsequent abnormal behavior state judgment by increasing the data acquisition density and rich data dimensions, embodying the adaptive strategy switching from regular monitoring to emergency response.

[0045] For example, in the regular monitoring mode, a low-frequency energy-saving strategy is adopted: the acceleration sensor and the gyroscope continuously collect device motion change data at 10 Hz; when an abnormal motion is detected, such as a sharp acceleration change with an acceleration change rate exceeding the threshold, the S120 emergency mode is switched to immediately, at which time the audio acquisition unit and the shooting unit are activated to collect the environmental perception data of the surrounding scene; at the same time, the sampling frequency of the acceleration sensor and the gyroscope is increased to 50 Hz to achieve high-precision motion capture; the collection frequency of the heart rate sensor and the sampling frequency of the galvanic skin response (GSR) sensor can be once every 5 seconds. For easy processing, the processor can obtain and uniformly process the aggregated data of these sensors once every 5 seconds.

[0046] Optionally, the specific implementation of S120 can include: when the acceleration change rate is greater than the preset acceleration change threshold and / or the angular velocity change rate is greater than the preset angular velocity change threshold, triggering the audio acquisition unit and the shooting unit of the wearable device to be enabled; collecting environmental audio data through the audio acquisition unit and collecting environmental image data through the shooting unit; controlling the acceleration sensor and / or the gyroscope to collect device motion change data based on a second frequency, and controlling the heart rate sensor and / or the galvanic skin response sensor to collect object physiological change data of the device wearing object based on the second frequency.

[0047] The preset acceleration change threshold refers to a preset acceleration change rate threshold for judging whether the motion state of the device wearing object is abnormal. The preset angular velocity change threshold refers to a preset angular velocity change rate threshold for judging whether the motion state of the device wearing object is abnormal. The two thresholds are quantitatively calibrated based on the acceleration change and angular velocity characteristics generated during normal human activities. When the real-time monitored acceleration change rate exceeds the corresponding threshold, it indicates that the wearing object may have made a sudden or sudden abnormal action.

[0048] Specifically, when the acceleration change rate of the device wearing object breaks through the preset acceleration change threshold; or the angular velocity change rate of the device wearing object breaks through the preset angular velocity change threshold; or both the acceleration change rate and the angular velocity change rate of the device wearing object break through the preset acceleration change threshold and the preset angular velocity change threshold, the following response process can be immediately started: on the one hand, the audio acquisition unit and the shooting unit of the wearable device are enabled, so as to collect environmental audio data through the audio acquisition unit and collect environmental image data through the shooting unit; on the other hand, the sampling frequency of the motion sensor (i.e. the acceleration sensor and / or the gyroscope) and the physiological sensor (i.e. the heart rate sensor and / or the galvanic skin response sensor) is increased from the basic first frequency to the higher second frequency, realizing fine and synchronous monitoring of device motion parameters and wearing object physiological indicators, thereby constructing a multi-dimensional feature space containing environmental perception data, high-precision motion data and real-time physiological data, providing a complete evidence chain with spatiotemporal correlation for abnormal behavior recognition. Through this dynamic perception strategy from single motion triggering to multi-source data collaborative analysis, the recognition ability and judgment accuracy of potential risks in complex scenarios are significantly improved.

[0049] S130, if any of the following abnormal behavior detection conditions is met, it is determined that the device wearing object has an abnormal behavior state; the abnormal behavior detection conditions include: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s 2 ; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10μS; the environmental audio data detects an abnormal audio segment; the environmental image data detects an abnormal image frame.

[0050] wherein the abnormal behavior detection condition refers to a pre-set multi-dimensional data fusion rule for comprehensive judgment of the abnormal behavior state, and its essence is to establish a quantitative judgment standard for the abnormal features of the combination of the device motion change data, the object physiological change data and the environment perception data or part of them. The abnormal behavior state refers to a specific state deviating from the normal behavior mode and having potential risks, which is determined by the multi-modal data fusion analysis.

[0051] wherein 3g / s is the first acceleration change threshold value, which is used to preliminarily judge the critical value of the acceleration change rate of the device wearer in case of sudden movement. The threshold value is quantitatively calibrated based on the acceleration change characteristics generated by the human body during normal activities. When the acceleration sensor monitors the acceleration change amplitude in unit time in real time, and the acceleration change amplitude exceeds the threshold value, it indicates that the wearer may have a sudden abnormal action. 150° / s 2 is the first angular velocity change threshold value, which is used to preliminarily identify the critical value of the angular acceleration of the sudden posture change of the device wearer. The threshold value is quantitatively calibrated based on the angular velocity change range generated by the natural rotation of the joints of the human body during regular activities. When the gyroscope monitors the angular velocity change amplitude in unit time in real time, and the angular velocity change amplitude exceeds the threshold value, it indicates that the wearer may have an abnormal rapid turning or body rolling action. 20% is the first heart rate change threshold value, which is used to preliminarily judge the critical value of the heart rate fluctuation of the device wearer in case of physiological stress reaction. The threshold value is individually calibrated based on the basic heart rate level of the wearer and the heart rate change characteristics during normal activities. When the heart rate rise amplitude in unit time is monitored in real time, and the heart rate rise amplitude exceeds the threshold value, it indicates that the wearer may have a sudden physiological abnormal state. 10μS is the first skin conductivity change threshold value, which is used to preliminarily evaluate the critical value of the skin conductivity change of the device wearer in case of autonomic nervous excitement. The threshold value is quantitatively calibrated based on the basic skin conductivity level of the human body in a calm state and its normal fluctuation range. When the skin conductivity change amplitude in unit time is detected, and the skin conductivity change amplitude exceeds the threshold value, it indicates that the wearer may have significant emotional fluctuation or stress reaction.

[0052] wherein the abnormal audio segment refers to a sound signal segment identified from the environmental sound, which has a pre-defined risk feature. For example, an acoustic fingerprint conforming to the abnormal behavior associated feature extracted by a voiceprint pattern recognition model. The segment needs to meet the double abnormal standards in the time domain, such as duration, suddenness, and the frequency domain, such as specific frequency energy distribution. When the environmental audio data breaks through the safety threshold in the dimensions such as sound pressure level, spectral envelope or time-frequency characteristics, it is marked as a sound event that needs attention by comparing the acoustic feature library containing the digital features of typical risk sounds such as help, collision and quarrel.

[0053] The abnormal image frame refers to a visual scene fragment identified from the environmental video stream and meeting a preset risk feature. For example, a visual pattern containing an abnormal behavior associated element can be extracted based on an image recognition model. The frame needs to meet dual abnormality determination criteria of spatial features such as human posture, object position, and temporal features such as motion continuity and event burstiness. By comparing a pre-trained visual feature recognition model of typical abnormal scenes containing falling, chasing, and violence, when the environmental image data breaks through the safety threshold in the dimensions of motion vector, human skeleton point change, or scene semantics, the environmental image data is marked as a potential risk visual event.

[0054] In the embodiment, the data collected by various sensors can be preprocessed in real time, which can include steps such as time domain filtering, frequency domain transformation, and feature extraction. Then, the acceleration change rate is compared with the first acceleration change threshold, the angular velocity change rate is compared with the first angular velocity change threshold, the heart rate is compared with the first heart rate change threshold, and the skin conductance change rate is compared with the first skin conductance rate change threshold. At the same time, the abnormal audio segment in the environmental audio data is identified through an audio analysis model, and the abnormal image frame in the environmental image data is identified through a visual analysis model. When at least one type or combination type of data in the device motion parameters (acceleration change rate / angular velocity change rate), physiological indicators (heart rate / skin conductance change rate), and environmental information (audio / image) meets the preset abnormality determination criteria, it is confirmed that the wearer is in an abnormal behavior state that needs intervention.

[0055] S140, cooperatively detecting the multi-modal sensor data to determine abnormal level information corresponding to the abnormal behavior state, and prompting a corresponding target user based on a target prompt mode corresponding to the abnormal level information.

[0056] The multi-modal sensor data refers to multiple data forms collected by different types of sensors, which have complementarity and heterogeneity. The data can specifically include at least one of the acceleration change rate collected by the acceleration sensor, the angular velocity change rate collected by the gyroscope, the heart rate collected by the heart rate sensor, and the skin conductance change rate collected by the skin conductance sensor. The abnormal level information refers to a hierarchical evaluation result obtained by quantitatively grading the severity or urgency of the detected abnormal behavior state. The abnormal level information is not a simple Boolean judgment of a single sensor trigger condition, but a graded output generated after comprehensive decision-making based on the trigger conditions of multiple abnormal indicators (such as motion, physiology, and environment), the intensity of data features, the duration, and the cooperative correlation between multi-modal data. The information is used to distinguish different risk levels of abnormal events, thereby providing differentiated decision-making basis for subsequent target prompt modes (such as reminder strength and notification object range).

[0057] The target prompt mode refers to selecting the optimal warning means from the preset multiple alarm strategies according to the severity level of the abnormal behavior state. The essence is to map abnormal states of different risk levels to differentiated notification modes through a hierarchical response mechanism. Based on the level evaluation result of the abnormal behavior state, the combination of prompt parameters including but not limited to vibration intensity, light color, sound frequency, message push priority, etc. is dynamically selected, and the information transmission path is determined, such as at least one of local alarm, cloud push and direct contact with guardians.

[0058] The target user refers to a specific information receiving object automatically selected according to the nature of the abnormal behavior state and the preset rules, and the core feature is the preset associated party directly related to the safety of the device wearer. The target user contains at least one of the two dimensions: one is the device wearer himself, such as when the abnormal state needs to be responded autonomously; the other is the preset guardian group, such as when the abnormal state needs external intervention, through analyzing the type, urgency and preset response strategy of the abnormal behavior, the final receiving subject of information push is dynamically determined, this double coverage mechanism not only guarantees the self-processing space of mild abnormality, but also ensures the timely external intervention of major risk, forming a hierarchical user response system.

[0059] In this embodiment, after it is determined that the wearing object is in an abnormal behavior state, instead of analyzing single path data such as acceleration, angular velocity, heart rate, skin conductance, audio, image, etc. in isolation, the multi-modal data obtained by each sensor within the same time window is cross-modal aligned, feature fused and weight allocated, the abnormal degree quantitative index is calculated through a unified algorithm model, the abnormal level information is obtained, and the target prompt mode matched therewith is automatically selected according to the level, the warning content is pushed to the preset target user, the hierarchical, accurate and real-time risk notification is realized. For example, according to the preset abnormal level-response strategy mapping table, a combination scheme including local prompt (vibration mode / LED color / screen display) and remote notification (SMS / APP push / voice call) can be selected, i.e. the target prompt mode is obtained; then, based on the abnormal type, time sensitivity and preset permission setting, the information receiving subject (target user) is determined, such as the person himself and / or the guardian; finally, the warning information is fed back to the target user through the target prompt mode to prompt the target user that the device wearing object may have abnormal risk, at the same time, the warning information can be attached with metadata such as abnormal type, timestamp and geographic location, and the prompt delivery state is monitored in real time through the feedback mechanism.

[0060] In the embodiment, the threshold values in the risk action detection condition and the abnormal behavior detection condition are periodically updated based on historical device motion change data and historical object physiological change data of the device wearing object. By continuously accumulating and analyzing all raw data generated by the acceleration, gyroscope, heart rate and skin electrical response sensors of the wearer in the past period, through periodic modeling and statistical calculation, the threshold values for triggering the determination in the risk action detection condition and the abnormal behavior detection condition are automatically corrected, so that these threshold values dynamically fit the physiological characteristics and behavior patterns of the individual wearer in each update period, thereby avoiding false positives or false negatives caused by fixed threshold values, and realizing adaptive optimization of threshold parameters for individual long-term evolution.

[0061] In the embodiment, personalized and update-adjustable detection threshold values can be determined for different users. The specific implementation can include: periodically obtaining historical device motion change data and historical object physiological change data of the device wearing object based on a preset time task; adjusting the device motion change threshold values in the risk action detection condition and the abnormal behavior detection condition based on the historical device motion change data, and adjusting the object physiological change threshold values in the risk action detection condition and the abnormal behavior detection condition based on the historical object physiological change data.

[0062] The preset time task refers to a data collection and analysis job automatically performed by a preconfigured time period and trigger condition. The historical device motion change data refers to a time series data set reflecting the past motion characteristics of the device wearer continuously recorded in the preset time task period. A user personalized behavior pattern database is constructed by long-term collected and stored motion parameters such as acceleration change rate and angular velocity change rate. The historical object physiological change data refers to a time series parameter set reflecting the basic physiological state of the device wearer continuously recorded in the preset time task period. A personalized biological feature model is constructed by long-term monitored and accumulated physiological indicators such as heart rate variability and skin electrical conductivity. The device motion change threshold value refers to the acceleration change threshold value and the angular velocity change threshold value. The object physiological change threshold value refers to the heart rate change threshold value and the skin electrical conductivity change threshold value.

[0063] Specifically, the motion sensor raw data and the physiological sensor time series record in a specified time window can be extracted from the device memory by a timing trigger at a preset period; then, through a data preprocessing pipeline, noise filtering, outlier rejection and feature standardization are performed to generate a feature dataset with time labels, i.e., the historical device motion change data and the historical object physiological change data. Further, time-frequency analysis can be performed on the historical device motion change data and the historical object physiological change data to extract feature distribution. For example, mean, variance, extreme value, and then personalized acceleration change threshold and angular velocity change threshold are calculated by using the sliding percentile method; at the same time, a Gaussian mixture model is established for the historical object physiological change data, and the abnormal boundary of heart rate / skin conductance is determined by the 3σ principle; then, a time decay factor can be introduced to give higher weight to recent data, and then the new and old threshold values are fused by Kalman filtering. Finally, the threshold rationality can be verified, such as ensuring not to be lower than the basic safety value, and after verification, the threshold value is incrementally updated to the threshold parameter library of the risk action detection condition and the abnormal behavior detection condition, so as to realize periodic updating of the detection threshold value, which can automatically adapt to the change of the motion habit of different users and the difference of physiological characteristics, so as to optimize the static preset threshold value to a dynamic personalized judgment standard, which not only significantly reduces the false positive rate caused by individual differences, but also ensures the continuous sensitivity to the real abnormal behavior of the user, and finally forms an adaptive monitoring mechanism of "one person one threshold value".

[0064] The technical scheme of the embodiment of the application acquires device motion change data of a wearable device including at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope based on a first frequency; thereby, when it is detected that the device motion change data meets a risk action detection condition, environmental perception data is acquired, and device motion change data and object physiological change data of a device-wearing object are acquired based on a second frequency; wherein the object physiological change data includes at least one of a heart rate collected by a heart rate sensor and a skin conductance change rate collected by a skin galvanic response sensor, and the environmental perception data includes environmental audio data collected by an audio acquisition unit and environmental image data collected by a shooting unit; further, if any of the following abnormal behavior detection conditions is met, it is determined that the device-wearing object has an abnormal behavior state; the abnormal behavior detection conditions include: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s 2a rate of change of heart rate greater than 20%; a rate of change of skin conductance greater than 10 muS; environmental audio data detecting an abnormal audio segment; environmental image data detecting an abnormal image frame; and finally, performing collaborative detection on the multi-modal sensor data to determine abnormality level information corresponding to the abnormal behavior state, and prompting a target user in a target prompting manner corresponding to the abnormality level information. The technical scheme provided in the embodiment can achieve an emergency call detection accuracy of 95% or more, a false alarm rate of less than 5%, and a system response time of less than 2 seconds, thereby ensuring that the guardian can quickly learn about the emergency; the scheme combines dynamic monitoring of acceleration and angular velocity motion parameters with physiological index analysis of heart rate and skin conductance, thereby achieving active identification of abnormal states of the human body; at the same time, cross verification of environmental audio and image data can significantly reduce the false alarm rate, and a determination strategy including a grading warning mechanism is adopted, i.e., a graded response according to the emergency level, low-risk recording only, medium-risk notification pushing, high-risk automatic alarm and phone dialing, thereby significantly improving the warning efficiency, and effectively overcoming the response lag problem caused by manual triggering of alarm information and single-dimension abnormality detection of the traditional wearable device, and improving the accuracy of object risk behavior determination and the timeliness of emergency response.

[0065] On the basis of the above-mentioned embodiment, the method can further include: when it is detected that the battery power of the wearable device is less than a preset power threshold, reducing the first frequency and the second frequency, the first frequency being less than the second frequency.

[0066] The preset power threshold refers to a power percentage or an absolute power value preset in the power management system of the wearable device.

[0067] In the embodiment, the wearable device can monitor the battery power in real time, and once the battery power is lower than the power threshold preset by the system, the first sampling frequency for collecting motion change data and the second sampling frequency for simultaneously collecting motion and physiological change data are immediately reduced, and the first frequency after the reduction is always lower than the second frequency, so as to reduce the power consumption of the sensor, the processor and the wireless module as a whole, thereby prolonging the battery life under the condition of limited power, while maintaining a relatively higher second frequency to maintain the minimum monitoring capability for potential abnormal conditions.

[0068] Embodiment Two

[0069] Figure 2 A schematic diagram of an object abnormal behavior prompting method provided in the embodiment of the application is shown in the foregoing embodiment, and S130 and 140 are further refined, and the specific implementation manner can be referred to the technical scheme of the embodiment. The same or corresponding technical terms as in the above-mentioned embodiment are not described herein again.

[0070] As Figure 2As shown, the method specifically includes the following steps:

[0071] S210, collecting and acquiring device motion change data of the wearable device based on the first frequency.

[0072] The device motion change data includes at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope.

[0073] S220, when it is detected that the device motion change data meets the risk action detection condition, acquiring environment perception data, and collecting device motion change data and object physiological change data of the device wearing object based on the second frequency.

[0074] The object physiological change data includes at least one of a heart rate collected by a heart rate sensor and a skin conductance change rate collected by a skin conductance sensor, and the environment perception data includes environment audio data collected by an audio collection unit and environment image data collected by a shooting unit.

[0075] S230, if any of the following abnormal behavior detection conditions is met, it is determined that the device wearing object has an abnormal behavior state; the abnormal behavior detection conditions include: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s 2 ; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10μS; the environment audio data detects an abnormal audio segment; and the environment image data detects an abnormal image frame.

[0076] S240, determining abnormal level information corresponding to the abnormal behavior state based on the acceleration change rate, the angular velocity change rate, the skin conductance change rate, and / or the heart rate, and a plurality of preset thresholds corresponding to different preset abnormal levels.

[0077] Specifically, when at least one type or combined type data of the acceleration change rate, the angular velocity change rate, the heart rate, and the skin conductance change rate reaches a preset level determination standard, the corresponding abnormal level information can be determined.

[0078] On the basis of the above embodiments, optionally, the abnormal level information can include a low-risk abnormal level, a medium-risk abnormal level, and a high-risk abnormal level, and the specific implementation of S240 can include:

[0079] (1) If the acceleration change rate falls within a second acceleration change threshold range; and / or, the angular velocity change rate falls within a second angular velocity change threshold range; and / or, the skin conductance change rate falls within a second skin conductance change threshold range, it is determined that the abnormal level information corresponding to the abnormal behavior state is a low-risk abnormal level.

[0080] The second acceleration change threshold range refers to a pre-set acceleration change rate interval threshold for identifying a low-risk abnormal behavior state, and the numerical range is between the basic safety threshold and the medium-risk determination threshold. The second angular velocity change threshold range refers to a pre-set angular velocity change rate interval threshold for identifying a low-risk abnormal rotation action, and the lower limit of the numerical range is the upper limit of the normal activity range, and the upper limit is the medium-risk determination threshold. The second skin conductivity change threshold range is used to identify a low-risk autonomic nervous excitation state skin conductivity change interval threshold, and the numerical range is between the basic physiological response threshold and the medium-risk determination threshold.

[0081] Specifically, the acceleration change rate, the angular velocity change rate, and the skin conductivity change rate can be respectively matched with the corresponding second threshold range for interval matching detection. If the above interval matching results meet at least one of the following, the abnormal level information corresponding to the abnormal behavior state is determined to be a low-risk abnormal level. Specifically, it includes: the acceleration change rate falls within the second acceleration change threshold range; the angular velocity change rate falls within the second angular velocity change threshold range; the skin conductivity change rate falls within the second skin conductivity change threshold range.

[0082] (2) If the acceleration change rate is greater than the third acceleration change threshold; and / or, the angular velocity change rate is greater than the third angular velocity change threshold; and / or, the heart rate is greater than the second heart rate change threshold; and / or, the skin conductivity change rate is greater than the third skin conductivity change threshold, the abnormal level information corresponding to the abnormal behavior state is determined to be a medium-risk abnormal level.

[0083] The third acceleration change threshold and the third angular velocity change threshold are used to identify an abnormal motion amplitude of medium intensity, and the numerical value is higher than the upper limit of the low-risk range but lower than the high-risk threshold; the second heart rate change threshold and the third skin conductivity change threshold are used to determine a physiological stress reaction of medium degree. When the monitoring parameters exceed these thresholds but do not reach the high-risk standard, a medium-risk abnormal level response will be triggered.

[0084] Specifically, the acceleration change rate, the angular velocity change rate, the heart rate, and the skin conductivity change rate can be respectively compared with the corresponding medium-level threshold for comparison detection. If the above comparison results meet at least one of the following, the abnormal level information corresponding to the abnormal behavior state is determined to be a medium-risk abnormal level. Specifically, it includes: the acceleration change rate is greater than the third acceleration change threshold; the angular velocity change rate is greater than the third angular velocity change threshold; the heart rate is greater than the second heart rate change threshold; the skin conductivity change rate is greater than the third skin conductivity change threshold.

[0085] (3) if the acceleration change rate is greater than a fourth acceleration change threshold value; and / or, the angular velocity change rate is greater than a fourth angular velocity change threshold value; and / or, the heart rate is greater than a third heart rate change threshold value; and / or, the skin conductance change rate is greater than a fourth skin conductance rate change threshold value, then determining that the abnormal behavior state corresponds to high-risk abnormal level information.

[0086] The fourth acceleration and angular velocity change threshold values are used to identify a high-risk abnormal motion amplitude, and the values are significantly higher than the medium-risk threshold values; the third heart rate change threshold value and the fourth skin conductance rate change threshold value are used to determine a severe physiological stress state, and when the monitoring parameters exceed these threshold values, it indicates that the wearer may be in a high-risk or extremely stressful state, and an emergency response mechanism of the high-risk abnormal level will be triggered immediately.

[0087] Specifically, the acceleration change rate, the angular velocity change rate, the heart rate, and the skin conductance change rate can be compared with the corresponding high-level threshold values, respectively; if the comparison results satisfy at least one of the following conditions, it is determined that the abnormal behavior state corresponds to high-risk abnormal level information. Specifically, the acceleration change rate is greater than the fourth acceleration change threshold value; the angular velocity change rate is greater than the fourth angular velocity change threshold value; the heart rate is greater than the third heart rate change threshold value; and the skin conductance change rate is greater than the fourth skin conductance rate change threshold value.

[0088] S250, prompting a target user according to a target prompt mode corresponding to the abnormal level information.

[0089] In this embodiment, different abnormal level information corresponds to different target prompt modes. The target prompt mode can be determined according to the mapping relationship between different abnormal level information and corresponding prompt modes.

[0090] Optionally, determining the target prompt mode based on the abnormal level information comprises:

[0091] (1) if the abnormal level information is a low-risk abnormal level, the target prompt mode is to trigger a vibration prompt information of the wearable device.

[0092] The vibration prompt information refers to a specific vibration mode generated by the built-in motor of the wearable device, which is used to transmit a tactile warning signal of the low-risk abnormal state to the wearer.

[0093] In this embodiment, in the case of low-risk abnormal level, only the local prompt is executed by the built-in vibration motor of the wearable device, and the core feature is to use non-audio and light tactile feedback mode to deliver warning information to the device wearer through specific vibration patterns such as duration, interval frequency and intensity combination. In this way, it not only ensures that the abnormal state is reminded in time, but also avoids unnecessary environmental attention or interference with others, while reducing device energy consumption. This prompt mode is suitable for minor abnormal situations that do not require immediate external intervention.

[0094] (2) If the abnormal level information is a medium-risk abnormal level, the target prompt mode is to trigger the first voice prompt information of the wearable device, and to generate abnormal state record information based on environmental perception data and object position information, and to push the abnormal state record information to the terminal device of the corresponding target user in the first push mode.

[0095] Among them, the first voice prompt information refers to the pre-recorded voice warning content triggered for the medium-risk abnormal level, which is a voice alarm signal played through the device speaker and optimized in volume and tone. The object position information refers to the spatio-temporal data set representing the geographical coordinates of the device wearer obtained in real time by the built-in positioning module of the wearable device. The abnormal state record information refers to the structured log record containing multi-dimensional event data, which integrates the abnormal occurrence timestamp, motion characteristic parameter, physiological indicator data, environmental perception content and precise positioning information standardization data package. This information is packaged through a specific data format, retaining the traceability of the original sensor data while extracting key feature values, providing an intuitive panoramic description of abnormal events for the guardian of the device wearer, establishing a standardized data set for subsequent behavior analysis, supporting event backtracking, false alarm analysis and optimization, and is a key information carrier connecting real-time monitoring and subsequent disposal. The first push mode refers to the standardized information transmission protocol designed for the medium-risk abnormal level, which sends the abnormal state record information to the target user terminal through an asynchronous communication channel with regular priority.

[0096] In this embodiment, in the case of medium-risk abnormal level, dual warning measures are executed synchronously: first, the built-in voice module of the device plays the pre-recorded first voice prompt information, wherein the first voice prompt information can adopt moderate volume and gentle tone; at the same time, the abnormal state record information containing environmental audio clips, image snapshots and precise positioning data is structured and packaged, and transmitted to the preset guardian terminal through the priority data channel, forming a cooperative warning mode of local voice reminder combined with remote data push. This mechanism not only ensures the timely communication of abnormal information, but also avoids excessive interference with the wearer, and provides sufficient decision basis for the guardian through detailed event records, embodying the balanced disposal strategy for medium-level risk.

[0097] (3) If the abnormality level information is a high-risk abnormality level, the target prompting mode is to trigger second voice prompt information of the wearable device, generate abnormality state record information based on the environment perception data and the object position information, push the abnormality state record information to the terminal device of the corresponding target user in a second pushing mode, and trigger a preset event of dialing an emergency call phone to the terminal device.

[0098] Among them, the second voice prompt information refers to the enhanced voice warning content triggered by the wearable device for the high-risk abnormality level, which is a pre-recorded emergency warning voice played by the device speaker with high volume and urgent tone. This prompt adopts a repeated playing mode, and the content contains clear danger type description and emergency disposal guide. The second pushing mode refers to the emergency information transmission protocol designed for the high-risk abnormality level, which realizes the instant and reliable transmission mechanism of the abnormality state record information through the high-priority communication link.

[0099] Specifically, in the case of abnormality level information being a high-risk abnormality level, first, the emergency voice warning is played in a high-intensity mode by the device speaker, i.e. the second voice prompt information; at the same time, the abnormality state record information containing the environmental audio segment, image snapshot and precise positioning data is structured and encapsulated; then the second pushing protocol with the highest priority is started to ensure that the record information is delivered to the guardian terminal instantly through the redundant communication channel; the preset emergency call process is triggered synchronously to automatically dial the guardian phone to establish a voice communication link, forming a three-way guarantee system of device-end strong reminder, data emergency pushing and real-time voice contact, ensuring the reliability of information transmission and the timeliness of disposal response in high-risk situations.

[0100] The technical scheme of the embodiment of the present application determines whether the acceleration change rate is greater than a first acceleration change threshold value, and / or whether the angular velocity change rate is greater than a first angular velocity change threshold value, and / or whether the heart rate is greater than a first heart rate change threshold value, and / or whether the skin conductance change rate is greater than a first skin conductance rate change threshold value, and / or whether the environmental audio data detects an abnormal audio segment, and / or whether the environmental image data detects an abnormal image screen, if so, it is determined that the device wearing object has an abnormal behavior state, through multi-dimensional data detection and logical correlation analysis of motion parameters, physiological indicators and environmental perception data, not only the comprehensiveness of abnormal behavior recognition is improved, ensuring that complex risk events such as falling and coercion can be effectively captured, but also the probability of single sensor misjudgment is greatly reduced through multi-source data cross verification. In determining the target prompt mode corresponding to the abnormal behavior state, the abnormal level information corresponding to the abnormal behavior state can be determined based on the device motion change data, and / or the object physiological change data and the corresponding second preset condition; and the target prompt mode is determined based on the abnormal level information, through intelligent matching of motion characteristics, physiological response and the second preset condition, precise quantitative evaluation of the severity of abnormal behavior is realized, the most suitable warning strategy can be automatically selected according to the risk level, not only ensuring rapid and powerful response under high-risk conditions, but also avoiding excessive interference under low-risk scenarios.

[0101] Embodiment three

[0102] Figure 3 A structure schematic diagram of an object abnormal behavior prompt device provided by the embodiment of the present application, the device comprises: an automatic adsorption response module 310, a reference line determination module 320, an offset information determination module 330 and an object abnormal behavior prompt module 340.

[0103] The first data acquisition module 310 is configured to acquire and obtain device motion change data of the wearable device based on a first frequency; wherein the device motion change data comprises at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope.

[0104] The second data acquisition module 320 is configured to acquire environmental perception data when it is detected that the device motion change data satisfies a risk action detection condition, and acquire the device motion change data and object physiological change data of the device wearing object based on a second frequency; wherein the object physiological change data comprises at least one of a heart rate collected by a heart rate sensor and a skin conductance change rate collected by a skin electrical response sensor, and the environmental perception data comprises environmental audio data collected by an audio acquisition unit and environmental image data collected by a shooting unit.

[0105] The abnormal behavior detection module 330 is configured to determine that the device wearing object has an abnormal behavior state if any of the following abnormal behavior detection conditions is met: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10 μS; the environmental audio data detects an abnormal audio segment; and the environmental image data detects an abnormal image frame. 2

[0106] The abnormal behavior prompting module 340 is configured to cooperatively detect the multi-modal sensor data, determine abnormal level information corresponding to the abnormal behavior state, and prompt a target user based on a target prompting manner corresponding to the abnormal level information.

[0107] The threshold values in the risk action detection condition and the abnormal behavior detection condition are periodically updated based on historical device motion change data and historical object physiological change data of the device wearing object.

[0108] The technical scheme of the embodiment of the present application is based on first frequency acquisition and acquisition of device motion change data of the wearable device, which includes at least one of the acceleration change rate collected by the acceleration sensor and the angular velocity change rate collected by the gyroscope; thereby, when it is detected that the device motion change data meets the risk action detection condition, the environmental perception data is acquired, and the device motion change data and the object physiological change data of the device wearing object are acquired based on second frequency; wherein the object physiological change data includes at least one of the heart rate collected by the heart rate sensor and the skin conductance change rate collected by the skin conductance sensor, the environmental perception data includes the environmental audio data collected by the audio acquisition unit and the environmental image data collected by the shooting unit, further, if any of the following abnormal behavior detection conditions is met, it is determined that the device wearing object has an abnormal behavior state; the abnormal behavior detection condition includes: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s 2 ​a rate of change of the heart rate is greater than 20%; a rate of change of the skin conductance is greater than 10 μS; the environmental audio data detects an abnormal audio segment; the environmental image data detects an abnormal image frame; and finally, the multi-modal sensor data is cooperatively detected to determine abnormal level information corresponding to the abnormal behavior state, so as to prompt a corresponding target user based on a target prompt mode corresponding to the abnormal level information. The technical scheme provided in the embodiment can reach a detection accuracy of 95% or more for emergency help, with a false alarm rate of less than 5%, and a system response time of less than 2 seconds, so as to ensure that a guardian quickly learns of an emergency; the scheme combines dynamic monitoring of acceleration and angular velocity motion parameters with physiological index analysis of heart rate and skin conductance, to achieve active identification of an abnormal state of a human body; at the same time, cross verification of environmental audio and image data significantly reduces the false alarm rate, and a determination strategy including a grading warning mechanism is adopted, i.e., a graded response according to an emergency level, to significantly improve the warning efficiency, and effectively overcome the response lag problem caused by manual triggering of alarm information and single-dimension abnormal detection of a traditional wearable device, to improve the accuracy of object risk behavior determination and the timeliness of emergency response.

[0109] On the basis of the above device, optionally, the second data acquisition module 320 is specifically configured to trigger the audio acquisition unit and the shooting unit of the wearable device when it is detected that the acceleration rate of change is greater than a preset acceleration change threshold and / or the angular velocity rate of change is greater than a preset angular velocity change threshold; the environmental audio data is acquired through the audio acquisition unit, and the environmental image data is acquired through the shooting unit; the acceleration sensor and / or the gyroscope are controlled to acquire the device motion change data based on a second frequency, and the heart rate sensor and / or the skin conductance sensor are controlled to acquire the object physiological change data of the device wearing object based on the second frequency.

[0110] On the basis of the above device, optionally, the abnormal behavior prompt module 340 includes:

[0111] The abnormal level determination unit is configured to determine abnormal level information corresponding to the abnormal behavior state based on the acceleration rate of change, the angular velocity rate of change, the skin conductance rate of change, and / or the heart rate and multi-modal preset thresholds corresponding to different preset abnormal levels.

[0112] On the basis of the above device, optionally, the abnormal level determination unit is specifically configured to: if the acceleration change rate falls within a second acceleration change threshold range; and / or, the angular velocity change rate falls within a second angular velocity change threshold range; and / or, the skin conductance change rate falls within a second skin conductance change threshold range, determine that the abnormal behavior state corresponds to low-risk abnormal level information; if the acceleration change rate is greater than a third acceleration change threshold; and / or, the angular velocity change rate is greater than a third angular velocity change threshold; and / or, the heart rate is greater than a second heart rate change threshold; and / or, the skin conductance change rate is greater than a third skin conductance change threshold, determine that the abnormal behavior state corresponds to medium-risk abnormal level information; and / or, if the acceleration change rate is greater than a fourth acceleration change threshold; and / or, the angular velocity change rate is greater than a fourth angular velocity change threshold; and / or, the heart rate is greater than a third heart rate change threshold; and / or, the skin conductance change rate is greater than a fourth skin conductance change threshold, determine that the abnormal behavior state corresponds to high-risk abnormal level information.

[0113] On the basis of the above device, optionally, the prompt mode determination unit is specifically configured to: if the abnormal level information is low-risk abnormal level information, the target prompt mode is to trigger vibration prompt information of the wearable device; if the abnormal level information is medium-risk abnormal level information, the target prompt mode is to trigger first voice prompt information of the wearable device, generate abnormal state record information based on the environment perception data and object position information, and push the abnormal state record information to a terminal device of a corresponding target user in a first push mode; if the abnormal level information is high-risk abnormal level information, the target prompt mode is to trigger second voice prompt information of the wearable device, generate abnormal state record information based on the environment perception data and object position information, push the abnormal state record information to a terminal device of a corresponding target user in a second push mode, and trigger a preset event of dialing an emergency call phone to the terminal device; and prompt the corresponding target user based on the target prompt mode.

[0114] On the basis of the above device, optionally, the object abnormal behavior prompting device further comprises a threshold updating module.

[0115] The threshold updating module is configured to periodically acquire historical device motion change data and historical object physiological change data of the device wearing object based on a preset timing task; adjust a device motion change threshold in the risk action detection condition and the abnormal behavior detection condition based on the historical device motion change data, and adjust an object physiological change threshold in the risk action detection condition and the abnormal behavior detection condition based on the historical object physiological change data.

[0116] On the basis of the above device, optionally, the object abnormal behavior prompting device further comprises a low-power consumption control module.

[0117] The low-power consumption control module is configured to reduce the first frequency and the second frequency when detecting that the battery power of the wearable device is less than a preset power threshold, the first frequency being less than the second frequency.

[0118] The object abnormal behavior prompting device provided by the embodiment of the present application can execute the object abnormal behavior prompting method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0119] It is worth noting that the units and modules included in the above system are only divided according to the function logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for the convenience of mutual differentiation, and do not limit the protection scope of the embodiment of the present application.

[0120] Embodiment four

[0121] Figure 4 A structural schematic diagram of an electronic device provided by the embodiment of the present application. Figure 4 The electronic device 40 shown is merely an example, and should not bring any limitation to the function and use range of the embodiment of the present application.

[0122] As shown in Figure 4 , the electronic device 40 is in the form of a general-purpose computing device. The components of the electronic device 40 can include but are not limited to one or more processors or processing units 401, system memory 402, and bus 403 connecting different system components, including system memory 402 and processing unit 401.

[0123] The bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or local bus using any of a variety of bus structures. For example, these architectures include but are not limited to industry standard architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video electronics standards association (VESA) local bus, and peripheral component interconnect (PCI) bus.

[0124] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0125] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0126] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 407 typically perform the functions and / or methods described in the embodiments of the present invention.

[0127] Electronic device 40 can also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 810, etc.), and with one or more devices that enable a user to interact with electronic device 40, and / or with any device that enables electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 411. Furthermore, electronic device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 412. As shown, network adapter 412 communicates with other modules of electronic device 40 via bus 403. It should be understood that, although... Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0128] The processing unit 401 performs various functional applications and page processing by running programs stored in the system memory 402, such as implementing the object abnormal behavior prompting method provided by the embodiments of the present application.

[0129] In particular, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments of the present application include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the I / O interface 411, or installed from the storage system 406. When the computer program is executed by the processing unit 401, the above-mentioned functions defined in the method of the embodiments of the present application are performed.

[0130] Embodiment Five

[0131] The embodiments of the present application also provide a storage medium containing computer executable instructions for performing an object abnormal behavior prompting method when executed by a computer processor, applied to a wearable device, comprising:

[0132] acquiring and obtaining device motion change data of the wearable device based on a first frequency; wherein the device motion change data includes at least one of an acceleration change rate collected by an acceleration sensor and an angular velocity change rate collected by a gyroscope;

[0133] When it is detected that the device motion change data meets a risk action detection condition, acquiring environment perception data, and acquiring the device motion change data and object physiological change data of a device wearing object based on a second frequency; wherein the object physiological change data includes at least one of a heart rate collected by a heart rate sensor and a skin conductance change rate collected by a skin conductance response sensor, and the environment perception data includes environment audio data collected by an audio collection unit and environment image data collected by a shooting unit;

[0134] If any of the following abnormal behavior detection conditions is met, it is determined that the device wearing object has an abnormal behavior state; the abnormal behavior detection conditions include: the acceleration change rate is greater than 3g / s; the angular velocity change rate is greater than 150° / s 2 ; the heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10μS; the environment audio data detects an abnormal audio segment; and the environment image data detects an abnormal image frame;

[0135] The multi-modal sensor data is cooperatively detected to determine abnormal level information corresponding to the abnormal behavior state, so as to prompt a target user in a target prompting mode corresponding to the abnormal level information;

[0136] The threshold values in the risk action detection condition and the abnormal behavior detection condition are periodically updated based on historical device motion change data and historical object physiological change data of the device wearing object.

[0137] The computer storage medium of the embodiment of the application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component.

[0138] The computer readable signal medium can include a data signal propagating in a baseband or as part of a carrier wave propagating through a transmission medium, in which the computer readable program code is carried. Such a propagating data signal can take on many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can transmit, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or component.

[0139] The program code contained on the computer readable medium can be transmitted in any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0140] Computer program code for carrying out operations of embodiments of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0141] It is to be understood that the above description is directed to the preferred embodiments and that those skilled in the art will be able to devise various modifications which, although not specifically described herein, embody the principles of the application and are included within the spirit and scope of the application. Accordingly, while the preferred embodiments have been described above, those skilled in the art will understand that they are not to be limited to the preferred embodiments, but are to include all such embodiments falling within the scope of the application as defined by the appended claims.

Claims

1. A method for indicating abnormal object behavior, applied to wearable devices, characterized in that, include: The device motion change data of the wearable device is collected and acquired based on a first frequency; wherein, the device motion change data includes at least one of the acceleration change rate collected by the accelerometer and the angular velocity change rate collected by the gyroscope; When the device motion change data is detected to meet the risk action detection conditions, environmental perception data is acquired, and the device motion change data and the physiological change data of the device wearer are collected based on the second frequency; wherein, the physiological change data of the wearer includes at least one of heart rate collected by heart rate sensor and skin conductance change rate collected by skin conductance response sensor, and the environmental perception data includes environmental audio data collected by audio acquisition unit and environmental image data collected by imaging unit; If any of the following abnormal behavior detection conditions are met, it is determined that the device wearer is exhibiting abnormal behavior; the abnormal behavior detection conditions include: the rate of change of acceleration is greater than 3 g / s; the rate of change of angular velocity is greater than 150° / s. 2 The heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10 μS; abnormal audio segments are detected in the environmental audio data; abnormal image frames are detected in the environmental image data. Multimodal sensor data is collaboratively detected to determine the anomaly level information corresponding to the abnormal behavior state, and the corresponding target user is prompted based on the target prompt method corresponding to the anomaly level information. The thresholds in the risk action detection conditions and the abnormal behavior detection conditions are periodically updated based on the historical device movement change data and historical physiological change data of the device wearer.

2. The method according to claim 1, characterized in that, When the detected device motion change data meets the risk action detection conditions, environmental perception data is acquired, and based on a second frequency, the device motion change data and the physiological change data of the device wearer are collected, including: When the rate of change of acceleration is detected to be greater than a preset acceleration change threshold, and / or the rate of change of angular velocity is greater than a preset angular velocity change threshold, the audio acquisition unit and the shooting unit of the wearable device are triggered to be activated. The environmental audio data is acquired by the audio acquisition unit, and the environmental image data is acquired by the shooting unit; The accelerometer and / or the gyroscope are controlled to collect motion change data of the device based on a second frequency, and the heart rate sensor and / or the skin conductance sensor are controlled to collect physiological change data of the object and the wearer of the device based on a second frequency.

3. The method according to claim 1, characterized in that, The multimodal sensor data includes: rate of change of acceleration, rate of change of angular velocity, rate of change of skin conductance, and heart rate. The collaborative detection of the multimodal sensor data to determine the abnormality level information corresponding to the abnormal behavioral state includes: Based on the acceleration change rate, angular velocity change rate, skin conductance change rate and / or heart rate, as well as multimodal preset thresholds corresponding to different preset abnormality levels, the abnormality level information corresponding to the abnormal behavior state is determined.

4. The method according to claim 3, characterized in that, The process of determining the abnormality level information corresponding to the abnormal behavior state based on the rate of change of acceleration, rate of change of angular velocity, rate of change of skin conductance, and / or heart rate, as well as multimodal preset thresholds corresponding to different preset abnormality levels, includes: If the rate of change of acceleration falls within the second acceleration change threshold range; and / or, the rate of change of angular velocity falls within the second angular velocity change threshold range; and / or, the rate of change of skin conductivity falls within the second skin conductivity change threshold range, then the abnormality level information corresponding to the abnormal behavior state is determined to be a low-risk abnormality level. If the rate of change of acceleration is greater than the third acceleration change threshold; and / or, the rate of change of angular velocity is greater than the third angular velocity change threshold; and / or, the heart rate is greater than the second heart rate change threshold; and / or, the rate of change of skin conductivity is greater than the third skin conductivity change threshold, then the abnormality level information corresponding to the abnormal behavior state is determined to be a medium-risk abnormality level. If the rate of change of acceleration is greater than the fourth acceleration change threshold; and / or, the rate of change of angular velocity is greater than the fourth angular velocity change threshold; and / or, the heart rate is greater than the third heart rate change threshold; and / or, the rate of change of skin conductivity is greater than the fourth skin conductivity change threshold, then the abnormality level information corresponding to the abnormal behavior state is determined to be a high-risk abnormality level.

5. The method according to claim 1, characterized in that, The method of prompting the corresponding target user based on the anomaly level information includes: If the anomaly level information is a low-risk anomaly level, the target prompt method is to trigger a vibration prompt message from the wearable device; If the anomaly level information is a medium-risk anomaly level, the target prompt method is to trigger the first voice prompt information of the wearable device, and generate an anomaly status record information based on the environmental perception data and object location information, and push the anomaly status record information to the terminal device of the corresponding target user in the first push method. If the abnormality level information is a high-risk abnormality level, the target prompt method is to trigger the second voice prompt information of the wearable device, and generate abnormal status record information based on the environmental perception data and object location information. The abnormal status record information is pushed to the terminal device of the corresponding target user in the second push method, and a preset event is triggered to make an emergency call to the terminal device. The system provides targeted prompts to the appropriate target users.

6. The method according to claim 1, characterized in that, The method further includes: Based on a preset timed task, historical device movement change data and historical physiological change data of the device wearer are periodically acquired; The threshold for device motion change in the risk action detection condition and the abnormal behavior detection condition is adjusted based on the historical device motion change data, and the threshold for object physiological change in the risk action detection condition and the abnormal behavior detection condition is adjusted based on the historical object physiological change data.

7. The method according to claim 1, characterized in that, The method further includes: When the battery level of the wearable device is detected to be lower than a preset power threshold, the first frequency and the second frequency are reduced, with the first frequency being lower than the second frequency.

8. An abnormal behavior alert device for an object, applied to a wearable device, characterized in that, include: The first data acquisition module is used to collect and acquire device motion change data of the wearable device based on a first frequency; wherein, the device motion change data includes at least one of the acceleration change rate collected by the accelerometer and the angular velocity change rate collected by the gyroscope; The second data acquisition module is used to acquire environmental perception data when the detected device motion change data meets the risk action detection conditions, and to collect the device motion change data and the object physiological change data of the device wearer based on the second frequency; wherein, the object physiological change data includes at least one of heart rate collected by heart rate sensor and skin conductance change rate collected by skin conductance response sensor, and the environmental perception data includes environmental audio data collected by audio acquisition unit and environmental image data collected by imaging unit; An abnormal behavior detection module is used to determine that the device wearer is exhibiting abnormal behavior if any of the following abnormal behavior detection conditions are met: the rate of change of acceleration is greater than 3 g / s; the rate of change of angular velocity is greater than 150° / s. 2 The heart rate change rate is greater than 20%; the skin conductance change rate is greater than 10 μS; abnormal audio segments are detected in the environmental audio data; abnormal image frames are detected in the environmental image data. An abnormal behavior alert module is used to collaboratively detect multimodal sensor data, determine the abnormality level information corresponding to the abnormal behavior state, and alert the corresponding target user based on the target alert method corresponding to the abnormality level information. The thresholds in the risk action detection conditions and the abnormal behavior detection conditions are periodically updated based on the historical device movement change data and historical physiological change data of the device wearer.

9. A wearable device, characterized in that, The wearable device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the object abnormal behavior prompting method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the object abnormal behavior prompting method according to any one of claims 1-7.

Images