Unintentional injury monitoring method, wearable device, electronic device, and storage medium

Abstract

This application discloses an accidental injury monitoring method, wearable device, electronic device, and storage medium, relating to the field of smart wearable technology. The method includes: acquiring video and acceleration data collected by the wearable device; determining the wearer's scene and behavioral state, and the spatial position of the wearable device, based on the video; determining the motion state of the wearable device based on the acceleration data; and determining the wearer's accidental injury monitoring result based on the wearer's scene and behavioral state, and the spatial position and motion state of the wearable device. The method and device disclosed in this application improve the accuracy of accidental injury monitoring and enhance the user experience of the wearable device.

Description

Technical Field

[0001] This application relates to the field of smart wearable technology, and more specifically, to an accidental injury monitoring method, wearable device, electronic device, and storage medium. Background Technology

[0002] With the rapid development of smart wearable technology, more and more users are using wearable devices in office or daily life settings, and using them to assist in the detection of accidental injuries.

[0003] Accidental falls and traffic accidents are frequent occurrences that can cause personal injury to users. If users do not receive timely treatment, their lives may be in danger. Summary of the Invention

[0004] Firstly, this application provides an accidental injury monitoring method applied to a wearable device, comprising:

[0005] Acquire video and acceleration data collected by the wearable device;

[0006] Based on the video, the scene and behavioral state of the wearer of the wearable device, as well as the spatial location of the wearable device, are determined;

[0007] Based on the acceleration data, the motion state of the wearable device is determined;

[0008] Based on the wearer's location and behavior, as well as the spatial position and movement of the wearable device, the accidental injury monitoring results of the wearer are determined.

[0009] In some embodiments, the method further includes:

[0010] Acquire the vital signs signals of the wearer collected by the wearable device;

[0011] The determination of the wearer's accidental injury monitoring results includes:

[0012] The disconnection status of the wearable device is determined based on the wearer's vital signs.

[0013] The accidental injury monitoring results are confirmed based on the detachment status of the wearable device.

[0014] In some embodiments, determining the scene and behavioral state of the wearer of the wearable device, as well as the spatial location of the wearable device, based on the video, includes:

[0015] The video is input into a first neural network to obtain the scene and behavioral state of the wearer as output by the first neural network;

[0016] Based on the wearer's current situation and behavior, it is determined that the wearer is at risk of accidental injury.

[0017] The video is input into a second neural network to obtain the spatial location of the wearable device as output by the second neural network.

[0018] In some embodiments, the first neural network is trained based on the following steps:

[0019] Acquire sample videos and sample acceleration data;

[0020] Determine the scene tags corresponding to the sample videos;

[0021] Based on the sample acceleration data, the shaking frequency and shaking amplitude of the wearable device are determined;

[0022] Based on the shaking frequency and shaking amplitude of the wearable device, the behavioral state label corresponding to the sample video is determined;

[0023] The sample videos are labeled based on the scene labels and behavior state labels, and the first neural network is trained based on the labeled sample videos.

[0024] In some embodiments, the first neural network is an attention-based neural network; the second neural network is a ResNet network or a MobileNet network.

[0025] In some embodiments, the accidental injury monitoring results include fall alarms and severity levels.

[0026] In some embodiments, the method further includes:

[0027] The wearer was determined to have fallen based on the accidental injury monitoring results.

[0028] Continuously send fall alarm signals and increase the urgency of the fall alarm signals based on the severity level;

[0029] Confirm that no alarm clearing command has been received within the preset time period;

[0030] An alarm message is generated based on the accidental injury monitoring results, and the alarm message is sent to the communication terminal corresponding to the emergency contact and / or the communication terminal corresponding to the emergency rescue organization.

[0031] In some embodiments, determining the wearer's accidental injury monitoring result based on the wearer's scene and behavioral state, as well as the spatial location and movement state of the wearable device, includes:

[0032] The wearer's current situation is matched with multiple preset scenarios, and the injury risk assessment value corresponding to the matched preset scenario is used as the wearer's first injury risk assessment value.

[0033] The wearer's behavioral state is matched with multiple preset behavioral states, and the injury risk assessment value corresponding to the matched preset behavioral state is used as the wearer's second injury risk assessment value.

[0034] The spatial location of the wearable device is matched with multiple preset spatial locations, and the injury probability assessment value corresponding to the matched preset spatial location is used as the first injury probability assessment value of the wearer.

[0035] The motion state of the wearable device is matched with multiple preset motion states, and the injury probability assessment value corresponding to the matched preset motion state is used as the second injury probability assessment value of the wearer.

[0036] Based on the first injury risk assessment value, the second injury risk assessment value, the first injury probability assessment value, and the second injury probability assessment value, the wearer's accidental injury assessment value is determined;

[0037] The accidental injury assessment value is determined to be greater than a preset threshold;

[0038] The wearer's accidental injury monitoring result was determined to be a fall.

[0039] Secondly, this application provides a wearable device, including a device body, and a camera, gyroscope and processor disposed in the device body;

[0040] The camera is used to capture video of the scene in which the wearer is located;

[0041] The gyroscope is used to collect acceleration data from the wearable device;

[0042] The processor, connected to the camera and the gyroscope, is used to determine the wearer's scene and behavioral state, as well as the spatial position of the wearable device, based on the video; to determine the motion state of the wearable device based on the acceleration data; and to determine the wearer's accidental injury monitoring results based on the wearer's scene and behavioral state, as well as the spatial position and motion state of the wearable device.

[0043] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the accidental injury monitoring method as described above.

[0044] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the accidental injury monitoring method as described above. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating an accidental injury monitoring method provided in one embodiment of this application;

[0047] Figure 2 This is a schematic diagram of the structure of a wearable device provided in one embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the structure of smart glasses provided in one embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in this application, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0052] In existing technologies, users can use wearable devices to monitor for accidental injuries such as falls or traffic accidents. These wearable devices primarily use built-in accelerometers to sense the user's behavior. While monitoring the user's acceleration while wearing the device helps determine if a fall or collision has occurred, this monitoring method suffers from false alarms, has low accuracy, and fails to provide richer information.

[0053] Therefore, this application provides an accidental injury monitoring method, wearable device, electronic device, and storage medium, which can accurately determine whether the wearer has suffered an accidental injury and can provide the type and scenario of the injury when the wearer is injured, thereby improving the protection of the wearer's personal health.

[0054] Figure 1 This is a flowchart illustrating an accidental injury monitoring method provided in one embodiment of this application, as shown below. Figure 1 As shown, the method includes steps 110, 120, 130, and 140. These method steps are merely one possible implementation of this application.

[0055] Step 110: Acquire video and acceleration data collected by the wearable device.

[0056] Specifically, the accidental injury monitoring method provided in this application is applied to wearable devices. Wearable devices refer to everyday wearable devices with intelligent designs, including smart glasses, smartwatches, smart bracelets, smart helmets, and smart headphones. Among them, smart glasses include virtual reality (VR) glasses, augmented reality (AR) glasses, and mixed reality (MR) glasses.

[0057] The wearer is the user of the wearable device. The video captured by the wearable device is video footage of the scene in which the wearer is located. The video records environmental information of the scene in which the wearer is located, such as pedestrians, vehicles, buildings, or signs around the wearer. This information can be used to indicate the wearer's location or status in the scene.

[0058] Wearable devices can be equipped with cameras for video capture. For example, smart glasses have cameras. When a wearer is outdoors wearing the smart glasses, the camera can capture images of the surrounding environment from the wearer's perspective, thus obtaining video.

[0059] Wearable devices can also be equipped with accelerometers that measure acceleration data. For example, an accelerometer could be a 3-axis gyroscope or an inertial measurement unit. Accelerometer data describes the motion acceleration of the wearable device.

[0060] When a wearer suffers an accidental injury, their body posture will inevitably change, which will in turn cause a change in the posture of the worn parts. This change can be detected by the accelerometer in the wearable device and reflected through acceleration data.

[0061] Step 120: Based on the video, determine the scene and behavior of the wearer of the wearable device, as well as the spatial location of the wearable device.

[0062] Specifically, the wearer's scenario refers to the place or environment in which the wearer is wearing the device, such as streets, sports roads, indoor spaces, and driving roads. By identifying the landmarks appearing in the video and determining where these landmarks frequently appear, the possible scenarios in which the wearer might be located can be determined.

[0063] Wearer behavior refers to the wearer's actions while wearing the device, such as walking, running, and cycling. Video can be segmented into frames to obtain multiple consecutive images. The same object appearing in these images can be identified, and the wearer's behavior can be determined based on changes in the object's position and / or the number of pixels within the image.

[0064] The spatial location of a wearable device refers to its position in the three-dimensional space in which the wearer is located. Examples include the ground (very close to the ground with the face touching it), the ground itself (at a certain distance from the ground), the eyes facing upwards (looking up), and other locations (where the camera is obstructed or the spatial location cannot be distinguished from the image). The system can identify the ground or ground landmarks appearing in the video to determine whether the wearable device is on the ground.

[0065] Step 130: Determine the motion state of the wearable device based on acceleration data.

[0066] Specifically, the motion state of a wearable device refers to its movement, such as falling, rising, swaying left, and swaying right. When the motion state of the wearable device changes, it can be reflected through acceleration data.

[0067] A 3-axis gyroscope can be installed on wearable devices to monitor their acceleration. By monitoring the acceleration of the wearable device along the X, Y, and Z axes of the world coordinate system, the motion of the wearable device in these three directions can be determined, thus determining its motion state. For example, when the wearable device experiences a sudden downward acceleration along the Z-axis, it can be determined that the wearable device is falling, and the magnitude of this acceleration can be used to judge the speed of the fall.

[0068] Step 140: Based on the wearer's location and behavior, as well as the spatial position and movement of the wearable device, determine the wearer's accidental injury monitoring results.

[0069] Specifically, accidental injury monitoring results can include whether the wearer fell or not.

[0070] The wearer's environment and behavioral state can be used to assess the risk of accidental injury. For example, the risk of accidental injury is higher when the wearer is running on a road, and lower when the wearer is sleeping indoors.

[0071] The spatial position and motion of wearable devices can be used to assess the likelihood of accidental injury to the wearer. For example, if the wearable device is on the ground and has experienced falling acceleration, the wearer is more likely to suffer accidental injury.

[0072] The results of accidental injury monitoring for the wearer can be determined based on the wearer's environment and behavior, as well as the spatial location and movement of the wearable device.

[0073] For example, the risk of accidental injury varies depending on the context in which the wearer is in the situation; similarly, the risk also varies depending on the wearer's behavioral state. These two factors reflect the degree of risk of accidental injury from the wearer's perspective.

[0074] The likelihood of accidental injury to the wearable device varies depending on its spatial location and the state of motion it is in. These two factors reflect the potential for accidental injury from the perspective of the wearable device.

[0075] The above factors can be quantitatively analyzed, and the weighted sum of the quantitative values ​​of each factor can be used to reflect the accidental injury monitoring results. If the weighted sum is greater than a preset threshold, the accidental injury monitoring result can be determined as a fall; if the weighted sum is less than or equal to the preset threshold, the accidental injury monitoring result can be determined as no fall.

[0076] The results of accidental injury monitoring can be sent to emergency medical services or the wearer's family so that the wearer can receive assistance as soon as possible, thereby avoiding greater personal injury.

[0077] The accidental injury monitoring method provided in this application determines the wearer's scene and behavioral state, as well as the spatial position of the wearer, based on video collected by the wearer device; determines the wearer's motion state based on acceleration data collected by the wearer device; and determines the wearer's accidental injury monitoring result based on the wearer's scene and behavioral state, as well as the wearer's spatial position and motion state. By combining data collected by a camera and a gyroscope, it accurately determines whether the wearer has suffered an accidental injury, provides the scene at the time of injury, improves the rescue effect for the wearer, maximizes the protection of the wearer's personal health, and enhances the wearer's user experience with the wearer device.

[0078] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0079] In some embodiments, step 140 includes:

[0080] The wearer's current situation is matched with multiple preset scenarios, and the injury risk assessment value corresponding to the matched preset scenario is used as the wearer's first injury risk assessment value.

[0081] The wearer's behavioral state is matched with multiple preset behavioral states, and the injury risk assessment value corresponding to the matched preset behavioral state is used as the wearer's second injury risk assessment value.

[0082] The spatial location of the wearable device is matched with multiple preset spatial locations, and the injury probability assessment value corresponding to the matched preset spatial location is used as the wearer's first injury probability assessment value.

[0083] The wearable device's motion state is matched with multiple preset motion states, and the injury probability assessment value corresponding to the matched preset motion state is used as the wearer's second injury probability assessment value.

[0084] Based on the first injury risk assessment value, the second injury risk assessment value, the first injury probability assessment value, and the second injury probability assessment value, the wearer's accidental injury assessment value is determined;

[0085] The accidental injury assessment value is determined to be greater than a preset threshold;

[0086] The wearer's accidental injury monitoring result was determined to be a fall.

[0087] Specifically, the wearer's environment and behavioral state can be used to reflect the degree of risk of accidental injury to the wearer, and the specific level of risk can be represented by an injury risk assessment value. The higher the level of risk, the greater the injury risk assessment value.

[0088] For example, the risk of injury faced by wearers in outdoor settings is greater than the risk of injury faced in indoor settings, therefore the injury risk assessment value corresponding to the former is greater than the injury risk assessment value corresponding to the latter.

[0089] Multiple preset scenarios can be set, and the first injury risk assessment value (expressed as a percentage) of the wearer in each scenario can be represented, as shown in Table 1. The preset scenarios can be added or removed as needed.

[0090] Table 1. Injury risk assessment values ​​corresponding to the preset scenarios.

[0091] indoor 10％ garden 50％ street 80％ outdoor 90％

[0092] Similarly, multiple preset behavioral states can be set to represent the second injury risk assessment value of the wearer in each behavioral state, as shown in Table 2. The preset behavioral states can be added or removed as needed.

[0093] Table 2 Injury Risk Assessment Values ​​Corresponding to Preset Behavioral States

[0094] Sitting 10％ walk 30％ Cycling 50％ running 90％

[0095] The spatial position and movement of wearable devices can be used to reflect the likelihood of accidental injury to the wearer. The specific likelihood can be expressed using an injury probability assessment value. The higher the likelihood, the higher the injury probability assessment value.

[0096] For example, the likelihood of a wearer being accidentally injured when the device is on the ground is greater than the likelihood of a wearer being accidentally injured when the device is on the ground (at a certain distance from the ground). Therefore, the injury risk assessment value corresponding to the former is greater than the injury risk assessment value corresponding to the latter.

[0097] Multiple preset spatial positions can be set, and the first probability of injury to the wearer (expressed as a percentage) can be represented when the wearable device is in each spatial position, as shown in Table 3. The preset spatial positions can be added or removed as needed.

[0098] Table 3. Injury probability assessment values ​​corresponding to preset spatial locations.

[0099] ground 90％ ground 50％ Lie face up 80％ other 50％

[0100] Similarly, multiple preset motion states can be set, and the second injury probability assessment value (expressed as a percentage) of the wearer when the wearable device is in each motion state can be represented, as shown in Table 4. The preset spatial positions can be added or removed as needed.

[0101] Table 4. Injury probability assessment values ​​corresponding to preset exercise states.

[0102] Move up 10％ Move down 90％ Move to the left 50％ Move to the right 50％

[0103] Based on the first injury risk assessment value p1, the second injury risk assessment value p2, the first injury probability assessment value p3, and the second injury probability assessment value p4, the wearer's accidental injury assessment value P is determined, which can be expressed by the formula:

[0104] P = w1*p1 + w2*p1 + w3*p3 + w4*p4

[0105] Where w1 is the first calculation weight corresponding to the first injury risk assessment value, w2 is the second calculation weight corresponding to the second injury risk assessment value, w3 is the third calculation weight corresponding to the first injury probability assessment value, and w4 is the fourth calculation weight corresponding to the second injury probability assessment value. The initial value of the above four calculation weights can be set to 0.25.

[0106] The four calculation weights mentioned above are used to represent the degree of influence of each factor on the result, and can be adjusted according to the actual situation. For example, after multiple uses of the wearable device, if it is found that the motion state of the wearable device is more accurate in reflecting the wearer's accidental injury than its spatial position, the fourth calculation weight can be increased and the third calculation weight decreased.

[0107] The higher the accidental injury assessment value, the more likely the accidental injury monitoring result is a fall. A preset threshold can be set to measure whether the wearer has fallen. This preset threshold can be adjusted according to actual conditions. For example, the preset threshold can be set to 70%. When the accidental injury assessment value is greater than 70%, the accidental injury monitoring result can be considered a fall.

[0108] The wearer's environment and behavior, as well as the spatial location and movement of the wearable device, can be matched with their respective preset values ​​to determine the corresponding assessment values. Then, based on the calculation formula mentioned above, the accidental injury assessment value is obtained.

[0109] For example, if the monitored scene is a "park", the wearer's behavior is "sitting", the device's spatial location is "on the ground", and the device's movement is "moving to the left", then according to the above calculation formula, the accidental injury assessment value can be calculated as 40%. After comparing this with the preset threshold, it can be determined that the wearer's accidental injury monitoring result is that they did not fall.

[0110] For example, if the monitored scene is a "park", the wearer's behavior is "running", the device's location is "ground", and the device's movement is "moving downwards", then according to the above calculation formula, the accidental injury assessment value can be calculated to be 80%. After comparing this with the preset threshold, it can be determined that the wearer's accidental injury monitoring result is a fall.

[0111] The accidental injury monitoring method provided in this application provides a quantitative analysis of the factors contributing to the risk of accidental injury and the likelihood of injury to the wearer. Based on the weighted summation result, it accurately determines whether the wearer has suffered an accidental injury, thereby improving the wearer's user experience with the wearable device.

[0112] In some embodiments, the method further includes:

[0113] Acquire vital signs signals from wearable devices;

[0114] Determine the wearer's accidental injury monitoring results, including:

[0115] Determine the disconnection status of the wearable device based on the wearer's vital signs;

[0116] The results of accidental injury monitoring are confirmed based on the detachment status of wearable devices.

[0117] Specifically, wearable devices can also be equipped with vital sign sensors to acquire the wearer's vital sign signals.

[0118] For example, a vital signs sensor can be a photosensor. Since blood absorbs light at different wavelengths differently, wearable devices can be equipped with a light source. This light is reflected by the skin to the photosensor, which then uses a specific algorithm to count the wearer's heartbeat. If the photosensor cannot receive sufficient light from the emitted light source, or if the intensity of the reflected light received by the photosensor is inconsistent with the intensity of light reflected by the human skin, an abnormality in the vital signs signal can be determined, thus indicating that the wearable device has been detached.

[0119] If the wearable device is detached, it can be considered as an accidental injury, thus confirming the accidental injury monitoring results and improving their accuracy.

[0120] In some embodiments, step 120 includes:

[0121] The video is input into the first neural network to obtain the wearer's scene and behavioral state output by the first neural network;

[0122] Based on the wearer's situation and behavior, it is determined that the wearer is at risk of accidental injury.

[0123] The video is input into the second neural network to obtain the spatial location of the wearable device as output by the second neural network.

[0124] Specifically, the first neural network can be used to process the video to obtain the wearer's scene and behavioral state.

[0125] The first neural network can be pre-trained, specifically through the following training method: First, collect a large number of sample videos; second, label the scene and behavioral state of the wearer in each sample video to obtain a label for each sample video. Subsequently, train the first neural network using a large number of sample videos to reduce the difference between the first neural network's predicted values ​​and the true values ​​(labels) for the scene and behavioral state corresponding to the sample videos, thereby improving the first neural network's predictive ability for scene and behavioral state.

[0126] The sample videos can include not only those corresponding to accidental injuries but also those corresponding to safe conditions. The former can be used as positive samples, and the latter as negative samples. Using both positive and negative samples can improve the generalization ability of the first neural network and reduce the false detection and false recognition rates.

[0127] The first neural network can identify the wearer's environment and behavioral state, such as whether the wearer is jogging in a park or sitting on the sofa at home. This information can determine whether the wearer is at risk of accidental injury. Whether the wearer actually suffers accidental injury still needs to be determined in conjunction with the location of the wearable device. This requires analyzing the spatial position of the wearable device through video.

[0128] At this point, the second neural network can be used to process the video and obtain the spatial location of the wearable device.

[0129] Compared to the first deep learning model, the training samples for the second neural network are also sample videos, but the difference lies in the labels of the samples. The labels for the training samples of the second neural network include the spatial location of the wearable device in each sample video.

[0130] The training process for the second neural network can be similar to that of the first neural network, using labeled sample videos for training, which will not be elaborated here.

[0131] The accidental injury monitoring method provided in this application involves inputting video into a first neural network to obtain the wearer's location and behavioral state as output by the first neural network; and inputting video into a second neural network to obtain the spatial location of the wearable device as output by the second neural network. This method improves the utilization rate of video data and increases the accuracy of monitoring whether the wearer has suffered accidental injury.

[0132] In some embodiments, the first neural network is trained based on the following steps:

[0133] Acquire sample videos and sample acceleration data;

[0134] Determine the scene labels corresponding to the sample videos;

[0135] Based on sample acceleration data, determine the shaking frequency and shaking amplitude of the wearable device;

[0136] Based on the shaking frequency and amplitude of the wearable device, determine the behavioral state label corresponding to the sample video;

[0137] The sample videos are labeled based on scene labels and behavior state labels, and the first neural network is trained based on the labeled sample videos.

[0138] Specifically, a large amount of sample video and sample acceleration data can be collected. This sample video and sample acceleration data can be obtained through wearable devices of the same or different models.

[0139] Scenes in each sample video are identified to obtain scene labels for each sample video. For example, based on road signs appearing in a sample video, the scene label is determined to be a driving road.

[0140] By identifying the acceleration data of each sample, we can obtain the shaking frequency and amplitude of the wearable device as reflected by each sample acceleration. The shaking frequency refers to the speed at which the wearable device shakes back and forth or swings up and down. The shaking amplitude refers to the positional deviation of the wearable device during its back-and-forth shaking or up-and-down swinging.

[0141] The frequency and amplitude of the wearable device's shaking vary depending on the wearer's activity level. For example, when the wearer is walking, the device shakes at a slower frequency and with a smaller amplitude; when the wearer is running, the device shakes at a faster frequency and with a larger amplitude.

[0142] Therefore, the behavioral state label corresponding to the sample video can be determined by the shaking frequency and shaking amplitude of the wearable device.

[0143] The sample videos are labeled with scene labels and behavior state labels, and the first neural network is trained based on the labeled sample videos to improve the first neural network's ability to recognize scenes and behavior states.

[0144] In addition, the sample video can be segmented into frames, and the wearer's initial behavioral state can be determined based on the changes in the position of objects in each frame and / or the changes in the number of pixels. However, the accuracy of the initial behavioral state is relatively low. It is also necessary to comprehensively judge the initial behavioral state based on the shaking frequency and shaking amplitude of the wearable device obtained from the analysis of the sample acceleration data, and obtain the behavioral state label corresponding to the sample video.

[0145] For example, given a sample video, we know someone is sitting on a park bench, but we can't directly label the scene as "park" and the behavior state as "sitting." Instead, we need to analyze the wearer's behavior. Analysis reveals that the wearer sees someone sitting on a park bench, but the wearer might be walking, running, or cycling on a park path. Therefore, we can label the sample video as "park" and the initial behavior state as "walking," "running," or "cycling." Further analysis using acceleration data is needed to determine the wearer's behavior state as "running."

[0146] The accidental injury monitoring method provided in this application improves the accuracy of visual detection in monitoring whether a wearer has suffered an accidental injury by using sample acceleration data to perform auxiliary annotation on sample videos.

[0147] In some embodiments, the first neural network is an attention-based neural network; the second neural network is a ResNet network or a MobileNet network.

[0148] Specifically, the first neural network can be an attention-based neural network, such as the Transformer model.

[0149] The second neural network can employ networks such as ResNet and MobileNet. For example, the second neural network may include a backbone network and a softmax layer. The backbone network can be MobileNet. As a lightweight neural network, MobileNet requires fewer system resources and can run efficiently in wearable devices.

[0150] In some embodiments, accidental injury monitoring results include fall alarms and severity levels.

[0151] Specifically, the severity level varies depending on the scenario. For example, if a wearer falls accidentally and the video contains images of explosions, fire, water, or blood, the wearer may face secondary injuries. Therefore, it is possible to identify the severity level of a scenario while simultaneously recognizing the scenario itself.

[0152] The severity level measures the degree of danger posed to the wearer by the surrounding environment. The severity level can be customized by the wearer. For example, it can be set to two levels: severe and moderate.

[0153] The first neural network can be used to implement multi-label classification. For example, three fully connected layers can be set in the first neural network.

[0154] The first fully connected layer is used to classify the wearer's scene, the second fully connected layer is used to classify the wearer's behavioral state, and the third fully connected layer is used to classify the severity level. Accordingly, for the training samples of the first neural network, three labels need to be determined: the first label is the scene label, the second label is the behavioral state label, and the third label is the severity level label.

[0155] The accidental injury monitoring method provided in this application includes fall alarms and severity levels, which can provide richer information for the wearer's rescue, enabling the wearer to receive rescue as soon as possible, thereby improving the rescue effect for the wearer.

[0156] In some embodiments, the method further includes:

[0157] The wearer was determined to have fallen based on the results of accidental injury monitoring.

[0158] Continuously send fall alarm signals and increase the urgency of the fall alarm signals based on the severity level;

[0159] Confirm that no alarm clearing command has been received within the preset time period;

[0160] An alarm message is generated based on the accidental injury monitoring results and sent to the communication terminal corresponding to the emergency contact and / or the communication terminal corresponding to the emergency rescue organization.

[0161] Specifically, the fall alarm signal can be one or more of the following: a light signal, a vibration signal, or a voice signal.

[0162] The urgency level describes the urgency of an alarm signal and can be determined based on its severity. The higher the severity level, the greater the urgency, and the stronger the alarm signal, such as higher light intensity, stronger vibration frequency, or louder volume.

[0163] Alarm messages can be text messages containing the results of accidental injury monitoring.

[0164] When the wearer's accidental injury monitoring results indicate an accidental fall, a fall alarm signal will be continuously sent.

[0165] If the wearer is only slightly injured or uninjured, upon seeing, hearing, or feeling the fall alarm signal, they can speak, press a button, touch the display, or perform a preset gesture. Alternatively, if someone nearby provides assistance, the rescuer can also speak or perform corresponding actions. Upon receiving the wearer's or someone's action or speech, the wearable device will generate an alarm cancellation command and stop sending fall alarm signals.

[0166] If the wearer is seriously injured and unable to speak or move, and the wearable device does not receive an alarm cancellation command within a preset time, it will generate an alarm message based on the accidental injury monitoring results and send the alarm message to the communication terminal corresponding to the emergency contact. It can also send the alarm message to the communication terminal corresponding to the emergency contact and the communication terminal corresponding to the emergency rescue organization at the same time.

[0167] The accidental injury monitoring method provided in this application generates an alarm message and sends it to the communication terminal corresponding to the emergency contact and / or the communication terminal corresponding to the emergency rescue organization when no alarm cancellation instruction is received within a preset time period. This can promptly notify the emergency contact or emergency rescue organization to help the wearer, thereby improving the rescue effect for the wearer.

[0168] The wearable device provided in the embodiments of this application is described below. The wearable device described below can be referred to in correspondence with the accidental injury monitoring method described above.

[0169] Figure 2 This is a schematic diagram of the structure of a wearable device provided in one embodiment of this application, as shown below. Figure 2 As shown, the wearable device 200 includes a device body 210, and a camera 220, a gyroscope 240 and a processor 230 disposed in the device body;

[0170] Camera 220 is used to capture video of the scene in which the wearer is located;

[0171] Gyroscope 240 is used to collect acceleration data from wearable devices;

[0172] The processor 230, connected to the camera 220 and the gyroscope 240, is used to determine the wearer's scene and behavior state, as well as the spatial position of the wearable device, based on video; to determine the motion state of the wearable device based on acceleration data; and to determine the wearer's accidental injury monitoring results based on the wearer's scene and behavior state, as well as the spatial position and motion state of the wearable device.

[0173] The wearable device provided in this application determines the wearer's scene and behavioral state, as well as the device's spatial location, based on video collected by the device; it determines the device's motion state based on acceleration data collected by the device; and it determines the wearer's accidental injury monitoring results based on the wearer's scene and behavioral state, as well as the device's spatial location and motion state. By combining data collected by cameras and gyroscopes, it accurately determines whether the wearer has suffered accidental injury, provides the scene of the injury, improves the rescue effect, maximizes the protection of the wearer's health, and enhances the user experience of the device.

[0174] In some embodiments, the wearable device is smart glasses.

[0175] Specifically, Figure 3 This is a schematic diagram of the structure of smart glasses provided in one embodiment of this application, as shown below. Figure 3 As shown, the smart glasses include a glasses body 310, and a switch 320, a camera 330 and a touch screen 340 are provided on one temple of the glasses body 310.

[0176] The temples also house a gyroscope, microphone, voice player, vibration sensor, communication module, and positioning module. These smart glasses support voice input, gesture input, and touchscreen input.

[0177] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application, as shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communications bus 440. The processor 410 can call logical commands stored in the memory 430 to execute the above-described method, which includes:

[0178] Acquire video and acceleration data collected by wearable devices; based on the video, determine the wearer's scene and behavioral state, as well as the spatial location of the wearable device; based on the acceleration data, determine the motion state of the wearable device; based on the wearer's scene and behavioral state, as well as the spatial location and motion state of the wearable device, determine the wearer's accidental injury monitoring results.

[0179] Furthermore, when the logical commands in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0180] The processor in the electronic device provided in this application embodiment can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effect, which will not be repeated here.

[0181] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0182] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0183] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0184] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for monitoring accidental injuries, characterized in that, Applications in wearable devices, including: Acquire video and acceleration data collected by the wearable device; Based on the video, the scene and behavioral state of the wearer of the wearable device, as well as the spatial location of the wearable device, are determined; Based on the acceleration data, the motion state of the wearable device is determined; Based on the wearer's scene and behavior, as well as the spatial location and movement of the wearable device, the accidental injury monitoring results of the wearer are determined. The determination of the wearer's accidental injury monitoring results based on the wearer's scene and behavioral state, as well as the spatial position and movement state of the wearable device, includes: The wearer's current situation is matched with multiple preset scenarios, and the injury risk assessment value corresponding to the matched preset scenario is used as the wearer's first injury risk assessment value. The wearer's behavioral state is matched with multiple preset behavioral states, and the injury risk assessment value corresponding to the matched preset behavioral state is used as the wearer's second injury risk assessment value. The spatial location of the wearable device is matched with multiple preset spatial locations, and the injury probability assessment value corresponding to the matched preset spatial location is used as the first injury probability assessment value of the wearer. The motion state of the wearable device is matched with multiple preset motion states, and the injury probability assessment value corresponding to the matched preset motion state is used as the second injury probability assessment value of the wearer. Based on the first injury risk assessment value, the second injury risk assessment value, the first injury probability assessment value, and the second injury probability assessment value, the wearer's accidental injury assessment value is determined; The accidental injury assessment value is determined to be greater than a preset threshold; The wearer's accidental injury monitoring result was determined to be a fall; Determining the wearer's accidental injury assessment value includes: Based on the calculation weights corresponding to each risk value, the first injury risk assessment value, the second injury risk assessment value, the first injury probability assessment value, and the second injury probability assessment value are weighted and summed to determine the wearer's accidental injury assessment value.

2. The accidental injury monitoring method according to claim 1, characterized in that, The method further includes: Acquire the vital signs signals of the wearer collected by the wearable device; The determination of the wearer's accidental injury monitoring results includes: The disconnection status of the wearable device is determined based on the wearer's vital signs. The accidental injury monitoring results are confirmed based on the detachment status of the wearable device.

3. The accidental injury monitoring method according to claim 1, characterized in that, The step of determining the scene and behavioral state of the wearer of the wearable device, as well as the spatial location of the wearable device, based on the video, includes: The video is input into a first neural network to obtain the scene and behavioral state of the wearer as output by the first neural network; Based on the wearer's current situation and behavior, it is determined that the wearer is at risk of accidental injury. The video is input into a second neural network to obtain the spatial location of the wearable device as output by the second neural network.

4. The accidental injury monitoring method according to claim 3, characterized in that, The first neural network was trained based on the following steps: Acquire sample videos and sample acceleration data; Determine the scene tags corresponding to the sample videos; Based on the sample acceleration data, the shaking frequency and shaking amplitude of the wearable device are determined; Based on the shaking frequency and shaking amplitude of the wearable device, the behavioral state label corresponding to the sample video is determined; The sample videos are labeled based on the scene labels and behavior state labels, and the first neural network is trained based on the labeled sample videos.

5. The accidental injury monitoring method according to claim 3, characterized in that, The first neural network is an attention-based neural network; the second neural network is a ResNet network or a MobileNet network.

6. The accidental injury monitoring method according to any one of claims 1 to 5, characterized in that, The accidental injury monitoring results include fall alarms and severity levels.

7. The accidental injury monitoring method according to claim 6, characterized in that, The method further includes: The wearer was determined to have fallen based on the accidental injury monitoring results. Continuously send fall alarm signals and increase the urgency of the fall alarm signals based on the severity level; Confirm that no alarm clearing command has been received within the preset time period; An alarm message is generated based on the accidental injury monitoring results, and the alarm message is sent to the communication terminal corresponding to the emergency contact and / or the communication terminal corresponding to the emergency rescue organization.

8. A wearable device, characterized in that, This includes the device itself, as well as the camera, gyroscope, and processor installed within the device itself; The camera is used to capture video of the scene in which the wearer is located; The gyroscope is used to collect acceleration data from the wearable device; The processor, connected to the camera and the gyroscope, is used to determine the scene and behavioral state of the wearer, as well as the spatial position of the wearable device, based on the video; and to determine the motion state of the wearable device based on the acceleration data. And based on the wearer's scene and behavior, as well as the spatial position and movement of the wearable device, determine the wearer's accidental injury monitoring results; The determination of the wearer's accidental injury monitoring results based on the wearer's scene and behavioral state, as well as the spatial position and movement state of the wearable device, includes: The wearer's current situation is matched with multiple preset scenarios, and the injury risk assessment value corresponding to the matched preset scenario is used as the wearer's first injury risk assessment value. The wearer's behavioral state is matched with multiple preset behavioral states, and the injury risk assessment value corresponding to the matched preset behavioral state is used as the wearer's second injury risk assessment value. The spatial location of the wearable device is matched with multiple preset spatial locations, and the injury probability assessment value corresponding to the matched preset spatial location is used as the first injury probability assessment value of the wearer. The motion state of the wearable device is matched with multiple preset motion states, and the injury probability assessment value corresponding to the matched preset motion state is used as the second injury probability assessment value of the wearer. Based on the first injury risk assessment value, the second injury risk assessment value, the first injury probability assessment value, and the second injury probability assessment value, the wearer's accidental injury assessment value is determined; The accidental injury assessment value is determined to be greater than a preset threshold; The wearer's accidental injury monitoring result was determined to be a fall; Determining the wearer's accidental injury assessment value includes: Based on the calculation weights corresponding to each risk value, the first injury risk assessment value, the second injury risk assessment value, the first injury probability assessment value, and the second injury probability assessment value are weighted and summed to determine the wearer's accidental injury assessment value.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the accidental injury monitoring method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the accidental injury monitoring method as described in any one of claims 1 to 7.

Images