User-based fall prevention and detection method, computer equipment

Abstract

The embodiments of the present application relate to the field of user fall prevention and detection, and specifically to a method and computer device for prevention and detection of user falls. A specific implementation of the method includes: extracting a user profile map of each video frame image in a video frame image sequence; extracting the profile features of the corresponding target user according to the user profile map sequence; extracting the body part features of the corresponding target user according to the body part map sequence of the target user; fusing the profile features and body part features of the target user; splicing the profile features, body part features and body profile fusion features into the walking posture features of the target user; performing fall detection on the walking posture features to generate a fall detection result. This implementation can accurately and timely identify the walking posture of the current user, and determine whether the user is at risk of falling based on the walking posture.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of user fall prevention and detection, and specifically to a method and computer device for preventing and detecting user falls. Background Art

[0002] With the development of science and technology, more and more research institutions and industries have begun to study the behavior recognition of the elderly to prevent them from falling. At present, the common method for detecting and preventing falls in the elderly is to use wearable sensors for fall detection. However, this method usually has the following problems: if the wearing position is incorrect, there will be a large error in the detection result; if the elderly forget to wear it, the fall of the elderly cannot be detected in time, which may easily cause safety hazards. Summary of the invention

[0003] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0004] Some embodiments of the present application propose a method for preventing and detecting user falls, a computer device, and a computer-readable storage medium to solve one or more of the technical problems mentioned in the above background technology section.

[0005] In a first aspect, some embodiments of the present application provide a method for preventing and detecting user falls, the method comprising: in response to receiving a user fall event detection instruction, controlling an associated camera device to collect a monitoring video of a target user in a target area in real time; performing frame processing on the monitoring video to obtain a video frame image sequence; extracting a user contour image of each video frame image in the video frame image sequence to obtain a user contour image sequence; extracting contour features corresponding to the target user according to the user contour image sequence; extracting human body part features corresponding to the target user according to a human body part image sequence of the target user, wherein the human body part image sequence represents images of various parts of the human body of the target user during walking; fusing the contour features and human body part features of the target user to obtain human body contour fusion features; splicing the contour features, the human body part features and the human body contour fusion features into walking posture features of the target user; performing fall detection on the walking posture features to generate a fall detection result; in response to determining that the fall detection result meets a preset fall condition, controlling an associated alarm device to emit an alarm prompt sound.

[0006] In a second aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the method described in any implementation of the first aspect is implemented.

[0007] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0008] The above-mentioned embodiments of the present application have the following beneficial effects: through the prevention and detection method based on user falls of some embodiments of the present application, the walking posture of the current user can be accurately and timely identified, and according to the walking posture, it can be judged whether the user has the risk of falling. An alarm can be given in time to reduce safety hazards. First, in response to receiving a user fall event detection instruction, the associated camera device is controlled to collect the monitoring video of the target user in the target area in real time; the above-mentioned monitoring video is frame-processed to obtain a video frame image sequence. Thus, it is convenient to perform fall detection analysis according to the user's image. Then, the user contour map of each video frame image in the above-mentioned video frame image sequence is extracted to obtain a user contour map sequence; according to the above-mentioned user contour map sequence, the contour features corresponding to the above-mentioned target user are extracted; according to the above-mentioned human body part map sequence of the above-mentioned target user, the human body part features corresponding to the above-mentioned target user are extracted, wherein the above-mentioned human body part map sequence represents the map of each part of the human body of the target user during walking; the contour features of the above-mentioned target user and the human body part features are fused to obtain the human body contour fusion features. Thus, the fusion of the human body part map and the user contour map can improve the accuracy of the recognition of the user's walking posture. Then, the above-mentioned contour features, the above-mentioned human body part features and the above-mentioned human body contour fusion features are spliced ​​into the walking posture features of the above-mentioned target user; the above-mentioned walking posture features are subjected to fall detection to generate a fall detection result. In this way, the walking posture of the current user can be accurately and timely identified, and according to the walking posture, it is judged whether the user is at risk of falling. Finally, in response to determining that the above-mentioned fall detection result meets the preset fall condition, the associated alarm device is controlled to emit an alarm prompt sound. In this way, an alarm can be issued in a timely manner to reduce safety hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0010] Figure 1 is a flow chart of some embodiments of the method for preventing and detecting user falls according to the present application;

[0011] Figure 2 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0012] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0013] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0014] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0015] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0016] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0017] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0018] Figure 1 The flowchart 100 of some embodiments of the method for preventing and detecting user falls according to the present application is shown. The method for preventing and detecting user falls comprises the following steps:

[0019] Step 101, in response to receiving a user fall event detection instruction, controlling an associated camera device to collect a monitoring video of a target user in a target area in real time.

[0020] In some embodiments, the execution subject (e.g., a computing device) of the method for preventing and detecting user falls may control the associated camera device to collect monitoring video of the target user in the target area in real time in response to receiving a user fall event detection instruction. The target area may refer to an area that needs to be monitored for user falls. For example, the target area may refer to a room / living room where the user lives, or a public place. For example, monitoring video of the target area within a preset time period may be collected in real time. The preset time period may be 5-10 minutes. The preset time period should not be too long to facilitate timely detection of whether a user has fallen. The user fall event detection instruction may refer to an instruction to detect whether there is a dangerous event in the target area. Dangerous events may include, but are not limited to: user fall events, user falling to the ground. The camera device may be a camera. The target user is included in the monitoring video.

[0021] Step 102, performing frame processing on the above surveillance video to obtain a video frame image sequence.

[0022] In some embodiments, the execution subject may perform frame processing on the surveillance video to obtain a video frame image sequence. The surveillance video may be frame processed at a preset frame interval to obtain a video frame image sequence.

[0023] Step 103: extract the user profile image of each video frame image in the video frame image sequence to obtain a user profile image sequence.

[0024] In some embodiments, the above-mentioned execution entity can extract the user profile of each video frame image in the above-mentioned video frame image sequence to obtain a user profile sequence. The user profile sequence can be used to characterize the information of the human body contour of the target user during walking. The user profile sequence can be obtained by collecting a walking video of the target user, or it can be obtained by screening out video frame images of the target user in different walking forms from the walking video. For example, the profile of the target user can be extracted from each video frame image to obtain a user profile sequence. There is no limitation on the method of extracting the profile. For example, the method of extracting the profile can be: automatic contour extraction, image edge and contour extraction based on deep learning.

[0025] For example, automatic contour extraction: This method uses advanced image processing algorithms to automatically identify and extract contour lines in images without the need for users to draw them manually. It can accurately identify details and complex contours in images, ensuring that the extracted contour lines are of high quality and meet user needs. In addition, this technology also supports exporting the extracted contour lines as vector graphics files, such as SVG, DXF, etc.

[0026] Image edge and contour extraction based on deep learning: This method uses deep learning technology to predict edges and contours in images by training models. For example, the DeepEdge architecture uses the Canny edge detector to extract candidate contour points, and then extracts patches of different scales around each candidate point, and extracts high-level features through five convolutional layers of the pre-trained KNet.

[0027] Step 104: extracting profile features corresponding to the target user according to the user profile graph sequence.

[0028] In some embodiments, the execution entity may extract profile features corresponding to the target user based on the user profile graph sequence.

[0029] Here, the execution subject can identify each user profile in the user profile sequence, thereby extracting the profile features of the target user in the user profile. Then, the profile features in each user profile can be sorted according to the order of each user profile in the sequence to obtain the profile features of the target user walking. As an example, the execution subject can use a multi-layer convolutional neural network (such as an OpenGait network) as a human body profile encoder to extract profile features from the user profile sequence.

[0030] Step 105 , extracting the body part features corresponding to the target user according to the body part map sequence of the target user.

[0031] In some embodiments, the execution subject may extract the body part features corresponding to the target user according to the body part image sequence of the target user, wherein the body part image sequence represents images of various body parts of the target user during walking.

[0032] Here, the execution subject can identify and extract each human body part map in the human body part map sequence to obtain the human body part features of the target user in the human body part map. Then, the part features in each human body part map can be sorted according to the order of each human body part map in the sequence to obtain the part features of the target user's walking process. As an example, the execution subject can use a multi-layer convolutional neural network (such as OpenGait network) as a human body part encoder to extract the body part features from the human body part map sequence.

[0033] It should be noted that the human body part map sequence here can also be obtained by collecting surveillance videos. In addition, in order to ensure the alignment and fusion of subsequent feature data, the human body part map in the human body part map sequence and the human body part map in the user profile map sequence at the same time usually represent the same video frame image. The human body part map and the human body part profile map can be obtained by performing different processing on the same video frame image to represent different information.

[0034] In some embodiments, first, the execution subject may obtain a posture graph sequence of the target user. The posture graph sequence may be used to characterize the posture information of the user during walking, such as the above-mentioned surveillance video. The posture graph in the posture graph sequence may be a grayscale graph, etc. Then, the execution subject may input the posture graph sequence into a preset human body segmentation network model, thereby outputting a user profile graph sequence of the target user. And the posture graph sequence may be input into a preset human body analysis network model, thereby outputting a human body part graph sequence of the target user.

[0035] The above-mentioned human body segmentation network model and the above-mentioned human body parsing network model can also adopt various machine learning models with image recognition and analysis functions. As an example, the human body segmentation network model can adopt the HR Net network structure. Among them, the user profile map can be marked with background identification and human body identification. The representation method of the identification is also not limited. For example, the user profile map may only contain 0 and 1, 0 represents the background, and 1 represents the human body. The human body parsing network model can adopt the CDGNet network structure. Among them, the human body part map can be marked with identification of various parts of the human body. The human body part map can include one of the head, torso, left arm, right arm, left hand, right hand, left leg, right leg, left foot and right foot.

[0036] Step 106: fusing the target user's contour features and body part features to obtain body contour fusion features.

[0037] In some embodiments, the execution subject may fuse the contour features and body part features of the target user to obtain body contour fusion features.

[0038] In an actual application scenario, the execution subject may fuse the contour features and body part features of the target user through the following steps:

[0039] In the first step, a global pooling layer is used to compress the above contour features and the above body part features in spatial dimensions, respectively, to obtain compressed contour features and compressed body part features. The global pooling layer may refer to a global average pooling layer (Global Average Pooling, GAP).

[0040] In the second step, the compressed contour features and the compressed human body part features are spliced ​​to obtain spliced ​​features.

[0041] The third step is to extract the above splicing features according to the preset feature dimension attenuation ratio, and obtain the segmentation result of the above splicing features in the channel dimension. The fully connected layer (FC) is used to extract the above splicing features, and obtain the segmentation result of the above splicing features in the channel dimension.

[0042] The fourth step is to multiply and sum the segmentation results of the above-mentioned contour features, the above-mentioned human body part features and the corresponding above-mentioned splicing features to obtain the overall fusion features.

[0043] In the fifth step, the above-mentioned contour features and the above-mentioned body part features are divided according to the distribution positions of various parts of the human body, so as to obtain contour sub-feature groups and body part sub-feature groups.

[0044] In the sixth step, the above-mentioned contour sub-feature group and the above-mentioned human body part sub-feature group are compressed in spatial dimension to obtain a compressed contour sub-feature group and a compressed part sub-feature group.

[0045] In the seventh step, the plurality of compression contour sub-features are locally aligned with the corresponding plurality of compression position sub-features to obtain a locally aligned sub-feature group.

[0046] In the eighth step, the above local alignment sub-feature groups are dimensionally spliced ​​to obtain local fusion features.

[0047] As an example, for the outline feature, the execution subject can divide the outline feature into three set parts: upper, middle and lower along the horizontal dimension according to the height of the user's outline. Correspondingly, for the part feature, it can be divided into three parts: upper (head), middle (torso, left arm, right arm, left hand, right hand) and lower (left leg, right leg, left foot, right foot).

[0048] For example, for part features, the execution subject can adopt a partitioning strategy to determine the weight and mask corresponding to the part feature according to the set part to which each part feature belongs in the part feature. And the difference between the set value and the weight and mask is used as the weight and mask corresponding to the part features belonging to other set parts. Then, the sum of the product values ​​of each part feature in the part feature and the corresponding weight and mask can be determined as the partition result of the partitioning strategy. In this way, the part feature can be divided into three set parts: upper, middle and lower along the horizontal dimension.

[0049] In the ninth step, the above-mentioned overall fusion features and the above-mentioned local fusion features are combined into the human body contour fusion features.

[0050] In this way, the complementarity of the user's contour features and part features can be fully utilized to provide more robust features for gesture recognition.

[0051] Step 107: splice the above-mentioned contour features, the above-mentioned human body part features and the above-mentioned human body contour fusion features into the walking posture features of the above-mentioned target user.

[0052] In some embodiments, the execution entity may combine the contour features, the body part features and the body contour fusion features into the walking posture features of the target user.

[0053] Step 108: performing fall detection on the walking posture features to generate a fall detection result.

[0054] In some embodiments, the execution entity may perform fall detection on the walking posture feature to generate a fall detection result.

[0055] In an actual application scenario, the execution subject can perform fall detection based on the walking posture features through the following steps:

[0056] In the first step, the above walking posture features are input into the pre-trained user posture recognition model to generate the user posture recognition result. The user posture recognition result can represent the user's current posture, whether it is upright. For example, the user posture recognition model can be an SVM (Support Vector Machine) classifier, which is used to determine whether the user is in a non-upright state (squatting, falling, etc.). For another example, the user posture recognition model can be a MoveNet model, a PoseNet model, a DCPose model, and a DensePose model. Among them, MoveNet was developed by Google researchers and built using TensorFlow.js. It has super fast and highly accurate characteristics and can detect 17 key points of the human body. It has two versions: the Lightning version is for applications with low latency requirements, and the Thunder version is designed for applications that focus on achieving higher accuracy. PoseNet is a deep learning model that uses TensorFlow to detect different body parts and provides comprehensive skeleton information by adding other key points. It can detect postures in real time and can effectively perform single and multi-posture detection of the human body.

[0057] The user gesture recognition model can be trained by the following steps:

[0058] The first step is to obtain a user posture image training sample set, wherein the user posture image training sample includes: a sample user posture image and a sample user posture label.

[0059] The second step is to determine the initial user posture recognition model. The initial user posture recognition model includes: an initial feature extraction network, an initial user posture recognition network, the initial feature extraction network includes: an initial convolution model sequence, and the initial convolution model in the initial convolution model sequence includes: an initial subconvolution model and an initial activation model.

[0060] Here, the input of the initial convolution sub-model included in the initial convolution model in the initial convolution model sequence may be: the output of the initial activation model included in the previous initial convolution model. The input of the initial convolution sub-model included in the first initial convolution model in the initial convolution model sequence may be: the sample user posture image. The input of the initial activation model of the initial convolution model in the initial convolution model sequence may be: the output of the initial convolution sub-model included in the initial convolution model. For example, the number of initial convolution models in the initial convolution model sequence may be 22.

[0061] The initial user gesture recognition network may be a model that takes the initial user gesture image features as input and takes the initial user gesture recognition results as output. The initial user gesture recognition network is used to: filter the initial user gesture image features through a preset filtering algorithm to generate the initial user gesture recognition results. For example, the preset filtering algorithm may be: NMS (Non Maximum Suppression) algorithm.

[0062] The third step is to select a target user posture image training sample from the user posture image training sample set. A user posture image training sample can be randomly selected from the user posture image training sample set as the target user posture image training sample.

[0063] In the fourth step, the sample user posture images included in the target user posture image training samples are input into the initial feature extraction network to obtain the initial user posture image features.

[0064] The fifth step is to input the initial user posture image features into the initial user posture recognition network to obtain the initial user posture recognition result.

[0065] Step 6: Based on a preset loss function, determine the loss value between the initial user posture recognition result and the corresponding sample user posture label. For example, the preset loss function may be a hinge loss function or a cross entropy loss function.

[0066] In a seventh step, in response to determining that the loss value is less than or equal to the preset difference value, the initial user gesture recognition model is determined as the trained user gesture recognition model.

[0067] Therefore, relatively accurate user posture information can be detected according to a relatively accurate user posture recognition model, so as to subsequently identify whether the user has fallen through the relatively accurate user posture information.

[0068] In the second step, in response to determining that the user posture recognition result meets the fall detection condition, the walking posture feature is input into a pre-trained fall detection model to generate a fall detection result. For example, the fall detection model can be an SVM classifier, which is used to distinguish between users in a fall or other non-upright state, and extracts features such as Zernike moment features, the height of the moving user, the width of the moving user, the contour area of ​​the moving user, and the perimeter of the contour of the moving user to train the classifier and make classification decisions.

[0069] The fall detection model can be trained by the following steps:

[0070] The first step is to obtain a group of walking posture graph sequences, where one walking posture graph sequence corresponds to one user.

[0071] The second step is to construct a walking posture graph training sample set based on the walking posture graph sequence group, wherein the walking posture graph training sample includes: a walking posture graph sequence and a sample label.

[0072] The third step is to select a target walking posture graph training sample from the walking posture graph training sample set. A walking posture graph training sample can be randomly selected from the walking posture graph training sample set as the target walking posture graph training sample.

[0073] In the fourth step, the walking posture graph sequence included in the target walking posture graph training sample is input into the initial fall detection model to obtain the initial fall detection result. The initial fall detection model can be an untrained fall detection model. For example, the initial user fall detection model can be a CDG Net network structure.

[0074] The fifth step is to determine the detection loss value between the initial fall detection result and the corresponding sample label based on the preset loss function.

[0075] In a sixth step, in response to determining that the detection loss value is less than or equal to the preset detection loss value, the initial fall detection model is determined as the trained fall detection model.

[0076] Step 109: in response to determining that the fall detection result satisfies a preset fall condition, controlling the associated alarm device to emit an alarm tone.

[0077] In some embodiments, the execution subject may control the associated alarm device to emit an alarm tone in response to determining that the fall detection result satisfies a preset fall condition. The preset fall condition may be: the fall detection result indicates that the user has fallen.

[0078] Further, in response to determining that the above fall detection result does not meet the preset fall condition, feature extraction is performed on each video frame image in the above video frame image sequence to obtain a video frame image feature sequence.

[0079] In some embodiments, the execution subject may extract features from each video frame image in the video frame image sequence in response to determining that the fall detection result does not meet the preset fall condition, thereby obtaining a video frame image feature sequence. A pre-trained image feature extraction network may be used to extract features from the video frame images, thereby obtaining video frame image features. The image feature extraction network may be a convolutional neural network (CNN).

[0080] Furthermore, user behavior prediction is performed based on the above-mentioned video frame image feature sequence to generate user behavior prediction text.

[0081] In some embodiments, the execution subject may perform user behavior prediction based on the video frame image feature sequence to generate a user behavior prediction text. For example, the video frame image feature sequence may be input into a pre-trained time series model to predict the user's behavior events. Then, the execution subject may generate the corresponding user behavior prediction text based on the predicted behavior events.

[0082] Furthermore, in response to determining that the user behavior prediction text determines that there is a safety issue, safety prompt information is generated, and the safety prompt information is sent to the mobile terminal of the target user, and the fall sensor associated with the target user is controlled to issue a vibration prompt.

[0083] In some embodiments, the execution subject may generate safety prompt information in response to determining that the user behavior prediction text determines that there is a safety problem, and send the safety prompt information to the mobile terminal of the target user, and control the fall sensor associated with the target user to send a vibration prompt. The mobile terminal may be a mobile phone terminal of the target user. The fall sensor may be a vibration sensor set on the user, used to prompt the user to prevent falling and take a rest in time. The safety prompt information may refer to voice prompt information, which is used to prompt the user to find an object to lean on in time to prevent falling.

[0084] Furthermore, a multimodal language model is used to label the user behavior prediction text to generate labeled user behavior prediction text.

[0085] In some embodiments, the execution subject may use a multimodal language model to mark the user behavior prediction text to generate a marked user behavior prediction text. The marked user behavior prediction text is used to highlight events that characterize user behavior and safety hazards of falling. For example, data that characterizes user behavior and safety hazards in the user behavior prediction text may be marked.

[0086] Furthermore, feature extraction is performed on the above-mentioned marked user behavior prediction text to generate a marked user behavior prediction text feature vector, and security level prediction is performed on the above-mentioned marked user behavior prediction text feature vector.

[0087] In some embodiments, the execution subject may perform feature extraction on the marked user behavior prediction text to generate a marked user behavior prediction text feature vector, and perform security level prediction on the marked user behavior prediction text feature vector. A text feature extraction model may be used to perform feature extraction on the marked user behavior prediction text to generate a marked user behavior prediction text feature vector. For example, the text feature extraction model may be a Bert model.

[0088] In an actual application scenario, the execution subject can perform security level prediction on the marked user behavior prediction text feature vector through the following steps:

[0089] The first step is to determine the vector similarity between the above-mentioned marked user behavior prediction text feature vector and each text feature vector in a pre-built text feature vector database, wherein the above-mentioned text feature vector database stores text feature vectors and corresponding security levels.

[0090] In the second step, the text feature vector with the highest vector similarity is determined as the target text feature vector, and the security level corresponding to the target text feature vector is determined as the security level of the marked user behavior prediction text feature vector.

[0091] The present application also provides a computer device 200. Figure 2 As shown, the computer device 200 includes: a bus 201, a processor 202, a memory 203 and a communication interface 204. The processor 202, the memory 203 and the communication interface 204 communicate with each other through the bus 201. The computer device 200 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computer device 200.

[0092] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 The bus 201 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 201 may include a path for transmitting information between various components of the computer device 200 (for example, the processor 202, the memory 203, and the communication interface 204).

[0093] The processor 202 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0094] The memory 203 may include a volatile memory, such as a random access memory (RAM). The memory 203 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0095] The memory 203 stores executable program codes, and the processor 202 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module, sampling module, determination module, and mixing module, thereby implementing the aforementioned method for preventing and detecting user falls. That is, the memory 203 stores instructions for executing the aforementioned method for preventing and detecting user falls.

[0096] The communication interface 204 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computer device 200 and other devices or a communication network.

[0097] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the above-mentioned prevention and detection method based on user falls.

[0098] The embodiment of the present application also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The above-mentioned available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk). The computer-readable storage medium includes instructions, and the above-mentioned instructions instruct the computing device to execute the above-mentioned prevention and detection method based on user falls.

[0099] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for preventing and detecting user falls, comprising: In response to receiving a user fall event detection instruction, controlling an associated camera device to collect a monitoring video of a target user in a target area in real time; Performing frame processing on the surveillance video to obtain a video frame image sequence; Extracting a user profile image of each video frame image in the video frame image sequence to obtain a user profile image sequence; Extracting profile features corresponding to the target user according to the user profile graph sequence; Extracting body part features corresponding to the target user according to the body part map sequence of the target user, wherein the body part map sequence represents maps of various body parts of the target user during walking; Fusing the target user's contour features and body part features to obtain a body contour fusion feature; splicing the contour feature, the body part feature and the body contour fusion feature into a walking posture feature of the target user; Performing fall detection on the walking posture feature to generate a fall detection result; In response to determining that the fall detection result meets a preset fall condition, controlling an associated alarm device to emit an alarm prompt sound; The performing fall detection on the walking posture feature to generate a fall detection result includes: Obtaining a user posture image training sample set, wherein the user posture image training sample includes: a sample user posture image and a sample user posture label; Determine an initial user gesture recognition model, wherein the initial user gesture recognition model includes: an initial feature extraction network, an initial user gesture recognition network, the initial feature extraction network includes: an initial convolution model sequence, and the initial convolution model in the initial convolution model sequence includes: an initial subconvolution model and an initial activation model; Selecting a target user posture image training sample from the user posture image training sample set; Inputting the sample user posture image included in the target user posture image training sample into an initial feature extraction network to obtain initial user posture image features; Inputting the initial user posture image features into the initial user posture recognition network to obtain the initial user posture recognition result; Based on a preset loss function, determining a loss value between an initial user posture recognition result and a corresponding sample user posture label; In response to determining that the loss value is less than or equal to the preset difference value, determining the initial user gesture recognition model as the trained user gesture recognition model; Inputting the walking posture feature into a pre-trained user posture recognition model to generate a user posture recognition result; Obtaining a walking posture graph sequence group, wherein one walking posture graph sequence corresponds to one user; According to the walking posture graph sequence group, a walking posture graph training sample set is constructed, wherein the walking posture graph training sample includes: a walking posture graph sequence and a sample label; Selecting a target walking posture graph training sample from the walking posture graph training sample set; Inputting the walking posture graph sequence included in the target walking posture graph training sample into the initial fall detection model to obtain an initial fall detection result; Based on a preset loss function, determine a detection loss value between an initial user fall detection result and a corresponding sample label; In response to determining that the detection loss value is less than or equal to a preset detection loss value, determining the initial fall detection model as a trained fall detection model; In response to determining that the user posture recognition result satisfies a fall detection condition, inputting the walking posture feature into a pre-trained fall detection model to generate a fall detection result; The step of fusing the target user's contour features and body part features to obtain a body contour fusion feature includes: A global pooling layer is used to compress the contour features and the human body part features in spatial dimensions to obtain compressed contour features and compressed human body part features; Splicing the compressed contour feature and the compressed human body part feature to obtain a spliced ​​feature; According to a preset feature dimension attenuation ratio, feature extraction is performed on the splicing feature to obtain a segmentation result of the splicing feature in the channel dimension; Multiplying the contour feature, the human body part feature and the segmentation result of the corresponding splicing feature and summing them up to obtain an overall fusion feature; According to the distribution positions of various parts of the human body, the contour features and the human body part features are divided respectively to obtain contour sub-feature groups and human body part sub-feature groups; Compressing the contour sub-feature group and the human body part sub-feature group in spatial dimension to obtain a compressed contour sub-feature group and a compressed part sub-feature group; Locally aligning the plurality of compression contour sub-features with the corresponding plurality of compression position sub-features to obtain a local aligned sub-feature group; Dimensionally concatenating the local alignment sub-feature groups to obtain local fusion features; The overall fusion feature and the local fusion feature are combined into a human body contour fusion feature.

2. The method for preventing and detecting user falls according to claim 1, wherein: The method further comprises: In response to determining that the fall detection result does not meet the preset fall condition, performing feature extraction on each video frame image in the video frame image sequence to obtain a video frame image feature sequence; Predicting user behavior based on the video frame image feature sequence to generate user behavior prediction text; In response to determining that the user behavior prediction text determines that there is a safety issue, generating safety prompt information, sending the safety prompt information to the mobile terminal of the target user, and controlling a fall sensor associated with the target user to issue a vibration prompt.

3. A computer device, wherein: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 2 are implemented.

4. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images