Intelligent elderly care risk prediction method and system based on artificial intelligence and video analysis

Abstract

The present invention relates to the field of artificial intelligence technology, and in particular to a smart elderly care danger prediction method and system based on artificial intelligence and video analysis. The method constructs a simulated dangerous action model; collects real-time action videos of the elderly, matches the action information in the real-time action videos with the simulated action information in the simulated dangerous action model, and issues a danger warning based on the matching results. Through reinforcement learning, various prediction situations in the interaction between the elderly and scene information are simulated, the real-time action status of the elderly is matched with the simulated action status, and a danger warning is issued based on the matching results, thereby improving the intelligence level of the security monitoring system.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a smart elderly care risk prediction method and system based on artificial intelligence and video analysis. Background Art

[0002] With the increasing aging population and the rise of the "silver economy", the smart elderly care industry has ushered in a good period of development under the background of the country's strong support for the elderly care industry. In the existing technology, the monitoring equipment for smart elderly care mainly includes smart wheelchairs, monitoring beds, etc., which use monitoring data to monitor the physical condition of the elderly.

[0003] Technical personnel in this field have discovered that the existing technology has the following technical problems: health and safety monitoring of the elderly only through detection equipment will be limited by equipment abnormalities, equipment data errors and equipment sensitivity, which will ultimately lead to inaccurate monitoring results and inability to take corresponding measures in a timely manner. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a smart elderly care risk prediction method and system based on artificial intelligence and video analysis. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a smart elderly care risk prediction method based on artificial intelligence and video analysis, the method comprising:

[0006] Constructing a simulated dangerous action model; the simulated dangerous action model refers to a scenario simulation of an elderly person in a process of walking to a dangerous state, combining all actions and the surrounding environment;

[0007] Collect real-time action videos of the elderly, match the action information in the real-time action videos with the simulated action information in the simulated dangerous action model, and issue a danger warning based on the matching result.

[0008] Furthermore, the method for constructing the simulated dangerous action model includes:

[0009] According to the movement information of the elderly, the attention-salient areas corresponding to different movements in each walking state are obtained, and target detection is performed on the detected attention-salient areas to obtain the total number of targets;

[0010] The total number of targets is used as the prediction weight of the corresponding action between adjacent walking states, and the prediction weight adjustment coefficient of the action is set at the same time;

[0011] Reinforcement learning is used to simulate various action states of the elderly in a time series, and the simulated dangerous action model is obtained by adjusting the prediction weight adjustment coefficient between each walking state in each action state through feedback in the learning reinforcement process.

[0012] Furthermore, the feedback adjustment method of the prediction weight adjustment coefficient includes:

[0013] Obtain the probability of danger for each action through historical data of dangers encountered by the elderly;

[0014] Based on the dangerous termination walking state, the sum of the number of targets in the attention salient area corresponding to all actions between the initial walking state and the previous walking state is obtained, and the influence function of the number of targets is constructed according to the sum of the number of targets;

[0015] According to the number of walking states between the walking state corresponding to the feedback adjustment and the termination behavior state, the learning rate of the feedback adjustment is obtained;

[0016] The product of the influence function, the learning rate, the danger probability of the action and the gradient value in the reinforcement learning process is obtained, and the updated prediction weight adjustment coefficient of the corresponding action in the feedback-adjusted walking state is obtained by combining the product and the set prediction weight adjustment coefficient.

[0017] Furthermore, the method for acquiring the attention-salient area includes:

[0018] The human head rotation angle corresponding to the action and the position of the elderly are input into the attention area judgment network, and the attention salient area corresponding to the action in the surrounding environment is output.

[0019] Furthermore, the training process of the attention region judgment network is as follows:

[0020] A plurality of regional images are acquired by changing the angle and focal length of the camera, and these regional images are used as annotation images;

[0021] Obtain the attention-salient area image corresponding to the head rotation angle in the surrounding environment when the elderly walk in various positions;

[0022] The loss function of the attention region judgment network is constructed according to the cosine similarity of the corresponding feature vectors between the annotated image and the attention salient region image.

[0023] Furthermore, the set prediction weight adjustment coefficient is 1.

[0024] Furthermore, the method for performing a danger warning according to the matching result includes:

[0025] The action state of the elderly in the real-time action video is matched with various simulated action states in the simulated dangerous action model to obtain the matching number of corresponding actions under the behavior state. The ratio between the total number of corresponding simulated actions between different simulated behavior states under the current simulated action state in the simulated dangerous action model and the matching number is used to issue a danger warning. When the ratio is greater than the danger warning threshold, a warning message is issued in advance.

[0026] Furthermore, the product is positively correlated with the updated prediction weight adjustment coefficient, and the set prediction weight adjustment coefficient is positively correlated with the updated prediction weight adjustment coefficient.

[0027] Furthermore, a smart elderly care risk prediction system based on artificial intelligence and video analysis includes a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that the processor implements the steps of any one of the above methods when executing the computer program.

[0028] The embodiments of the present invention have at least the following beneficial effects: various prediction situations in the process of interaction between the elderly and scene information are simulated through reinforcement learning, the real-time action status of the elderly is matched with the simulated action status, and danger warnings are issued according to the matching results, thereby improving the intelligence level of the security monitoring system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flowchart of the steps of a smart elderly care risk prediction method based on artificial intelligence and video analysis is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a smart elderly care risk prediction method and system based on artificial intelligence and video analysis proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0032] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0033] The following is a detailed description of a specific scheme of a smart elderly care risk prediction method and system based on artificial intelligence and video analysis provided by the present invention in conjunction with the accompanying drawings.

[0034] The specific scenario targeted by the embodiments of the present invention is: safety monitoring scenario for the elderly in a nursing home. The monitoring camera is an infrared camera that can capture panoramic images of the interior. The monitoring camera is located in the corner of the elderly's room. Camera motion blur and the influence of light sources are not considered.

[0035] See also Figure 1 , which shows a flowchart of a method for predicting risks in smart elderly care based on artificial intelligence and video analysis provided by an embodiment of the present invention, the method comprising the following steps:

[0036] Step S001, constructing a simulated dangerous action model; the simulated dangerous action model refers to a scenario simulation of an elderly person in the process of walking until danger occurs, combining all actions and the surrounding environment.

[0037] Specifically, first, the attention area in each walking state is selected according to the action information of the elderly, where each walking state refers to each walking action. Let S t Indicates the tth walking state. According to the movements of other parts of the elderly in the current walking state, the attention salient area in the current walking state is obtained.

[0038] Method for obtaining the salient area of ​​attention: From one walking state to the next, the elderly may have various other movements of other parts of their body, such as turning their heads, reaching out for objects, using leverage, etc., and then define the various other possible movements as: a1, a2, …, a n , where a nis the nth other action; then, in the current walking state, the human head posture information output by the OpenPose network corresponding to other actions and the position information of the elderly in the panoramic image are used as the input of the attention area judgment network, and the output is the attention salient area corresponding to the other actions in the panoramic image. The specific training process of the attention area judgment network is: by changing the angle and focal length of the camera, the images of each area in the initial panoramic image are obtained, and one angle and one focal length of the camera correspond to a regional image; each regional image is used as an annotated image, and mapped to the attention salient area image corresponding to the rotation angle of the head when the elderly walks at various positions in the room in the panoramic image, and then the attention salient area image corresponding to the rotation angle of the head at different positions of the elderly is obtained; the loss function is: Loss1=1-cos(A,B), where A represents the feature vector of the annotated image, B represents the feature vector of the attention salient area image in the panoramic image, and cos(·) represents the cosine similarity measure.

[0039] It should be noted that the acquisition of the attention-salient area image is to simulate the rotation angle of the elderly's head to capture images within the field of view of the camera, and the area image actually captured by the camera and the attention-salient image are used to obtain the attention-salient image corresponding to each other action of the elderly. Therefore, the angle of the camera corresponds to the rotation angle of the head, and the focal length of the camera corresponds to the position of the elderly.

[0040] Then, in the initial walking state S0 of the old man, obtain all other possible actions of the old man Corresponding attention salient area image All the salient area images are stacked into a multi-channel image, and the target detection network is used to perform target detection on each channel image in the multi-channel image. The target detection network structure is encoder-fully connected layer. The multi-channel image is used as the input of the target detection network. The encoder convolves each channel image to obtain the feature map of each channel, and uses the fully connected layer to classify (to determine whether it is a target or background) and regress (to obtain the bounding box of the target) the convolution features on the feature map to obtain the bounding box of the target in the salient area image. The loss function of the target detection network is: Loss2=L1+L2, where L1 represents the cross entropy loss function and L2 represents the mean square error loss function.

[0041] It should be noted that the target in the embodiment of the present invention refers to a bed, a table, a cabinet, a kettle, a wheelchair, etc.

[0042] According to the initial walking state of the elderly, the total number of targets in the image of the salient area of ​​attention corresponding to each other action As the prediction weight of other actions between the initial walking state S0 and the next walking state S1, As the prediction weight adjustment coefficient of other actions between the initial walking state S0 and the next walking state S1, during the initial simulation, the prediction weight adjustment coefficient of each other action is in It represents the predicted weight adjustment coefficient corresponding to the i-th other action in the j-th walking state. The predicted weight adjustment coefficient between each walking state is subsequently adjusted using the feedback in the reinforcement learning process.

[0043] Furthermore, reinforcement learning is used to simulate various action states of the elderly in the time series. In the embodiment of the present invention, the fall of the elderly is used as an example for risk monitoring. Therefore, the fall probability under each other action is sampled from the historical data of the elderly falling, and then other action sets are obtained. The corresponding fall probability set According to the fall probability corresponding to each other action, N simulations are performed, and the number of simulations N is proportional to the fall probability. When the probability of falling is , it is necessary to simulate 5 times and a fall occurs again. In N simulations, if some other action causes a fall in the next walking state, the prediction weight adjustment coefficient is feedback-adjusted according to other actions between all walking states in the current simulation process to obtain a simulated dangerous action model.

[0044] The specific process of feedback adjustment of the prediction weight adjustment coefficient is as follows: according to the termination walking state S of the fall z Start, to the last walking state S before the end walking state z-1 , obtain the initial walking state S0 to the walking state S z-1 The sum of the number of targets in the attention salient area corresponding to all other actions between Using the sum of target quantities To adjust the action prediction weight adjustment coefficient between different walking states, the adjustment formula is:

[0045]

[0046] in, Indicates walking state S z-1 Other actions The corresponding updated prediction weight adjustment coefficient; Indicates the walking state S during the initial simulation z-1 Other actions The corresponding prediction weight adjustment coefficient, and in the initial simulation process, the prediction weight adjustment coefficients of the corresponding actions in each walking state are all 1; Indicates walking state S z-1 The following corresponds to other actions The probability of falling; Represents the gradient value of the reinforcement learning model for feedback regulation after a fall; ε n represents the learning rate from the termination walking state (fall) to the initial walking state, n represents the number of states from the state layer of feedback regulation to the termination layer, that is, the number of walking states between the walking state corresponding to feedback regulation and the walking state corresponding to the fall. It is believed that the farther away from the termination walking state, the greater the learning rate of regulation and the higher the uncertainty of the fall; The influence function representing the total number of targets in the attention-salient areas corresponding to other actions under different walking states, the mapping range of the influence function is [0,2]; the overall prediction weight adjustment coefficient is normalized, γ∈[0,1].

[0047] Step S002, collect the real-time action video of the elderly, match the action information in the real-time action video with the simulated action information in the simulated dangerous action model, and issue a danger warning based on the matching result.

[0048] Specifically, through the actual action video of the elderly captured by the surveillance camera, the actual actions corresponding to the walking states of the elderly in the video are matched with the simulated actions corresponding to the simulated walking states in the simulated dangerous action model. When the matching result satisfies the simulated walking state and simulated action in the simulated dangerous action model, a danger warning is issued based on the simulated behavior state and simulated action between the termination walking states when a fall occurs in the current simulated dangerous action model.

[0049] It should be noted that the simulated dangerous action model corresponds to multiple action state processes, and the multiple action state processes correspond to different walking states and other actions between different walking states, until the walking state is terminated and danger (falling) occurs, and the length of the walking state corresponding to each action state is different.

[0050] Danger warning method: According to the action state of the elderly in the real-time action video, the various simulated action states in the simulated dangerous action model are matched to obtain the number of matching actions H under the behavior state, and the total number of corresponding simulated actions H between different simulated behavior states in the current simulated fall process in the simulated dangerous action model is used. max The ratio between the number of matches H Perform danger warning and set the danger warning threshold M1 = 0.8. When the elderly are in danger, early warning information will be issued to predict that the elderly may be in danger and need to increase the caregiver's attention or rescue measures.

[0051] In summary, the embodiments of the present invention provide a smart elderly care danger prediction method based on artificial intelligence and video analysis, which constructs a simulated dangerous action model; the simulated dangerous action model refers to a scene simulation of the elderly in the process of walking to the danger, combining all actions and the surrounding environment; collecting the real-time action video of the elderly, matching the action information in the real-time action video with the simulated action information in the simulated dangerous action model, and issuing a danger warning based on the matching result. Through reinforcement learning to simulate various prediction situations in the process of interaction between the elderly and scene information, the real-time action status of the elderly is matched with the simulated action status, and a danger warning is issued based on the matching result, thereby improving the intelligence level of the security monitoring system.

[0052] Based on the same inventive concept as the above method, an embodiment of the present invention also provides a smart elderly care risk prediction system based on artificial intelligence and video analysis, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that when the processor executes the computer program, it implements the steps of any one of the above-mentioned smart elderly care risk prediction methods based on artificial intelligence and video analysis.

[0053] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0054] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A smart elderly care risk prediction method based on artificial intelligence and video analysis, It is characterized in that The method includes: Constructing a simulated dangerous action model; the simulated dangerous action model refers to a scenario simulation of an elderly person in a process of walking to a dangerous state, combining all actions and the surrounding environment; Collecting real-time action videos of the elderly, matching the action information in the real-time action videos with the simulated action information in the simulated dangerous action model, and issuing a danger warning according to the matching result; The method for constructing the simulated dangerous action model comprises: According to the movement information of the elderly, the attention-salient areas corresponding to different movements in each walking state are obtained, and target detection is performed on the detected attention-salient areas to obtain the total number of targets; The total number of targets is used as the prediction weight of the corresponding action between adjacent walking states, and the prediction weight adjustment coefficient of the action is set at the same time; Reinforcement learning is used to simulate various action states of the elderly in a time series, and in the learning reinforcement process, the prediction weight adjustment coefficient between each walking state under each action state is adjusted by feedback to obtain the simulated dangerous action model; The feedback adjustment method of the prediction weight adjustment coefficient includes: Obtain the probability of danger for each action through historical data of dangers encountered by the elderly; Based on the dangerous termination walking state, the sum of the number of targets in the attention salient area corresponding to all actions between the initial walking state and the previous walking state is obtained, and the influence function of the number of targets is constructed according to the sum of the number of targets; According to the number of walking states between the walking state corresponding to the feedback adjustment and the termination behavior state, the learning rate of the feedback adjustment is obtained; Obtain the product of the influence function, the learning rate, the danger probability of the action and the gradient value in the reinforcement learning process, and obtain the updated prediction weight adjustment coefficient of the corresponding action in the feedback-adjusted walking state by combining the product and the set prediction weight adjustment coefficient; The method for acquiring the attention salient area comprises: The human head rotation angle corresponding to the action and the position of the elderly are input into the attention region judgment network, and the attention salient region corresponding to the action in the surrounding environment is output; The training process of the attention region judgment network is as follows: A plurality of regional images are acquired by changing the angle and focal length of the camera, and these regional images are used as annotation images; Obtain the attention-salient area image corresponding to the head rotation angle in the surrounding environment when the elderly walk in various positions; The loss function of the attention region judgment network is constructed according to the cosine similarity of the corresponding feature vectors between the annotated image and the attention salient region image.

2. The method according to claim 1, It is characterized in that The set prediction weight adjustment coefficient is 1.

3. The method according to claim 1, It is characterized in that The method for performing a danger warning according to the matching result includes: The action state of the elderly in the real-time action video is matched with various simulated action states in the simulated dangerous action model to obtain the matching number of corresponding actions under the behavior state. The ratio between the total number of corresponding simulated actions between different simulated behavior states under the current simulated action state in the simulated dangerous action model and the matching number is used to issue a danger warning. When the ratio is greater than the danger warning threshold, a warning message is issued in advance.

4. The method according to claim 1, It is characterized in that The product is positively correlated with the updated prediction weight adjustment coefficient, and the set prediction weight adjustment coefficient is positively correlated with the updated prediction weight adjustment coefficient.

5. A smart elderly care risk prediction system based on artificial intelligence and video analysis, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

Images