Family companionship method, device, electronic device and computer-readable storage medium

Abstract

The present invention discloses a family companionship method, device, electronic device and computer-readable storage medium, including: collecting facial data, behavioral data, physiological data and environmental data; processing the physiological data to obtain physiological recognition results, judging whether it is dangerous information, and obtaining dangerous instructions; processing the environmental data to obtain environmental recognition results, judging whether it is adjustment information, and obtaining adjustment instructions; processing the behavioral data and facial data to obtain human body recognition results, judging whether it is a distress message, when the facial data is recognized as distress facial data information, a first distress instruction will be generated, and when the behavioral data is recognized as distress behavioral data information, a second distress instruction will be generated; the first distress instruction, the second distress instruction, the dangerous instruction, and the adjustment instruction are fed back to the terminal mobile device to execute user services or emergency response. The present invention can better accompany family members, and provide services such as environmental adjustment and danger alarm by voice.

Description

Technical Field

[0001] The present invention belongs to the technical field of family companionship systems, and specifically relates to a family companionship method, device, electronic device and computer-readable storage medium. Background Art

[0002] Home companion robots can be used to provide services in the home environment. For example, they can use machine learning algorithms to analyze and process the collected data, and provide health management services by monitoring physiological parameters such as heart rate and activity level, especially for the elderly or people with special health needs; or the robot can be integrated with other smart devices in the home to achieve environmental control and automated management; or the robot can be integrated with emergency response functions to automatically take action when it detects that the user has urgent medical needs, such as notifying emergency services or family members.

[0003] Existing home companion robots only have some of the functions mentioned above, the actual user experience needs to be improved, and the information sharing between function judgment modules also needs to be improved. Summary of the invention

[0004] The purpose of the present invention is to provide a family companionship method, device, electronic device and computer-readable storage medium to solve the above-mentioned problems existing in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, a family companionship method includes: collecting facial data, behavioral data, physiological data and environmental data; processing the physiological data to obtain a physiological recognition result, comparing the actual physiological safety value information with the preset physiological safety value information, judging whether it is danger information, and obtaining danger instructions; processing the environmental data to obtain an environment recognition result, comparing the actual environment value information with the preset environment comfort value information, judging whether it is adjustment information, and obtaining adjustment instructions; processing the behavioral data and facial data to obtain a human body recognition result, judging whether it is a distress message, when the facial data is recognized as distress facial data information, a first distress instruction is generated, and when the behavioral data is recognized as distress behavioral data information, a second distress instruction is generated; the first distress instruction, the second distress instruction, the danger instruction, and the adjustment instruction are fed back to the terminal mobile device to execute user services or emergency responses.

[0006] Preferably, the current heart rate is obtained based on the physiological recognition result, and when the facial data is identified as facial data information for asking for help, a first distress instruction will be generated, including: determining whether the identified facial features are facial features for asking for help, and when they are facial features for asking for help, determining whether the current heart rate is greater than a preset heart rate, and if so, timing the current heart rate as a high heart rate duration, and when the high heart rate duration is greater than the preset high heart rate duration, it is confirmed as a distress situation; the facial features for asking for help include rolling eyes, seriousness, astonishment, pain and fear.

[0007] Preferably, when the behavior data is identified as distress behavior data information, a second distress instruction will be generated, including: determining whether the identified behavior feature is a distress behavior feature; if it is a distress behavior feature, timing the time the distress behavior feature is maintained, which is counted as the duration of the distress behavior; when the duration of the distress behavior is greater than the preset duration of the distress behavior, it is confirmed as a distress situation; the distress behavior features include holding the head with both hands, raising both hands, and falling down.

[0008] Preferably, the human body appearance features are obtained based on the environment recognition results, the human body physiological features are obtained based on the human body recognition results, the human body age is judged, it is judged whether it is dangerous information, and a danger instruction is obtained, including: when judging whether the human body is a child, if it is a child, it will be judged whether the actual physiological safety value information is outside the first preset physiological safety value information, if "yes", a first physiological danger instruction will be generated, when it is not a child, it will be judged whether the actual physiological safety value information is outside the second preset physiological safety value information, if "yes", a second physiological danger instruction will be generated.

[0009] Preferably, determining whether it is adjustment information and obtaining an adjustment instruction includes: determining whether the actual environmental value information is outside the preset environmental comfort range value information, and when it is outside the preset environmental comfort range value information, sending a signal of whether the environment needs to be adjusted, and receiving a signal of whether the environment needs to be adjusted, confirming it as a situation to be adjusted.

[0010] Preferably, actual vibration value information is obtained based on the environment recognition result; whether it is danger information is determined, and a danger instruction is obtained, including: when the actual vibration value information is less than the preset vibration value information, whether the human body appearance feature is a child's appearance feature, when it is a child's appearance feature, it is determined whether the actual physiological safety value information is outside the third preset physiological safety value information, if "yes", a third physiological danger instruction is generated, and when it is not a child's appearance feature, it is determined whether the actual physiological safety value information is outside the fourth preset physiological safety value information, if "yes", a fourth physiological danger instruction is generated.

[0011] Preferably, a distress message is obtained based on the result of human body recognition; whether it is danger message is determined, and a danger instruction is obtained, including: confirming that the person is in a distress state according to the distress message, determining whether the body shape features are those of a child and whether the body physiological features are those of a child; if they are the body shape features and the physiological features of a child, determining whether the actual physiological safety value information is outside the fifth preset physiological safety value information, and if so, generating the fifth physiological danger instruction; if they are not the body shape features and the physiological features of a child, determining whether the actual physiological safety value information is outside the sixth preset physiological safety value information, and if so, generating the sixth physiological danger instruction.

[0012] In a second aspect, a family companion device includes: a data acquisition module for collecting facial data, behavioral data, physiological data and environmental data; a danger judgment module for processing physiological data to obtain physiological recognition results, comparing actual physiological safety value information with preset physiological safety value information, judging whether it is danger information, and obtaining danger instructions; an adjustment judgment module for processing environmental data to obtain environmental recognition results, comparing actual environmental value information with preset environmental comfort value information, judging whether it is adjustment information, and obtaining adjustment instructions; a distress judgment module for processing behavioral data and facial data to obtain human body recognition results, judging whether it is distress information, generating a first distress instruction when facial data is recognized as distress facial data information, and generating a second distress instruction when behavioral data is recognized as distress behavioral data information; an interactive response module for feeding back the first distress instruction, the second distress instruction, the danger instruction, and the adjustment instruction to the terminal mobile device to perform user services or emergency responses.

[0013] According to a third aspect, an electronic device comprises a memory, a processor and a computer program, wherein the computer program is stored in the memory and executed by the processor to implement any one of the methods according to the first aspect.

[0014] According to a fourth aspect, a computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of an embodiment of the present application.

[0016] Figure 2 Schematic diagram of the process of step S300 in the family companionship method according to an embodiment of the present application.

[0017] Figure 3 Schematic diagram of the process of step S400 in the family companionship method according to an embodiment of the present application.

[0018] Figure 4This is a first flow chart of step S200 in the family companionship method according to an embodiment of the present application.

[0019] Figure 5 This is a second flow chart of step S200 in the family companionship method according to an embodiment of the present application.

[0020] Figure 6 This is a third flow chart of step S200 in the family companionship method according to an embodiment of the present application.

[0021] Figure 7 This is a schematic diagram of the structure of a family companion device according to an embodiment of the present application.

[0022] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0024] In one embodiment, reference Figure 1 , a family companionship method, including: S100, collects facial data, behavioral data, physiological data, and environmental data; Facial data includes but is not limited to the contours of eyes, eyebrows, mouth, and nose; behavioral data includes body movements and human posture; physiological data includes but is not limited to electrocardiogram, blood pressure, blood oxygen saturation, heart rate, respiration, and body temperature; environmental data includes but is not limited to temperature, humidity, oxygen content, airflow velocity, formaldehyde content, noise value, vibration value, light intensity, etc.; S200, processing the physiological data to obtain a physiological recognition result, comparing the actual physiological safety value information with the preset physiological safety value information, determining whether it is dangerous information, and obtaining a dangerous instruction; The preset physiological safety value can be divided into two interval values ​​according to age, namely, children and non-children. In the implementation of this application, the age range of children is 0-10 years old, and the age range of non-children is over 11 years old. According to scientific data, the preset physiological safety value of children is taken as a lower limit interval of a physiological value. For example, the preset physiological safety value of children's heart rate, infants (0-1 years old)-normal heart rate range is 120 to 160 beats per minute, children (1-10 years old)-normal heart rate range is 70 to 130 beats per minute, and the upper and lower limits of children's heart rate are taken as 70 to 160 beats per minute; for example, the preset physiological safety value of non-children's heart rate, adolescents (11-17 years old): normal heart rate range is 60 to 100 beats per minute, adults (18-65 years old): normal heart rate range is 60 to 100 beats per minute, and the elderly (65 years old and above): normal heart rate range is 60 to 100 beats per minute. In summary, the upper and lower limits of non-children's heart rate are taken as 60 to 100 beats per minute. The actual physiological safety value information is the physiological data detected by the sensor and processed and identified by the controller.

[0025] S300, processing the environmental data to obtain an environmental recognition result, comparing the actual environmental value information with the preset environmental comfort value information, determining whether it is adjustment information, and obtaining an adjustment instruction; The preset environmental comfort values ​​are manually written into the controller through the terminal mobile device according to the personal comfort values ​​preferred by different family members.

[0026] S400, processing the behavior data and the facial data to obtain a human body recognition result, determining whether it is a distress message, and generating a first distress instruction when the facial data is recognized as distress facial data information, and generating a second distress instruction when the behavior data is recognized as distress behavior data information; S500, feeding back the first distress instruction, the second distress instruction, the danger instruction, and the adjustment instruction to the terminal mobile device to perform user services or emergency response; The terminal mobile device can be a mobile communication device, a mobile phone, an alarm, a hospital display screen, etc., which has a controller that can identify and judge the first distress command, the second distress command, the danger command, and the adjustment command. User services can be adjusting the environment, reminding to take medicine, etc., and emergency responses can be distress alarms, danger prompts, etc.

[0027] refer to Figure 3 , based on the physiological recognition result, the current heart rate is obtained. When the facial data is recognized as a distress facial data information, a first distress instruction will be generated, including: S411, determining whether the recognized facial features are facial features for seeking help; Facial features of seeking help include rolling eyes, seriousness, surprise, pain, and fear. The model is used to compare and judge the recognized facial features. For example, for rolling eyes, the human eye closure detection algorithm model based on the human eye aspect ratio calculation is used to judge the state of the human eye, and it can be determined whether it is the state of rolling eyes. S412, when the facial feature is a distress call, determining whether the current heart rate is greater than a preset heart rate; The preset heart rate can be set according to the actual situation of different family members. For example, Zhang San is the father of the family. His preset heart rate is 70 to 73 beats per minute. When his current heart rate is 74 beats per minute, it will be judged that the current heart rate is greater than the preset heart rate, and the high heart rate duration will start to be counted. S413, if it is greater than the preset heart rate, the current heart rate will be timed and counted as the high heart rate duration; S414, when the high heart rate duration is longer than the preset high heart rate duration, it is confirmed as a distress situation; The preset high heart rate duration is set according to actual needs. For example, the preset high heart rate duration can be 20-40 seconds. When the high heart rate duration is greater than 40 seconds, it will be confirmed as a distress situation, that is, the first distress command will be sent; When the present application is implemented in the above method, if Zhang San is rolling his eyes and his heart rate is 74 beats per minute, after 40 seconds of timing, a distress alarm message is sent to the terminal mobile device to inform family, friends or security departments.

[0028] refer to Figure 3 When the behavior data is identified as distress behavior data information, a second distress instruction will be generated, including: S421, determining whether the identified behavior feature is a help-seeking behavior feature; The characteristics of help-seeking behaviors include holding the head with both hands, raising both hands, and falling. The model is used to compare and judge the identified behavior characteristics. For example, for raising both hands, a deep learning algorithm model is used to accumulate image data of family members performing this behavior multiple times, and how to locate the coordinates of key points of the human body in an image is continuously learned to judge the characteristics of the help-seeking behavior. S422, when the feature is a help-seeking behavior, the time the help-seeking behavior feature is maintained is counted and counted as the duration of the help-seeking behavior; The duration of the help-seeking behavior can be the duration of holding the head with both hands, raising both hands, and falling down, such as holding the head with both hands for 5-7 seconds, 10-12 seconds, etc.; S423, when the duration of the help-seeking behavior is greater than the preset help-seeking behavior duration, it is confirmed as a help-seeking situation; The duration of the preset help behavior can be set as a time point according to actual needs. For example, it can be set to 7 seconds. When both hands hold the head for more than 7 seconds, it can be confirmed as a help situation and a second help command can be sent; When the present application is implemented in the above method, if Zhang San holds his head with both hands for 10-12 seconds, a distress alarm message will be sent to the terminal mobile device to inform family, friends or security departments.

[0029] Based on the results of environmental recognition, the human body shape features are obtained. The human body shape features are formed by the human body contour. Based on machine vision, the size of the contours of different family members can be identified. For example, the method of detecting the area range of a quadrilateral enclosed is used to judge. It can be understood that in the height direction, after identifying the human body coordinate points, four points are taken. The absolute value of the difference between the horizontal coordinates of the two points is the largest as the width of the quadrilateral, and the absolute value of the difference between the vertical coordinates of the other two points is the largest as the length of the quadrilateral. In different scenes, the area of ​​the quadrilateral enclosing Zhang San is 0.4-0.5 square meters, and the area of ​​the quadrilateral enclosing Zhang San's child is 0.2-0.25 square meters. This method can distinguish between Zhang San and Zhang San's child. Based on the results of human body recognition, the physiological characteristics of the human body are obtained. When processing facial data and behavioral data, the facial features of the human body will be identified, such as the state of the eyes, the length of the arms and legs, etc., which will be different at different ages, and the age of the human body can be judged. refer to Figure 4 , judging whether it is dangerous information and obtaining dangerous instructions, including: S211, judging whether the human body is a child; the physiological values ​​of children are different from those of non-children, and judgment must be made first; S212, when the child is a child, it is determined whether the actual physiological safety value information is outside the first preset physiological safety value information, and if "yes", a first physiological danger instruction is generated; For example, when judging whether the heart rate is safe, the normal heart rate range in the first preset physiological safety value information is 70 to 160 beats per minute. When the child's heart rate range is 170 beats per minute, the first physiological danger instruction will be sent; S213, when the child is not a child, it is determined whether the actual physiological safety value information is outside the second preset physiological safety value information, and if "yes", a second physiological risk instruction is generated; For example, when judging whether the heart rate is safe, the normal heart rate range in the second preset physiological safety value information is 60 to 100 beats per minute. When the heart rate range of non-children is 120 beats per minute, the second physiological danger instruction will be sent.

[0030] refer to Figure 2 , determine whether it is adjustment information, and obtain adjustment instructions, including: S311, determine whether the actual environmental value information is outside the preset environmental comfort range value information, and the preset environmental comfort range value information is pre-input according to the needs of each family member.

[0031] S312, when the environment is outside the preset comfortable range value information, sending a signal whether the environment needs to be adjusted; The signal of whether the environment needs to be adjusted can be sent to the home user through the touch screen or voice. Through the touch screen, a pop-up window will be sent to the user's display device, such as a mobile phone, iPad, PC, etc.; through voice, a voice question and answer prompt will be sent to the user. The deep learning model is used to recognize the user's voice, and the recognition result is calculated based on the two parameter weights of the mouth shape of different home users' answers (obtained by facial recognition results) and the sound frequency of different home users. The initial parameter weights are calculated based on the relevant characteristic values ​​of the two parameters of a certain amount of data from home users before the implementation of this application, and are continuously optimized based on the recognition results to ensure that the user's voice can be recognized in a noisy environment.

[0032] S313, receiving a signal that the environment needs to be adjusted, and confirming it as a situation to be adjusted; When the present application is implemented, for example, when the actual temperature is greater than the maximum value of the preset comfortable temperature range, the actual temperature is 25 degrees, and the preset comfortable temperature range is 20 to 23 degrees, a pop-up window with the information of "Do you want to adjust the temperature", "Yes" or "No" is sent through the touch screen, and the user clicks the pop-up window of "Yes" and receives a signal that the ambient temperature needs to be adjusted, that is, it is confirmed that the situation is to be adjusted and an instruction to adjust the ambient temperature is sent; for example, when the actual relative humidity is less than the minimum value of the preset comfortable relative humidity range, the actual relative humidity is 30%, and the preset comfortable relative humidity range is 40% to 50%, the voice message "Do you want to adjust the humidity" is sent, please answer "yes or no", the user's answer of "yes" is recognized, and a signal that the ambient relative humidity needs to be adjusted is received, that is, it is confirmed that the situation is to be adjusted and an instruction to adjust the ambient humidity is sent to the controller of the air conditioner, fan, etc.

[0033] Based on the environmental recognition results, the actual vibration value information is obtained; whether it is dangerous information is determined, and dangerous instructions are obtained, including: refer to Figure 5 , S221, when the actual vibration value information is less than the preset vibration value information, S222, determine whether the human body appearance features are child appearance features, S223, when it is a child appearance feature, determine whether the actual physiological safety value information is outside the third preset physiological safety value information, if "yes" a third physiological danger instruction will be generated, S224, when it is not a child appearance feature, determine whether the actual physiological safety value information is outside the fourth preset physiological safety value information, if "yes" a fourth physiological danger instruction will be generated.

[0034] The preset vibration value considers the maximum vibration value of the environment without accidental collision in the room and excluding high-frequency electrical appliances. The maximum vibration value under daily conditions in the room for a period of time can be collected as this value. When the actual vibration value information is greater than the preset vibration value, the physiological safety of the family members will be judged. When some items fall to the ground, objects collide accidentally, etc., causing people to be frightened, false alarms will be triggered. This method increases the vibration-related condition judgment to reduce the probability of false alarms.

[0035] refer to Figure 6 , based on the human body recognition results, obtain the distress information, whether it is dangerous information, and obtain dangerous instructions, including: S231, confirm that the person is in a state of asking for help based on the help information, S232, determine whether the human body appearance features are those of a child and whether the human body physiological features are those of a child, S233, when they are the appearance features of a child and the physiological features of a child, determine whether the actual physiological safety value information is outside the fifth preset physiological safety value information, if "yes", generate the fifth physiological danger instruction, S234, when they are not the appearance features of a child and the physiological features of a child, determine whether the actual physiological safety value information is outside the sixth preset physiological safety value information, if "yes", generate the sixth physiological danger instruction.

[0036] When it is determined that the child is in a state of seeking help, the appearance characteristics of the child and the physiological characteristics of the human body can be used to determine that the family members are in a state of double danger. Through the fifth physiological danger instruction and the sixth physiological danger instruction, a double alarm prompt will be sent to the terminal mobile device to inform the family members that they are in urgent need of help.

[0037] refer to Figure 7 , a family companion device 600, comprising: a data collection module 601, for collecting facial data, behavioral data, physiological data and environmental data; The danger judgment module 602 is used to process the physiological data to obtain the physiological recognition result, compare the actual physiological safety value information with the preset physiological safety value information, judge whether it is dangerous information, and obtain the danger instruction; The adjustment judgment module 603 is used to process the environmental data to obtain an environmental recognition result, compare the actual environmental value information with the preset environmental comfort value information, determine whether it is adjustment information, and obtain an adjustment instruction; The help-seeking judgment module 604 is used to process the behavior data and the facial data to obtain a human body recognition result, and judge whether it is a help-seeking message. When the facial data is recognized as a help-seeking facial data message, a first help-seeking instruction will be generated; when the behavior data is recognized as a help-seeking behavior data message, a second help-seeking instruction will be generated; The interactive response module 605 is used to feed back the first distress instruction, the second distress instruction, the danger instruction, and the adjustment instruction to the terminal mobile device to perform user services or emergency response.

[0038] In one embodiment, the distress judgment module 604 is used to obtain the current heart rate based on the physiological recognition results, and judge whether the recognized facial features are distress facial features. If they are distress facial features, judge whether the current heart rate is greater than the preset heart rate. If it is greater than the preset heart rate, the current heart rate will be timed and counted as the high heart rate duration. When the high heart rate duration is greater than the preset high heart rate duration, it is confirmed as a distress situation.

[0039] In one embodiment, the distress judgment module 604 is used to determine whether the identified behavior characteristics are distress behavior characteristics. If they are distress behavior characteristics, the time the distress behavior characteristics are maintained will be timed and counted as the duration of the distress behavior. When the duration of the distress behavior is greater than the preset duration of the distress behavior, it is confirmed as a distress situation.

[0040] In one embodiment, the danger judgment module 602 is used to obtain human body appearance features based on the environment recognition results, obtain human body physiological features based on the human body recognition results, judge the human body's age, and judge whether the human body is a child. If the human body is a child, it will judge whether the actual physiological safety value information is outside the first preset physiological safety value information. If "yes", a first physiological danger instruction will be generated. If the human body is not a child, it will judge whether the actual physiological safety value information is outside the second preset physiological safety value information. If "yes", a second physiological danger instruction will be generated.

[0041] In one embodiment, the adjustment judgment module 603 is used to determine whether the actual environmental value information is outside the preset environmental comfort range value information. When it is outside the preset environmental comfort range value information, a signal is sent to determine whether the environment needs to be adjusted. If a signal that the environment needs to be adjusted is received, it will be confirmed as a situation to be adjusted.

[0042] In one embodiment, the danger judgment module 602 is used to obtain actual vibration value information based on the environment recognition result. When the actual vibration value information is less than the preset vibration value information, it is judged whether the human body appearance feature is a child's appearance feature. When it is a child's appearance feature, it is judged whether the actual physiological safety value information is outside the third preset physiological safety value information. If "yes", a third physiological danger instruction will be generated. When it is not a child's appearance feature, it is judged whether the actual physiological safety value information is outside the fourth preset physiological safety value information. If "yes", a fourth physiological danger instruction will be generated.

[0043] In one embodiment, the danger judgment module 602 is used to obtain a distress message based on the human body recognition result, confirm that the person is in a distress state according to the distress message, and judge whether the human body appearance features are the appearance features of a child and whether the human body physiological features are the physiological features of a child. When they are the appearance features of a child and the physiological features of a child, it will be judged whether the actual physiological safety value information is outside the fifth preset physiological safety value information. If "yes", the fifth physiological danger instruction will be generated. When they are not the appearance features of a child and the physiological features of a child, it will be judged whether the actual physiological safety value information is outside the sixth preset physiological safety value information. If "yes", the sixth physiological danger instruction will be generated.

[0044] refer to Figure 8 In one embodiment, an electronic device is also provided, including a memory, a processor and a computer program, wherein the computer program is stored in the memory and executed by the processor to implement the steps in the above method embodiments.

[0045] A computer-readable storage medium stores a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0046] The above-mentioned controller is a part of the processor or exists independently. The processor or module can be a central processing unit (CPU), or one or more other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete hardware components, etc.; the modules can be integrated together and implemented in the form of a system-on-a-chip (SOC). The above-mentioned modules also include various sensors such as temperature, humidity, infrared, gas, etc., or devices that collect physical signals, as well as cameras, radars, wearable devices, etc. The analog signal input controller or processor is converted into a digital signal and outputs specific feedback to the execution hardware device.

[0047] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A family companionship method, characterized in that: include: Collect facial data, behavioral data, physiological data, and environmental data; Processing physiological data to obtain physiological recognition results, comparing actual physiological safety value information with preset physiological safety value information, determining whether it is dangerous information, and obtaining dangerous instructions; Process the environmental data to obtain the environmental recognition result, compare the actual environmental value information with the preset environmental comfort value information, determine whether it is adjustment information, and obtain the adjustment instruction; Processing the behavior data and facial data to obtain a human body recognition result, and determining whether it is a distress message. When the facial data is recognized as distress facial data information, a first distress instruction will be generated; when the behavior data is recognized as distress behavior data information, a second distress instruction will be generated; The first distress command, the second distress command, the danger command, and the adjustment command are fed back to the terminal mobile device to perform user services or emergency responses.

2. A family companionship method according to claim 1, characterized in that: Based on the physiological recognition result, the current heartbeat frequency is obtained. When the facial data is recognized as a distress facial data information, a first distress instruction will be generated, including: determining whether the recognized facial features are distress facial features, and when they are distress facial features, determining whether the current heartbeat frequency is greater than a preset heartbeat frequency. If it is greater than the preset heartbeat frequency, the current heartbeat frequency will be timed and counted as a high heartbeat frequency duration. When the high heartbeat frequency duration is greater than the preset high heartbeat frequency duration, it is confirmed as a distress situation; The facial features of asking for help include rolling eyes, seriousness, surprise, pain and fear.

3. A family companionship method according to claim 1, characterized in that: When the behavior data is identified as distress behavior data information, a second distress instruction will be generated, including: determining whether the identified behavior feature is a distress behavior feature, and if it is a distress behavior feature, timing the duration of the distress behavior feature as the duration of the distress behavior, and when the duration of the distress behavior is greater than the preset duration of the distress behavior, confirming it as a distress situation; The characteristics of the help-seeking behavior include holding the head with both hands, raising both hands, and falling down.

4. A family companionship method according to claim 1, characterized in that: Based on the environmental recognition results, the human body's appearance characteristics are obtained, based on the human body recognition results, the human body's physiological characteristics are obtained, and the human body's age is judged; Determine whether it is dangerous information and obtain dangerous instructions, including: When judging whether the human body is a child, if it is a child, it will be judged whether the actual physiological safety value information is outside the first preset physiological safety value information. If "yes", a first physiological danger instruction will be generated. When it is not a child, it will be judged whether the actual physiological safety value information is outside the second preset physiological safety value information. If "yes", a second physiological danger instruction will be generated.

5. A family companionship method according to claim 1, characterized in that: Determine whether it is adjustment information and obtain adjustment instructions, including: It is determined whether the actual environment value information is outside the preset environment comfort range value information. If it is outside the preset environment comfort range value information, a signal is sent to determine whether the environment needs to be adjusted. If a signal is received to determine whether the environment needs to be adjusted, it is confirmed as a situation to be adjusted.

6. A family companionship method according to claim 1 or 4, characterized in that: Acquire actual vibration value information based on the environmental recognition result; Determine whether it is dangerous information and obtain dangerous instructions, including: When the actual vibration value information is less than the preset vibration value information, it is determined whether the human body appearance features are those of a child. If they are those of a child, it is determined whether the actual physiological safety value information is outside the third preset physiological safety value information. If so, a third physiological danger instruction is generated. When it is not a child's appearance features, it is determined whether the actual physiological safety value information is outside the fourth preset physiological safety value information. If so, a fourth physiological danger instruction is generated.

7. A family companionship method according to claim 1 or 4, characterized in that: Obtaining distress information based on human body recognition results; Determine whether it is dangerous information and obtain dangerous instructions, including: According to the distress message, confirm that the person is in a distress state, judge whether the human body appearance features are the appearance features of a child and whether the human body physiological features are the physiological features of a child. When they are the appearance features of a child and the physiological features of a child, judge whether the actual physiological safety value information is outside the fifth preset physiological safety value information. If "yes", generate the fifth physiological danger instruction. When they are not the appearance features of a child and the physiological features of a child, judge whether the actual physiological safety value information is outside the sixth preset physiological safety value information. If "yes", generate the sixth physiological danger instruction.

8. A family companion device, characterized in that: include: Data collection module, used to collect facial data, behavioral data, physiological data and environmental data; The danger judgment module processes the physiological data to obtain the physiological recognition result, compares the actual physiological safety value information with the preset physiological safety value information, determines whether it is dangerous information, and obtains the danger instruction; The adjustment judgment module processes the environmental data to obtain the environmental recognition result, compares the actual environmental value information with the preset environmental comfort value information, determines whether it is adjustment information, and obtains the adjustment instruction; The help judgment module processes the behavior data and facial data to obtain the human body recognition result, and judges whether it is a help message. When the facial data is recognized as a help facial data information, a first help instruction will be generated; when the behavior data is recognized as a help behavior data information, a second help instruction will be generated; The interactive response module feeds back the first distress instruction, the second distress instruction, the danger instruction, and the adjustment instruction to the terminal mobile device to perform user services or emergency response.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor executes to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images