User state detection method and device, earphone and storage medium

Abstract

The invention relates to a user state detection method and device, an earphone and a storage medium. And acquiring PPG photoplethysmography data of the user, wherein the PPG data is acquired by a PPG sensor carried on an earphone worn by the user. And then, analyzing the PPG data to obtain the physiological indexes of the user. Then, according to the physiological indexes of the user, the brain state of the user is determined; and finally, generating feedback information according to the brain state of the user, and sending the feedback information to the earphone so as to carry out voice reminding on the user. Therefore, the convenience of detecting the user state and the accuracy of determining the brain state are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for detecting a user status, a headset, and a storage medium. Background Art

[0002] Currently, detecting user status relies on visual solutions or complex and expensive equipment. For example, in driving scenarios, in-car cameras capture and analyze facial images of the user to determine whether the user is focused or distracted. However, cameras are not always available in other scenarios, making this approach unsuitable for daily use.

[0003] Another example is an electroencephalogram (EEG) device, which uses multiple electrodes to detect a user's bioelectrical signals and determine their brain state. However, EEG devices are too complex and expensive to be used daily.

[0004] Therefore, how to conveniently detect user status is a problem that needs to be solved urgently. Summary of the Invention

[0005] The present disclosure provides a method, device, earphone and storage medium for detecting user status to address the deficiencies in related technologies.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for detecting a user status is provided, comprising:

[0007] Obtaining PPG photoplethysmography data of the user, where the PPG data is collected by a PPG sensor mounted on headphones worn by the user;

[0008] Analyzing the PPG data to obtain physiological indicators of the user;

[0009] determining the user's brain state based on the user's physiological indicators;

[0010] Feedback information is generated according to the brain state of the user, and the feedback information is sent to the headset to provide a voice reminder to the user.

[0011] Optionally, the physiological indicator includes: blood oxygen saturation; the mobile terminal maintains a correspondence between at least one brain state and a value range of the blood oxygen saturation;

[0012] Determining the user's brain state based on the user's physiological indicators includes:

[0013] The correspondence between the at least one brain state and the value range of the blood oxygen saturation is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation.

[0014] Optionally, the physiological indicator further includes: heart rate amplitude; the mobile terminal maintains a correspondence between at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude;

[0015] Determining the user's brain state based on the user's physiological indicators includes:

[0016] The correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation and the value range of the heart rate amplitude.

[0017] Optionally, the physiological indicator further includes: respiratory rate; the mobile terminal maintains a correspondence between at least one brain state, a value range of the blood oxygen saturation, and a value range of the respiratory rate;

[0018] Determining the user's brain state based on the user's physiological indicators includes:

[0019] The correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the respiratory rate is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation and the value range of the respiratory rate.

[0020] Optionally, a prediction model for predicting the user's brain state based on the user's physiological indicators is run on the mobile terminal; the prediction model is obtained by supervised training based on a number of physiological indicator samples labeled with brain states;

[0021] Determining the user's brain state based on the user's physiological indicators includes:

[0022] Inputting the user's physiological indicators into a pre-trained prediction model and obtaining a prediction result output by the prediction model;

[0023] Based on the prediction result, the brain state of the user is determined.

[0024] Optionally, before generating feedback information according to the user's brain state, the method further includes:

[0025] Acquiring location information and movement status information of a mobile terminal connected to the headset;

[0026] Determining a user scenario in which the mobile terminal is located based on the location information and the mobility status information;

[0027] Generate feedback information based on the user's brain state, including:

[0028] Feedback information is generated according to the user's brain state and a feedback strategy corresponding to the user scenario.

[0029] Optionally, different brain states correspond to different duration thresholds.

[0030] Optionally, the brain state includes a focused state, a non-focused state, and a tired state;

[0031] Generate feedback information based on the user's brain state, including:

[0032] If the duration of the user being in the focused state reaches a duration threshold corresponding to the focused state, generating feedback information to remind the user to take a break;

[0033] If the user is in a tired state, generating feedback information to remind the user to take a rest;

[0034] If the duration of the user being in the non-concentrated state reaches a duration threshold corresponding to the non-concentrated state, feedback information is generated to remind the user to concentrate on learning.

[0035] Optionally, the PPG data includes a timestamp corresponding to a collection moment, and the method further includes:

[0036] According to the timestamp corresponding to the acquisition moment, the duration of the user being in various brain states is counted, a user feedback report is generated, and the user feedback report is displayed to the user.

[0037] According to a second aspect of an embodiment of the present disclosure, a device for detecting a user status is provided, the device comprising:

[0038] an acquisition module, configured to acquire PPG photoplethysmography data of a user, the PPG data being collected by a PPG sensor mounted on headphones worn by the user;

[0039] An analysis module, configured to analyze the PPG data to obtain physiological indicators of the user;

[0040] a determination module, configured to determine the user's brain state based on the user's physiological indicators;

[0041] A generating module is used to generate feedback information according to the brain state of the user, and send the feedback information to the earphone to provide a voice reminder to the user.

[0042] Optionally, the physiological indicator includes: blood oxygen saturation; the mobile terminal maintains a correspondence between at least one brain state and a value range of the blood oxygen saturation; the determination module includes:

[0043] A query module is used to query the correspondence between the at least one brain state and the value range of the blood oxygen saturation, and determine the user's brain state based on the value range of the user's blood oxygen saturation.

[0044] Optionally, the physiological indicator further includes: heart rate amplitude; the mobile terminal maintains a correspondence between at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude; and the determination module includes:

[0045] A query module is configured to query a correspondence between the at least one brain state, the blood oxygen saturation value range, and the heart rate amplitude value range, and determine the user's brain state based on the value range of the user's blood oxygen saturation and the value range of the heart rate amplitude.

[0046] Optionally, the physiological indicator further includes: respiratory rate; the mobile terminal maintains a correspondence between at least one brain state, a value range of the blood oxygen saturation, and a value range of the respiratory rate; and the determination module includes:

[0047] A query module is used to query the correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the respiratory rate, and determine the user's brain state based on the value range of the user's blood oxygen saturation and the value range of the respiratory rate.

[0048] Optionally, a prediction model for predicting the user's brain state based on the user's physiological indicators is run on the mobile terminal; the prediction model is obtained by supervised training based on a number of physiological indicator samples labeled with brain states; and the determination module includes:

[0049] An input module, configured to input the user's physiological indicators into a pre-trained prediction model and obtain a prediction result output by the prediction model;

[0050] A determination module is used to determine the brain state of the user based on the prediction result.

[0051] Optionally, the device further comprises:

[0052] An acquisition module, configured to acquire location information and movement status information of a mobile terminal connected to the headset;

[0053] a determination module, configured to determine a user scenario in which the mobile terminal is located based on the location information and the mobility status information;

[0054] The generation module includes:

[0055] The generating submodule is used to generate feedback information according to the user's brain state and the feedback strategy corresponding to the user scenario.

[0056] Optionally, the PPG data includes a timestamp corresponding to a collection moment; and the generating module includes:

[0057] a determination module, configured to determine a duration of the user being in the current brain state based on the timestamp corresponding to the acquisition moment;

[0058] The generating submodule is configured to generate feedback information if the duration of the user being in the current brain state reaches a duration threshold corresponding to the current brain state.

[0059] Optionally, different brain states correspond to different duration thresholds.

[0060] Optionally, the brain state includes a focused state, a non-focused state, and a tired state; and the generating module includes:

[0061] A first generating module is configured to generate feedback information to remind the user to take a break if the duration of the user being in the focused state reaches a duration threshold corresponding to the focused state;

[0062] The second generating module is configured to generate feedback information to remind the user to take a rest if the user is in a tired state;

[0063] The third generating module is configured to generate feedback information to remind the user to focus on learning if the duration of the user being in the non-focused state reaches a duration threshold corresponding to the non-focused state.

[0064] Optionally, the PPG data includes a timestamp corresponding to a collection time, and the apparatus further includes:

[0065] The display module is used to count the duration of the user's various brain states according to the timestamp corresponding to the acquisition time, generate a user feedback report, and display the user feedback report to the user.

[0066] According to a third aspect of an embodiment of the present disclosure, there is provided an earphone equipped with a PPG sensor, comprising:

[0067] processor;

[0068] a memory for storing processor-executable instructions;

[0069] Wherein, the processor is used to implement the above-mentioned method for detecting user status.

[0070] According to a fourth aspect of an embodiment of the present disclosure, a wearable device is provided, wherein the wearable device is equipped with a PPG sensor, including:

[0071] processor;

[0072] a memory for storing processor-executable instructions;

[0073] Wherein, the processor is used to implement the above-mentioned method for detecting user status.

[0074] According to a fifth aspect of an embodiment of the present disclosure, a mobile terminal is provided, wherein the mobile terminal establishes a connection with a headset equipped with a PPG sensor, including:

[0075] processor;

[0076] a memory for storing processor-executable instructions;

[0077] Wherein, the processor is used to implement the above-mentioned method for detecting user status.

[0078] According to a sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it is used to implement the above-mentioned method for detecting user status.

[0079] The technical solutions provided by the embodiments of the present disclosure may at least have the following beneficial effects:

[0080] According to the embodiments of the present disclosure, a user's PPG photoplethysmography (PPG) data is acquired from a PPG sensor mounted on the user's headphones. This PPG data is then analyzed to obtain the user's physiological indicators. The user's brain state is then determined based on these indicators. Finally, feedback information is generated based on the user's brain state and sent to the headphones to provide a voice reminder. This improves the convenience of detecting the user's state and the accuracy of the determined brain state. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0082] Figure 1 It is a schematic flow chart of a method for detecting user status according to an embodiment of the present disclosure.

[0083] Figure 2 The figure is a schematic diagram showing data transmission between a mobile terminal and a headset according to an embodiment of the present disclosure.

[0084] Figure 3 is based on Figure 1 A schematic diagram of another method for detecting user status based on the illustrated embodiment is shown.

[0085] Figure 4 is based on Figure 1 The illustrated embodiment is based on a schematic diagram of another method for detecting a user status.

[0086] Figure 5 is based on Figure 1 A schematic diagram of another method for detecting user status based on the illustrated embodiment is shown.

[0087] Figure 6 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the illustrated embodiment is shown.

[0088] Figure 7 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the illustrated embodiment is shown.

[0089] Figure 8 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the illustrated embodiment is shown.

[0090] Figure 9 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the illustrated embodiment is shown.

[0091] Figure 10 is based on Figure 1 A schematic diagram of another method for detecting user status based on the illustrated embodiment is shown.

[0092] Figure 11 This is a schematic block diagram of a device for detecting a user status according to an embodiment of the present disclosure.

[0093] Figure 12 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0094] Figure 13is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0095] Figure 14 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0096] Figure 15 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0097] Figure 16 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0098] Figure 17 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0099] Figure 18 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0100] Figure 19 is based on Figure 11 A schematic block diagram of another device for detecting a user status is shown based on the illustrated embodiment.

[0101] Figure 20 2 is a schematic block diagram of a device 2000 for detecting user status according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0102] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0103] Currently, detecting user status relies on visual solutions or complex and expensive equipment. For example, in driving scenarios, in-car cameras capture and analyze facial images of the user to determine whether the user is focused or distracted. However, cameras are not always available in other scenarios, making this approach unsuitable for daily use.

[0104] Another example is an electroencephalogram (EEG) device, which uses multiple electrodes to detect a user's bioelectrical signals and determine their brain state. However, EEG devices are too complex and expensive to be used daily.

[0105] See Figure 1 , Figure 1 FIG. 1 is a schematic flow chart of a method for detecting a user status according to an embodiment of the present disclosure. Figure 1 As shown, the method for detecting the user status may include the following steps:

[0106] In step 102, PPG photoplethysmography data of the user is obtained, where the PPG data is collected by a PPG sensor mounted on headphones worn by the user.

[0107] In step 104, the PPG data is analyzed to obtain the user's physiological indicators.

[0108] In practical applications, PPG sensors are often integrated into devices worn on the wrist or finger, such as smart watches, smart bracelets, and blood oximeters. However, when the PPG sensor is located on the wrist or finger, the wrist may move frequently, causing the PPG sensor to be subject to more motion interference. In addition, since ambient light interference can affect the optical measurement of the PPG sensor, when the PPG sensor is located on the wrist or finger, the PPG sensor is also subject to more light interference.

[0109] Compared to other body parts (wrists, fingers, etc.), the ear canal is less subject to frequent friction or external forces caused by movement. Furthermore, the ear canal is relatively deep, receiving less interference from external light. Furthermore, without large muscles or bones blocking the ear canal, the skin of the ear canal is generally more accessible to blood flow than the skin on the wrist.

[0110] Based on this, integrating the PPG sensor into the earphones can make it easier to obtain a stable PPG signal, thereby improving the accuracy of the PPG sensor.

[0111] In one possible implementation, PPG photoplethysmography data of the user is obtained. The PPG data mentioned here is collected by a PPG sensor mounted on headphones worn by the user.

[0112] Then, the PPG data is analyzed to obtain the user's physiological indicators, including blood oxygen saturation, heart rate amplitude, and respiratory rate.

[0113] The method for determining heart rate amplitude is as follows: heart rate is calculated using photoplethysmography (PPG). Photoplethysmography uses a PPG sensor to detect differences in the intensity of reflected light after absorption by blood and tissue, charting changes in vascular volume during the cardiac cycle. Heart rate is then calculated from the resulting pulse waveform.

[0114] The method for determining respiratory rate is as follows: the user's PPG data is input into a pre-trained respiratory rate prediction model to determine the user's respiratory rate. The respiratory rate prediction model mentioned here is obtained through supervised training based on a number of labeled respiratory rate samples.

[0115] The method for determining blood oxygen saturation is as follows: The oxygenated hemoglobin and hemoglobin in the blood are measured. Since oxygenated hemoglobin has a higher absorption coefficient for light with a wavelength of 800-1000nm, while hemoglobin has a higher absorption coefficient for light with a wavelength of 600-800nm, the PPG signals of oxygenated hemoglobin and hemoglobin can be detected using red light (600-800nm) and light near IR (800-1000nm), respectively. The blood oxygen saturation is then calculated by calculating the ratio of oxygenated hemoglobin to hemoglobin in the blood.

[0116] In step 106, the user's brain state is determined based on the user's physiological indicators.

[0117] In step 108, feedback information is generated according to the brain state of the user, and the feedback information is sent to the headset to provide a voice reminder to the user.

[0118] In practical applications, detecting user status relies on visual solutions or complex and expensive equipment, which is not suitable for daily use.

[0119] Based on this, the user's brain state can be determined based on the PPG data collected by the currently more common PPG sensors, thereby improving the convenience of detecting the user's state.

[0120] In one possible implementation, the user's brain state is determined based on the user's physiological indicators. The brain state mentioned here includes: focused state, non-focused state, fatigue state, etc.

[0121] Then, feedback information is generated based on the user's brain state and sent to the headset to provide voice reminders to the user.

[0122] For example, if the user's brain state is fatigued, feedback information is generated to remind the user to take a break, and a voice reminding the user to take a break is played through the earphones.

[0123] For another example, if the user's brain state is a non-concentrated state, feedback information is generated to remind the user to concentrate on studying, and a voice reminding the user to study is played through the earphones.

[0124] As can be seen from the above embodiments, the PPG data is analyzed to obtain the user's physiological indicators, and the user's brain state is further determined based on the user's physiological indicators, thereby improving the convenience of detecting the user's state.

[0125] See Figure 2 , Figure 2 The figure is a schematic diagram showing data transmission between a mobile terminal and a headset according to an embodiment of the present disclosure.

[0126] exist Figure 2 In this case, when a headset equipped with a PPG sensor lacks a processor, a mobile terminal can establish a connection with the headset and obtain the user's PPG data. The mobile terminal then analyzes the PPG data and generates feedback information. Finally, the feedback information is sent to the headset to provide a voice reminder to the user.

[0127] Of course, when there is a processor in the earphones equipped with a PPG sensor, the earphones can analyze the PPG data, and ultimately generate feedback information and play the feedback information to the user.

[0128] Furthermore, in addition to headphones, other wearable devices can also be equipped with PPG sensors to obtain users' PPG data, such as smart glasses and VR helmets.

[0129] See Figure 3 , Figure 3 is based on Figure 1 FIG1 is a schematic diagram of another method for detecting user status based on the embodiment shown. Figure 3 As shown, the method may further include:

[0130] In step 302, the correspondence between the at least one brain state and the value range of the blood oxygen saturation is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation.

[0131] In actual applications, when a user is focused, the user's brain consumes more oxygen, so the detected blood oxygen saturation is lower. When the user is not focused, the user's brain consumes less oxygen, so the detected blood oxygen saturation is higher.

[0132] Based on this, the mobile terminal can maintain a correspondence between at least one brain state and a value range of blood oxygen saturation to determine the user's brain state.

[0133] In one possible implementation, a correspondence between at least one brain state and a value range of blood oxygen saturation is queried, and the user's brain state is determined based on the value range of the user's blood oxygen saturation.

[0134] For example, the focused state corresponds to a first blood oxygen saturation value range, and the non-focused state corresponds to a second blood oxygen saturation value range. When the user's blood oxygen saturation is within the first blood oxygen saturation value range, the user's brain state is determined to be focused. When the user's blood oxygen saturation is within the second blood oxygen saturation value range, the user's brain state is determined to be non-focused. The first blood oxygen saturation value range and the second blood oxygen saturation value range are determined based on expert experience.

[0135] See Figure 4 , Figure 4 is based on Figure 1 A schematic diagram of another method for detecting user status is shown based on the embodiment shown. Figure 3 As shown, the method may include:

[0136] In step 402, the correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude is queried, and the user's brain state is determined based on the value range of the user's blood oxygen saturation and the value range of the heart rate amplitude.

[0137] In actual applications, when a user is focused, the detected blood oxygen saturation is low and the user's heart rate amplitude is high. When the user is not focused, the detected blood oxygen saturation is high and the user's heart rate amplitude is low. When the user is tired, the detected blood oxygen saturation is low and the user's heart rate amplitude is low.

[0138] Based on this, the mobile terminal can maintain a correspondence between at least one brain state, a value range of blood oxygen saturation, and a value range of heart rate amplitude to determine the user's brain state.

[0139] In one possible implementation, a correspondence between at least one brain state, a range of blood oxygen saturation values, and a range of heart rate amplitude values is queried, and the user's brain state is determined based on the range of values of the user's blood oxygen saturation and the range of values of the heart rate amplitude.

[0140] For example, the focused state corresponds to a first blood oxygen saturation value range and a first heart rate amplitude value range, the non-focused state corresponds to a second blood oxygen saturation value range and a second heart rate amplitude value range, and the fatigued state corresponds to the first blood oxygen saturation value range and a second heart rate amplitude value range. When the user's blood oxygen saturation is within the first blood oxygen saturation value range and the heart rate amplitude is within the first heart rate amplitude value range, the user's brain state is determined to be focused. When the user's blood oxygen saturation is within the second blood oxygen saturation value range and the heart rate amplitude is within the second heart rate amplitude value range, the user's brain state is determined to be non-focused. When the user's blood oxygen saturation is within the first blood oxygen saturation value range and the heart rate amplitude is within the second heart rate amplitude value range, the user's brain state is determined to be fatigued. The first blood oxygen saturation value range, the second blood oxygen saturation value range, the first heart rate amplitude value range, and the second heart rate amplitude value range are determined based on expert experience.

[0141] See Figure 5 , Figure 5 is based on Figure 1 FIG1 is a schematic diagram of another method for detecting user status based on the embodiment shown. Figure 5 As shown, the method may further include:

[0142] In step 502, the correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the respiratory rate is queried, and the user's brain state is determined based on the value range of the user's blood oxygen saturation and the value range of the respiratory rate.

[0143] In actual applications, when the user is in a focused state, the detected blood oxygen saturation of the user is low and the user's breathing rate is low. When the user is not focused, the detected blood oxygen saturation of the user is high and the user's breathing rate is high.

[0144] Based on this, the mobile terminal can maintain a correspondence between at least one brain state, a value range of blood oxygen saturation, and a value range of respiratory rate, so as to determine the user's brain state.

[0145] In one possible implementation, a correspondence between at least one brain state, a value range of blood oxygen saturation, and a value range of respiratory rate is queried, and the user's brain state is determined based on the value range of the user's blood oxygen saturation and the value range of the respiratory rate.

[0146] For example, the focused state corresponds to a first blood oxygen saturation value range and a first respiratory rate value range, and the non-focused state corresponds to a second blood oxygen saturation value range and a second respiratory rate value range. When the user's blood oxygen saturation is within the first blood oxygen saturation value range and the heart rate amplitude is within the first heart rate amplitude value range, it is determined that the user's brain state is a focused state. When the user's blood oxygen saturation is within the second blood oxygen saturation value range and the heart rate amplitude is within the second heart rate amplitude value range, it is determined that the user's brain state is a non-focused state. Among them, the first blood oxygen saturation value range, the second blood oxygen saturation value range, the first heart rate amplitude value range and the second heart rate amplitude value range are determined by expert experience.

[0147] Furthermore, in a possible implementation, the correspondence between at least one brain state, the value range of blood oxygen saturation, the value range of heart rate amplitude, and the value range of respiratory rate is queried, and the user's brain state is determined based on the value range of the user's blood oxygen saturation, the value range of the heart rate amplitude, and the value range of the respiratory rate.

[0148] See Figure 6 , Figure 6 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the embodiment shown is shown. Figure 5 As shown, the method may further include:

[0149] In step 602, the user's physiological indicators are input into a pre-trained prediction model, and a prediction result output by the prediction model is obtained.

[0150] In step 604, the user's brain state is determined based on the prediction result.

[0151] In one possible implementation, a prediction model for predicting a user's brain state based on the user's physiological indicators is run on the mobile terminal. The prediction model is obtained through supervised training based on a number of physiological indicator samples labeled with brain states.

[0152] Then, the user's physiological indicators are input into the pre-trained prediction model, and the prediction results output by the prediction model are obtained.

[0153] Next, based on the prediction results, the user's brain state is determined.

[0154] It can be seen from the above embodiments that a more accurate user's brain state can be obtained through a pre-trained prediction model.

[0155] See Figure 7 , Figure 7 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the embodiment shown is shown. Figure 7 As shown, the method may further include:

[0156] In step 702, the location information and movement status information of the mobile terminal connected to the headset are obtained.

[0157] In step 704, the user scenario in which the mobile terminal is located is determined based on the location information and the movement status information.

[0158] In step 706, feedback information is generated according to the user's brain state and a feedback strategy corresponding to the user scenario.

[0159] In practice, different user scenarios require different feedback strategies. For example, in a library setting, a user might need a voice reminder to take a break after 45 minutes of focused study, or a voice reminder to study after 10 minutes of distraction. Another example is a driving scenario where a user might need a voice reminder to stop and rest after 60 minutes of focused driving, or a voice reminder to focus on driving after 1 minute of distraction.

[0160] Based on this, it is necessary to first determine the user scenario in which the user is located, and then generate feedback information based on the feedback strategy of the user scenario in which the user is located.

[0161] In a possible implementation, location information and movement status information of a mobile terminal connected to the headset are obtained.

[0162] Then, the user scenario of the mobile terminal is determined based on the location information and the movement status information.

[0163] For example, if the location information is a library and the movement state information is stationary, the user scenario is determined to be a library scenario. For another example, if the location information is a national highway and the movement state information is high-speed, the user scenario is determined to be a driving scenario.

[0164] Next, feedback information is generated based on the user's brain state and the feedback strategy corresponding to the user scenario.

[0165] For example, in a library scenario, a user may need a voice reminder to take a break after 45 minutes of focused attention, or a voice reminder to study after 10 minutes of inattention. Another example is a driving scenario where a user may need a voice reminder to stop and rest after 60 minutes of focused attention, or a voice reminder to focus on driving after 1 minute of inattention.

[0166] See Figure 8 , Figure 8 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the embodiment shown is shown. Figure 8 As shown, the method may further include:

[0167] In step 802, the duration of the user's current brain state is determined based on the timestamp corresponding to the acquisition moment.

[0168] In step 804, if the duration of the user being in the current brain state reaches a duration threshold corresponding to the current brain state, feedback information is generated.

[0169] In one possible implementation, the duration of the user's current brain state is determined based on a timestamp corresponding to the acquisition moment.

[0170] Then, if the duration of the user being in the current brain state reaches a duration threshold corresponding to the current brain state, feedback information is generated.

[0171] Different brain states correspond to different duration thresholds. For example, the duration threshold for a focused state is 45 minutes. If the user stays in a focused state for 45 minutes, feedback is generated to remind the user to take a break. For another example, the duration threshold for a non-focused state is 10 minutes. If the user stays in a non-focused state for 10 minutes, feedback is generated to remind the user to study.

[0172] See Figure 9 , Figure 9 is based on Figure 1 A schematic flow chart of another method for detecting user status based on the embodiment shown is shown. Figure 9 As shown, the method may further include:

[0173] In step 902, if the duration of the user being in the focused state reaches a duration threshold corresponding to the focused state, feedback information is generated to remind the user to take a break.

[0174] In step 904, if the user is in a tired state, feedback information is generated to remind the user to take a rest.

[0175] In step 906, if the duration of the user being in the non-concentrated state reaches a duration threshold corresponding to the non-concentrated state, feedback information is generated to remind the user to concentrate on learning.

[0176] In a possible implementation, if the duration of the user being in the focused state reaches a duration threshold corresponding to the focused state, feedback information is generated to remind the user to take a break.

[0177] If the user is in a tired state, feedback information is generated to remind the user to take a break.

[0178] If the duration of the user being in the non-concentrated state reaches a duration threshold corresponding to the non-concentrated state, feedback information is generated to remind the user to concentrate on learning.

[0179] It should be noted that this specification does not limit brain states and the feedback information corresponding to brain states. In addition to the brain states and feedback information corresponding to brain states mentioned above, technicians can also set other brain states and feedback information corresponding to other brain states based on other situations that the user may encounter. For example, when the user encounters a dangerous situation, the user's physiological indicators will fluctuate violently. The user's brain state can be set to an emergency state, and feedback information can be generated asking the user whether to call the emergency number or the police.

[0180] See Figure 10 , Figure 10 is based on Figure 1 FIG1 is a schematic diagram of another method for detecting user status based on the embodiment shown. Figure 10 As shown, the method may further include:

[0181] In step 1002, the duration of the user's stay in various brain states is counted based on the timestamp corresponding to the acquisition moment, a user feedback report is generated, and the user feedback report is displayed to the user.

[0182] In one possible implementation, the duration of the user's stay in various brain states is counted based on the timestamp corresponding to the acquisition moment, a user feedback report is generated, and the user feedback report is displayed to the user.

[0183] As can be seen from the above embodiments, the user feedback report can show the distribution of the user's brain state over a period of time, helping the user to understand their own state and take corresponding improvement measures to improve their concentration.

[0184] Corresponding to the aforementioned embodiment of the method for detecting user status, the present disclosure also provides an embodiment of an apparatus for detecting user status.

[0185] See Figure 11 , Figure 11 FIG. 1 is a schematic block diagram of a device for detecting a user state according to an embodiment of the present disclosure. Figure 11 As shown, the device may include:

[0186] an acquisition module 1102 for acquiring PPG photoplethysmography data of a user, where the PPG data is collected by a PPG sensor mounted on headphones worn by the user;

[0187] An analysis module 1104 is configured to analyze the PPG data to obtain physiological indicators of the user;

[0188] a determination module 1106, configured to determine the user's brain state based on the user's physiological indicators;

[0189] The generating module 1108 is used to generate feedback information according to the brain state of the user, and send the feedback information to the earphone to provide a voice reminder to the user.

[0190] See Figure 12 , Figure 12 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 12 As shown, the determination module includes:

[0191] The query module 1202 is used to query the correspondence between the at least one brain state and the value range of the blood oxygen saturation, and determine the brain state of the user based on the value range of the user's blood oxygen saturation.

[0192] See Figure 13 , Figure 13 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 13 As shown, the determination module includes:

[0193] The query module 1302 is used to query the correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude, and determine the user's brain state based on the value range of the user's blood oxygen saturation and the value range of the heart rate amplitude.

[0194] See Figure 14 , Figure 14 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 14 As shown, the determination module includes:

[0195] The query module 1402 is used to query the correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the respiratory rate, and determine the user's brain state based on the value range of the user's blood oxygen saturation and the value range of the respiratory rate.

[0196] See Figure 15 , Figure 15 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 15 As shown, the determination module includes:

[0197] An input module 1502 is configured to input the user's physiological indicators into a pre-trained prediction model and obtain a prediction result output by the prediction model;

[0198] The determination module 1504 is configured to determine the user's brain state based on the prediction result.

[0199] See Figure 16 , Figure 16 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 16 As shown, the device also includes:

[0200] An acquisition module 1602 is configured to acquire location information and movement status information of a mobile terminal connected to the headset;

[0201] A determination module 1604 is configured to determine a user scenario in which the mobile terminal is located based on the location information and the mobility status information;

[0202] The generation module includes:

[0203] The generating submodule 1606 is configured to generate feedback information according to the user's brain state and a feedback strategy corresponding to the user scenario.

[0204] See Figure 17 , Figure 17 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 17 As shown, the generation module includes:

[0205] a determination module 1702, configured to determine a duration of the user being in the current brain state based on the timestamp corresponding to the acquisition moment;

[0206] The generating submodule 1704 is configured to generate feedback information if the duration of the user being in the current brain state reaches a duration threshold corresponding to the current brain state.

[0207] See Figure 18 , Figure 18 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 18 As shown, the generation module includes:

[0208] The first generating module 1802 is configured to generate feedback information to remind the user to take a break if the duration of the user being in the focused state reaches a duration threshold corresponding to the focused state;

[0209] The second generating module 1804 is configured to generate feedback information to remind the user to take a rest if the user is in a tired state;

[0210] The third generating module 1806 is configured to generate feedback information to remind the user to focus on learning if the duration of the user being in the non-focused state reaches a duration threshold corresponding to the non-focused state.

[0211] See Figure 19 , Figure 19 is based on Figure 11 A schematic block diagram of another device for detecting user status is shown based on the embodiment shown. Figure 19 As shown, the device also includes:

[0212] The display module 1902 is used to count the duration of the user being in various brain states according to the timestamp corresponding to the acquisition time, generate a user feedback report, and display the user feedback report to the user.

[0213] Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments of the relevant methods and will not be elaborated on here.

[0214] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0215] Accordingly, the present disclosure further provides an earphone equipped with a PPG sensor, comprising: a processor;

[0216] a memory for storing processor-executable instructions;

[0217] Wherein, the processor is used to implement the above-mentioned method for detecting user status.

[0218] Accordingly, the present disclosure further provides a wearable device equipped with a PPG sensor, comprising: a processor;

[0219] a memory for storing processor-executable instructions;

[0220] Wherein, the processor is used to implement the above-mentioned method for detecting user status.

[0221] Accordingly, the present disclosure further provides a mobile terminal, which establishes a connection with a headset equipped with a PPG sensor, including: a processor;

[0222] a memory for storing processor-executable instructions;

[0223] Wherein, the processor is used to implement the above-mentioned method for detecting user status.

[0224] Correspondingly, the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for detecting the user status is implemented.

[0225] like Figure 20 As shown, Figure 20 2 is a schematic block diagram of an apparatus 2000 for detecting user status according to an embodiment of the present disclosure. For example, the apparatus 2000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0226] Reference Figure 20 , the device 2000 may include one or more of the following components: a processing component 2002 , a memory 2004 , a power component 2006 , a multimedia component 2008 , an audio component 2010 , an input / output (I / O) interface 2012 , a sensor component 2014 , and a communication component 2016 .

[0227] The processing component 2002 generally controls the overall operation of the device 2000, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 2002 may include one or more processors 2020 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 2002 may include one or more modules to facilitate interaction between the processing component 2002 and other components. For example, the processing component 2002 may include a multimedia module to facilitate interaction between the multimedia component 2008 and the processing component 2002.

[0228] The memory 2004 is configured to store various types of data to support the operations of the device 2000. Examples of such data include instructions for any application or method operating on the device 2000, contact data, phone book data, messages, pictures, videos, etc. The memory 2004 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0229] The power supply component 2006 provides power to the various components of the device 2000. The power supply component 2006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 2000.

[0230] The multimedia component 2008 includes a screen that provides an output interface between the device 2000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 2008 includes a front camera and / or a rear camera. When the device 2000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0231] The audio component 2010 is configured to output and / or input audio signals. For example, the audio component 2010 includes a microphone (MIC) that is configured to receive external audio signals when the device 2000 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 2004 or transmitted via the communication component 2016. In some embodiments, the audio component 2010 further includes a speaker for outputting audio signals.

[0232] I / O interface 2012 provides an interface between processing component 2002 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0233] Sensor assembly 2014 includes one or more sensors for providing various aspects of the status assessment of device 2000. For example, sensor assembly 2014 can detect the open / closed state of device 2000, the relative positioning of components, such as the display and keypad of device 2000. Sensor assembly 2014 can also detect changes in the position of device 2000 or a component of device 2000, the presence or absence of user contact with device 2000, the orientation or acceleration / deceleration of device 2000, and changes in the temperature of device 2000. Sensor assembly 2014 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 2014 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 2014 can also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0234] The communication component 2016 is configured to facilitate wired or wireless communication between the device 2000 and other devices. The device 2000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR or a combination thereof. In an exemplary embodiment, the communication component 2016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 2016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0235] In an exemplary embodiment, the apparatus 2000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the method described in any of the above embodiments.

[0236] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 2004 including instructions, and the instructions can be executed by the processor 2020 of the apparatus 2000 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0237] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0238] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0239] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0240] The above is a detailed introduction to the methods and devices provided in the embodiments of the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the methods and core ideas of the present disclosure. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present disclosure.

Claims

1. A method for detecting user status, characterized in that: include: Obtaining PPG photoplethysmography data of the user, where the PPG data is collected by a PPG sensor mounted on headphones worn by the user; Analyzing the PPG data to obtain physiological indicators of the user; determining the user's brain state based on the user's physiological indicators; Feedback information is generated according to the brain state of the user, and the feedback information is sent to the headset to provide a voice reminder to the user.

2. The method according to claim 1, characterized in that The physiological indicator includes: blood oxygen saturation; the mobile terminal maintains a correspondence between at least one brain state and a value range of the blood oxygen saturation; Determining the user's brain state based on the user's physiological indicators includes: The correspondence between the at least one brain state and the value range of the blood oxygen saturation is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation.

3. The method according to claim 2, characterized in that The physiological indicators further include: heart rate amplitude; the mobile terminal maintains a correspondence between at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude; Determining the user's brain state based on the user's physiological indicators includes: The correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the heart rate amplitude is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation and the value range of the heart rate amplitude.

4. The method according to claim 2, characterized in that The physiological indicators further include: respiratory rate; the mobile terminal maintains a correspondence between at least one brain state, the value range of the blood oxygen saturation, and the value range of the respiratory rate; Determining the user's brain state based on the user's physiological indicators includes: The correspondence between the at least one brain state, the value range of the blood oxygen saturation, and the value range of the respiratory rate is queried, and the brain state of the user is determined based on the value range of the user's blood oxygen saturation and the value range of the respiratory rate.

5. The method according to claim 1, wherein A prediction model for predicting a user's brain state based on the user's physiological indicators is run on the mobile terminal; the prediction model is obtained by supervised training based on a number of physiological indicator samples labeled with brain states; Determining the user's brain state based on the user's physiological indicators includes: Inputting the user's physiological indicators into a pre-trained prediction model and obtaining a prediction result output by the prediction model; Based on the prediction result, the brain state of the user is determined.

6. The method according to claim 1, characterized in that Before generating feedback information according to the user's brain state, the method further includes: Acquiring location information and movement status information of a mobile terminal connected to the headset; Determining a user scenario in which the mobile terminal is located based on the location information and the mobility status information; Generate feedback information based on the user's brain state, including: Feedback information is generated according to the user's brain state and a feedback strategy corresponding to the user scenario.

7. The method according to claim 1, characterized in that The PPG data includes a timestamp corresponding to the acquisition moment; Generate feedback information based on the user's brain state, including: determining, based on the timestamp corresponding to the acquisition moment, the duration of the user's current brain state; If the duration of the user being in the current brain state reaches a duration threshold corresponding to the current brain state, feedback information is generated.

8. The method according to claim 7, characterized in that Different brain states correspond to different duration thresholds.

9. The method according to claim 8, characterized in that The brain states include a focused state, a non-focused state, and a tired state; Generate feedback information based on the user's brain state, including: If the duration of the user being in the focused state reaches a duration threshold corresponding to the focused state, generating feedback information to remind the user to take a break; If the user is in a tired state, generating feedback information to remind the user to take a rest; If the duration of the user being in the non-concentrated state reaches a duration threshold corresponding to the non-concentrated state, feedback information is generated to remind the user to concentrate on learning.

10. The method according to claim 1, characterized in that The PPG data includes a timestamp corresponding to a collection time, and the method further includes: According to the timestamp corresponding to the acquisition moment, the duration of the user being in various brain states is counted, a user feedback report is generated, and the user feedback report is displayed to the user.

11. A user status detection device, characterized in that: The device comprises: an acquisition module, configured to acquire PPG photoplethysmography data of a user, the PPG data being collected by a PPG sensor mounted on headphones worn by the user; An analysis module, configured to analyze the PPG data to obtain physiological indicators of the user; a determination module, configured to determine the user's brain state based on the user's physiological indicators; A generating module is used to generate feedback information according to the brain state of the user, and send the feedback information to the earphone to provide a voice reminder to the user.

12. A headset, characterized in that: The earphones are equipped with a PPG sensor, including: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 10.

13. A wearable device, characterized in that: The wearable device is equipped with a PPG sensor, including: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 10.

14. A mobile terminal, characterized in that: The mobile terminal establishes a connection with the headset equipped with the PPG sensor, including: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 10.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images