User state detection method and system, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of data processing, and relates to a user state detection method which comprises the following steps: acquiring respiration data, electroencephalogram data and basic sensing data of a user; performing feature extraction on the respiration data and the electroencephalogram data to obtain respiration feature data and electroencephalogram feature data; performing data fusion on the respiration feature data and the electroencephalogram feature data to obtain a joint feature vector; determining a target state detection model in each preset state detection model according to the basic sensing data; and performing state detection on the user according to the target state detection model and the joint feature vector, and determining the state of the user. According to the method and the device, the physiological data of the user is acquired from different dimensions, and the acquired data is subjected to multi-modal fusion and conjoint analysis, so that the user state detection in different scenes is realized, diversified scene requirements are met, and the accuracy of the intelligent wearable equipment in detecting the user state is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology and is applied to multiple scenarios requiring physiological monitoring and status detection of users, and in particular to a user status detection method, system, computer device and storage medium. Background Art

[0002] With the development of science and technology, the research and application of smart wearable devices are becoming increasingly important. Smart wearable devices can collect users' physiological data to achieve physiological assessment and status detection, showing great potential in health management.

[0003] However, many smart wearable devices currently on the market still have significant limitations. Most smart wearable devices only use a single sensor to collect user physiological data, which results in the collected data being of a relatively single dimension and unable to fully reflect the user's status. This makes the accuracy of user status detection low, affecting user trust.

[0004] In view of this, this application is filed. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to propose a user status detection method, system, computer device and storage medium to solve the technical problem of low accuracy when smart wearable devices detect user status.

[0006] In order to solve the above technical problems, the present application provides a method for detecting user status, which adopts the following technical solutions: A method for detecting user status includes the following steps: Obtain user's breathing data, EEG data and basic sensor data; Extracting features from the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data; Performing data fusion on the respiratory feature data and the EEG feature data to obtain a joint feature vector; Determining a target state detection model among various preset state detection models based on the basic sensing data; The state of the user is detected according to the target state detection model and the joint feature vector to determine the state of the user.

[0007] Furthermore, the step of performing state detection on the user based on the target state detection model and the joint feature vector to determine the state of the user specifically includes: Extracting state feature key values ​​from the joint feature vector; Inputting the state feature key value into the target state detection model, and determining the state interval corresponding to the state feature key value according to the target state detection model; The user's status is determined according to the status interval.

[0008] Furthermore, the step of determining a target state detection model in each preset state detection model based on the basic sensing data specifically includes: Determining a target scene type corresponding to the user based on the basic sensor data; According to the target scene type, a corresponding state detection model is selected from each of the preset state detection models to serve as the target state detection model.

[0009] Furthermore, the step of fusing the respiratory feature data and the EEG feature data to obtain a joint feature vector specifically includes: Performing time series analysis on the respiratory characteristic data and the EEG characteristic data to determine respiratory characteristic sub-data and EEG characteristic sub-data corresponding to each time stamp; The respiratory feature sub-data and the EEG feature sub-data corresponding to the same timestamp are fused to obtain the joint feature vector.

[0010] Furthermore, after the step of detecting the user's state based on the target state detection model and the joint feature vector to determine the user's state, the method further includes: Determining feedback timing and feedback type based on the user's status; A target feedback strategy corresponding to the user is generated according to the feedback timing and the feedback type.

[0011] Furthermore, after the step of detecting the user's state based on the target state detection model and the joint feature vector to determine the user's state, the method further includes: Obtaining historical status data of the user; The target state detection model is updated according to the user's state and the historical state data.

[0012] Furthermore, the step of obtaining the user's respiratory data, EEG data, and basic sensor data specifically includes: Collecting the user's breathing signal, brain wave signal and basic sensor data; performing denoising processing on the respiratory signal to obtain the respiratory data; Perform frequency domain analysis on the brain wave signal to obtain the brain wave data.

[0013] In order to solve the above technical problems, the embodiment of the present application further provides a user status detection system, which adopts the following technical solution: A user status detection system, comprising: Acquisition module, used to obtain the user's breathing data, EEG data and basic sensor data; A feature extraction module is used to extract features from the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data; A data fusion module, configured to fuse the respiratory characteristic data and the EEG characteristic data to obtain a joint characteristic vector; A first determining module, configured to determine a target state detection model from among various preset state detection models based on the basic sensing data; The second determination module is configured to perform state detection on the user according to the target state detection model and the joint feature vector to determine the state of the user.

[0014] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution: A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the user status detection method described above when executing the computer-readable instructions.

[0015] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the user status detection method described above.

[0016] Compared with the prior art, the embodiments of the present application have the following beneficial effects: The user state detection method disclosed in the present application obtains the user's respiratory data, EEG data and basic sensor data; then performs feature extraction on the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data; and performs data fusion on the respiratory feature data and the EEG feature data to obtain a joint feature vector; then, based on the basic sensor data, determines the target state detection model in each preset state detection model; thereby, based on the target state detection model and the joint feature vector, performs state detection on the user to determine the user's state. The present application collects the user's physiological data from different dimensions and performs multimodal fusion and joint analysis on the collected data, thereby realizing user state detection in different scenarios to adapt to diverse scenario requirements, thereby improving the accuracy of smart wearable devices in detecting user states and enhancing user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 is an exemplary system architecture diagram to which the present application may be applied; Figure 2 is a flow chart of an embodiment of a user status detection method according to the present application; Figure 3 is a structural diagram of an embodiment of a user status detection system according to the present application; Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0020] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0022] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0023] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0024] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and the like.

[0025] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0026] It should be noted that the user status detection method provided in the embodiment of the present application is generally executed by a terminal device, and accordingly, the user status detection system is generally set in the terminal device.

[0027] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0028] Continue to refer Figure 2 , shows a flow chart of an embodiment of a user status detection method according to the present application. The user status detection method comprises the following steps: Step S201: Acquire the user's respiratory data, EEG data, and basic sensor data.

[0029] In this embodiment, the user status detection method is executed on the electronic device (eg Figure 1 The terminal device shown in the figure can send or receive data via a wired connection or a wireless connection. It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other wireless connection methods currently known or to be developed in the future.

[0030] In this embodiment, the terminal device is a smart wearable device that uses built-in sensors to obtain the user's respiratory data, EEG data, and basic sensor data. Specifically, the smart wearable device includes at least a smart nose pad (with an adjustable structure to accommodate different nose shapes) and EEG glasses (with integrated flexible electrodes). The smart nose pad collects the user's respiratory signals via built-in respiratory sensors (pressure / airflow sensors). Kalman filtering is then used to denoise the respiratory signals to obtain respiratory data, including the mean respiratory rate (12-40 breaths / minute), depth fluctuation variance (tidal volume variation), and rhythm stability (inhalation / expiration time ratio deviation). The EEG glasses collect the user's EEG signals via flexible electrodes on the inside of the temples. Fast Fourier transform (FFT) is then used to perform frequency domain analysis on the EEG signals to obtain EEG data. The EEG data includes digital signals corresponding to characteristic frequency bands such as δ, θ, α, β, and γ after frequency domain analysis. In addition, smart nose pads and brainwave glasses also collect various basic sensor data through other built-in sensors. For example, the user's body movement amplitude and posture angle are collected through accelerometers / gyroscopes, and the rate of change of light intensity is collected through ambient light sensors.

[0031] Step S202 , performing feature extraction on the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data.

[0032] In this embodiment, after obtaining the user's respiratory data, EEG data and basic sensor data, feature extraction can be performed on the respiratory data and EEG data to obtain respiratory feature data and EEG feature data. For example, the number of breaths per minute (respiratory frequency) can be calculated as respiratory feature data by real-time detection of the peaks and troughs of the respiratory waveform; the EEG data can be calculated to obtain the energy proportion of each band (such as β wave proportion = β wave energy / total energy) as EEG feature data.

[0033] Step S203: fusing the respiratory feature data and the EEG feature data to obtain a joint feature vector.

[0034] In this embodiment, a time series analysis can be performed on the respiratory feature data and the EEG feature data to determine the respiratory feature sub-data and EEG feature sub-data corresponding to each timestamp. The respiratory feature sub-data and EEG feature sub-data at the same timestamp are fused to form a set of vectors. Ultimately, a set of all vectors is obtained, namely, a "respiratory-EEG" joint feature vector. For example, the feature data extracted from the respiratory data includes at least the user's respiratory rate, and the feature data extracted from the EEG data includes at least the user's beta wave ratio and theta wave ratio. Then, the joint feature vector corresponding to a certain timestamp is: [respiratory frequency, beta wave ratio, theta wave ratio]. Based on the joint feature vector, the user's status can be subsequently detected to determine the user's status.

[0035] Step S204: determining a target state detection model from among various preset state detection models according to the basic sensing data.

[0036] In this embodiment, based on the basic sensor data collected by the smart wearable device, the target scene type corresponding to the user can be determined in each scene type. Then, the target state detection model is determined based on the target scene type, where the scene types include but are not limited to sports scenes, learning scenes, and emotional scenes. Specifically, in addition to breathing signals and brain wave signals, the smart nose pads and brain wave glasses also collect basic sensor data through other sensors such as built-in accelerometers / gyroscopes and ambient light sensors. The basic sensor data includes the user's body movement amplitude, posture angle, and light intensity change rate. Based on the basic sensor data, the scene type in which the user is located can be determined as the target scene type. After determining the target scene type, the corresponding state detection model can be selected from each preset state detection model according to the target scene type as the target state detection model. For example, the sports scene corresponds to the preset sports state detection model, the learning scene corresponds to the preset learning state detection model, and the emotional scene corresponds to the preset emotional state detection model.

[0037] Step S205 : performing state detection on the user according to the target state detection model and the joint feature vector to determine the state of the user.

[0038] In this embodiment, after obtaining the joint feature vector and determining the target state detection model, the state feature key values ​​can be extracted from the joint feature vector and input into the target state detection model. The target state detection model can then be used to analyze the state feature key values ​​and determine the state intervals corresponding to the state feature key values. The user's state can then be determined based on the state intervals. Specifically, each preset state detection model belongs to a rule engine and a lightweight machine learning fusion model. The motion state detection model, the learning state detection model, and the emotional state detection model each have different judgment logics. The state feature key values ​​extracted from the joint feature vector include respiratory rate, beta wave proportion, theta wave proportion, gamma wave proportion, delta wave proportion, etc., and these state feature key values ​​are aligned according to timestamps. Different target state detection models use different scenario-based state rule engines and supervised trained lightweight classifiers for analysis. By comparing the state feature key values ​​with the feature thresholds preset in the model, the state intervals corresponding to the state feature key values ​​can be determined, thereby determining the user's state.

[0039] Optionally, the smart nose pads and brainwave glasses in the embodiments of the present application can also collect other biometric data to detect the user's status. For example, the types of data collected include galvanic skin response, eye tracking, and electromyographic signals. Contact electrodes configured on the temples can be used to measure galvanic skin response to obtain emotional arousal, while a built-in miniature infrared camera in the frame can be used to track eye movements to obtain blink frequency and pupil diameter. Flexible electrodes extended on the temples can be used to collect electromyographic signals to obtain neck / forehead muscle tension.

[0040] This application collects the user's physiological data from different dimensions and performs multimodal fusion and joint analysis on the collected data to achieve user status detection in different scenarios to adapt to diverse scenario requirements, thereby improving the accuracy of smart wearable devices in detecting user status and enhancing user experience.

[0041] In some optional implementations of this embodiment, the step of detecting the user's state based on the target state detection model and the joint feature vector to determine the user's state includes: Extracting state feature key values ​​from the joint feature vector; Inputting the state feature key value into the target state detection model, and determining the state interval corresponding to the state feature key value according to the target state detection model; The user's status is determined according to the status interval.

[0042] In this embodiment, the state feature key values ​​extracted from the joint feature vector include at least respiratory rate, beta wave proportion, theta wave proportion, gamma wave proportion, and delta wave proportion. After the feature state key values ​​are input into the target state detection model, the target state detection model analyzes the feature state key values ​​to obtain the user's state. Specifically, the target state detection model is a fusion model of a rule engine and lightweight machine learning. It uses a scenario-based state rule engine to compare the state feature key values ​​with preset feature thresholds to determine the state interval corresponding to the state feature key values. A lightweight classifier is used for state classification. The lightweight classifier includes an input layer, a hidden layer, and an output layer. The input layer is used to receive the joint feature vector, the hidden layer is used to perform scenario-based feature weighting (e.g., the theta wave weight in the learning scenario is > 0.7), and the output layer is used to generate a state probability distribution. For example, when the user is in a learning scenario, the learning state detection model can judge based on the breathing rate, the proportion of beta waves, and theta waves, and determine the user's state as efficient concentration / cognitive fatigue / distraction. Among them, the state interval corresponding to the efficient concentration state is: breathing rate is greater than or equal to 15 times / minute and less than or equal to 18 times / minute (normal learning breathing), the theta wave proportion is less than or equal to 10% (no fatigue wave), and the beta wave proportion is greater than or equal to 60% (strong concentration wave); the state interval corresponding to the cognitive fatigue state is: breathing rate is greater than 22 times / minute (faster breathing), the theta wave proportion is greater than 20% (significant fatigue wave), and the beta wave proportion is less than 50% (weakened concentration wave); the state interval corresponding to the distracted state is: breathing rate is greater than or equal to 10 times / minute and less than or equal to 25 times / minute (large fluctuation), and the alpha wave proportion is greater than or equal to 50% (relaxation wave dominates).

[0043] This application realizes dynamic analysis of multimodal data through a scenario-based preset state detection model, so that the user state in different scenarios can be detected more accurately, and the accuracy of smart wearable devices in detecting user state is improved.

[0044] In some optional implementations of this embodiment, the step of determining a target state detection model from among various preset state detection models based on the basic sensing data includes: Determining a target scene type corresponding to the user based on the basic sensor data; According to the target scene type, a corresponding state detection model is selected from each of the preset state detection models to serve as the target state detection model.

[0045] In this embodiment, the target scene type corresponding to the user includes at least one of a learning scene, a sports scene, and an emotional scene. Each scene type corresponds to a preset state detection model (e.g., a learning state detection model, a sports state detection model, or an emotional state detection model). Based on the target scene type, the corresponding preset state detection model can be determined as the target state detection model. Specifically, the smart nose pads and EEG glasses can use built-in accelerometers / gyroscopes to collect the user's body movement amplitude and posture angle, and use built-in ambient light sensors to collect the light intensity change rate as the user's basic sensor data. The target scene type corresponding to the user is then determined based on this basic sensor data. For example, based on the user's body movement amplitude and posture angle, if the user is stationary (standard deviation <0.1g), the user is determined to be in a learning scene; if the user is in periodic motion (cadence >1Hz), the user is determined to be in a sports scene. Based on the light intensity change rate, if the user is in stable light (change rate <5% / min), the user is determined to be in a learning scene or an emotional scene; if the user is in strong outdoor light (>1000 lux), the user is determined to be in a sports scene (to assist in determining body movement amplitude and posture angle).

[0046] This application dynamically identifies the user's current scene type through multi-dimensional sensor data, so that the user's state can be detected subsequently through the corresponding state detection model, thereby improving the accuracy of smart wearable devices in detecting user states.

[0047] In some optional implementations of this embodiment, the step of fusing the respiratory feature data and the EEG feature data to obtain a joint feature vector includes: Performing time series analysis on the respiratory characteristic data and the EEG characteristic data to determine respiratory characteristic sub-data and EEG characteristic sub-data corresponding to each time stamp; The respiratory feature sub-data and the EEG feature sub-data corresponding to the same timestamp are fused to obtain the joint feature vector.

[0048] In this embodiment, respiratory data is feature extracted to obtain respiratory feature data, and EEG data is feature extracted to obtain EEG feature data. The respiratory data and EEG data are sorted by time to determine the respiratory feature data corresponding to each timestamp, which is the respiratory feature sub-data, and the EEG feature data corresponding to each timestamp is the EEG feature sub-data. Therefore, data fusion is performed according to the same timestamp to obtain a "respiratory-EEG" joint feature vector. The purpose of data fusion is to resolve the sampling rate difference between the respiratory signal (low frequency) and the EEG signal (high frequency). For example, with each minute as a timestamp, the respiratory feature data and the EEG feature data are time-series aligned, and the respiratory feature sub-data corresponding to the same timestamp are fused with the EEG feature sub-data to form a set of vectors. The set of all vectors is the joint feature vector, including the respiratory frequency (f), beta wave proportion, theta wave proportion, gamma wave proportion, delta wave proportion, and alpha wave proportion corresponding to the same timestamp.

[0049] This application achieves dimensionality reduction and efficiency improvement while retaining core physiological indicators through multimodal data fusion and feature extraction, providing high signal-to-noise ratio input data for subsequent user status detection, and improving the accuracy of smart wearable devices in detecting user status.

[0050] In some optional implementations of this embodiment, after the step of performing state detection on the user based on the target state detection model and the joint feature vector to determine the state of the user, the method further includes: Determining feedback timing and feedback type based on the user's status; A target feedback strategy corresponding to the user is generated according to the feedback timing and the feedback type.

[0051] In this embodiment, the user's status can be used to determine the feedback timing and type, thereby generating a target feedback strategy for the user based on the feedback timing and type. Specifically, the joint feature vector includes the key state feature values ​​corresponding to the user's status. Based on these key state feature values, the duration and fluctuation range of the status can be determined, thereby determining the feedback timing (e.g., "feedback must be triggered within 3 minutes"). Furthermore, based on the user's status, at least one associated feedback type (e.g., voice prompt, vibration reminder) can be selected from a preset feedback type library. These associated feedback types are sorted and combined to determine the user's corresponding feedback type. Through data integration, feedback types are mapped and matched to feedback timings, and a target feedback strategy corresponding to the user is configured, allowing the targeted feedback strategy to be subsequently implemented to provide feedback to the user.

[0052] This application avoids invalid or conflicting feedback by accurately identifying the user needs corresponding to the user's status and generating corresponding feedback strategies, thereby improving the application value of the detection results of the user status of the smart wearable device.

[0053] In some optional implementations of this embodiment, after the step of performing state detection on the user based on the target state detection model and the joint feature vector to determine the state of the user, the method further includes: Obtaining historical status data of the user; The target state detection model is updated according to the user's state and the historical state data.

[0054] In this embodiment, the user's historical status data is read from the database corresponding to the smart wearable device. The historical status data includes the user's status in the past period of time and the corresponding status feature key values. According to the user's status and historical status data, the target state detection model can be personalized analyzed, and the scenario-based state rule engine and lightweight classifier can be adjusted to achieve the update of the target state detection model.

[0055] This application enhances the objectivity and individual differences of status detection through personalized analysis of the user's status and historical status data, improves the reliability of the joint feature vector, and ultimately improves the accuracy of smart wearable devices in detecting user status.

[0056] In some optional implementations of this embodiment, the steps of obtaining the user's respiratory data, EEG data, and basic sensor data include: Collecting the user's breathing signal, brain wave signal and basic sensor data; performing denoising processing on the respiratory signal to obtain the respiratory data; Perform frequency domain analysis on the brain wave signal to obtain the brain wave data.

[0057] In this embodiment, the smart wearable device collects the user's respiratory data, EEG data, and basic sensor data through built-in sensors. Specifically, the user's body movement amplitude and posture angle are collected through the accelerometer / gyroscope, and the light intensity change rate is collected through the ambient light sensor as basic sensor data. In addition, the respiratory signal is denoised using Kalman filtering to obtain respiratory data. The respiratory data includes the mean respiratory rate (12-40 times / minute), depth fluctuation variance (tidal volume change), and rhythm stability (inhalation / expiration time ratio deviation). The EEG data can be obtained by performing frequency domain analysis on the EEG signal through fast Fourier transform. The EEG data includes the energy proportion of each band (such as beta wave proportion = beta wave energy / total energy).

[0058] This application significantly improves signal quality through targeted filtering and feature purification, improves the reliability of subsequent multimodal data fusion, and makes the final detection of user status more accurate.

[0059] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0060] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0061] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware using computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0062] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0063] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a user status detection system. Figure 2 Corresponding to the method embodiment shown, the system can be specifically applied to various electronic devices.

[0064] like Figure 3As shown, the user state detection system 300 of this embodiment includes: an acquisition module 301, a feature extraction module 302, a data fusion module 303, a first determination module 304 and a second determination module 305. Among them: Acquisition module 301, used to acquire the user's respiratory data, EEG data and basic sensor data; A feature extraction module 302 is used to extract features from the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data; A data fusion module 303 is used to fuse the respiratory feature data and the EEG feature data to obtain a joint feature vector; A first determining module 304 is configured to determine a target state detection model from among various preset state detection models based on the basic sensing data; The second determination module 305 is configured to perform state detection on the user according to the target state detection model and the joint feature vector to determine the state of the user.

[0065] The user status detection system provided in this application realizes user status detection in different scenarios by collecting the user's physiological data from different dimensions and performing multimodal fusion and joint analysis on the collected data to adapt to diverse scenario requirements, thereby improving the accuracy of smart wearable devices in detecting user status and enhancing user experience.

[0066] In some optional implementations of this embodiment, the second determining module 305 is further configured to: Extracting state feature key values ​​from the joint feature vector; Inputting the state feature key value into the target state detection model, and determining the state interval corresponding to the state feature key value according to the target state detection model; The user's status is determined according to the status interval.

[0067] The user status detection system provided in this application realizes dynamic analysis of multimodal data through a scenario-based preset status detection model, so that the user status in different scenarios can be detected more accurately, thereby improving the accuracy of smart wearable devices in detecting user status.

[0068] In some optional implementations of this embodiment, the first determining module 304 is further configured to: Determining a target scene type corresponding to the user based on the basic sensor data; According to the target scene type, a corresponding state detection model is selected from each of the preset state detection models to serve as the target state detection model.

[0069] The user status detection system provided in this application dynamically identifies the user's current scene type through multi-dimensional sensor data, so that the user's status can be subsequently detected through the corresponding status detection model, thereby improving the accuracy of smart wearable devices in detecting user status.

[0070] In some optional implementations of this embodiment, the data fusion module 303 is further configured to: Performing time series analysis on the respiratory characteristic data and the EEG characteristic data to determine respiratory characteristic sub-data and EEG characteristic sub-data corresponding to each time stamp; The respiratory feature sub-data and the EEG feature sub-data corresponding to the same timestamp are fused to obtain the joint feature vector.

[0071] The user status detection system provided in this application achieves dimensionality reduction and efficiency improvement while retaining core physiological indicators through multimodal data fusion and feature extraction, providing high signal-to-noise ratio input data for subsequent user status detection, and improving the accuracy of smart wearable devices in detecting user status.

[0072] In some optional implementations of this embodiment, the user state detection system 300 is further configured to: Determining feedback timing and feedback type based on the user's status; A target feedback strategy corresponding to the user is generated according to the feedback timing and the feedback type.

[0073] The user status detection system provided in this application avoids invalid or conflicting feedback by accurately identifying the user needs corresponding to the user status and generating corresponding feedback strategies, thereby improving the application value of the detection results of the user status of the smart wearable device.

[0074] In some optional implementations of this embodiment, the user state detection system 300 is further configured to: Obtaining historical status data of the user; The target state detection model is updated according to the user's state and the historical state data.

[0075] The user status detection system provided in this application enhances the objectivity and individual differences of status detection through personalized analysis of the user's status and historical status data, improves the reliability of the joint feature vector, and ultimately improves the accuracy of smart wearable devices in detecting user status.

[0076] In some optional implementations of this embodiment, the acquisition module 301 is further configured to: Collecting the user's breathing signal, brain wave signal and basic sensor data; performing denoising processing on the respiratory signal to obtain the respiratory data; Perform frequency domain analysis on the brain wave signal to obtain the brain wave data.

[0077] The user status detection system provided in this application significantly improves signal quality through targeted filtering and feature purification, improves the reliability of subsequent multimodal data fusion, and makes the final detection of user status more accurate.

[0078] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0079] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0080] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0081] The memory 41 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or internal memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. Of course, the memory 41 may also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for the user status detection method. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0082] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for executing the user status detection method.

[0083] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0084] The computer device provided in this application collects the user's physiological data from different dimensions and performs multimodal fusion and joint analysis on the collected data, thereby realizing user status detection in different scenarios to adapt to diverse scenario requirements, thereby improving the accuracy of smart wearable devices in detecting user status and enhancing user experience.

[0085] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the user status detection method as described above.

[0086] The computer-readable storage medium provided in this application realizes user status detection in different scenarios by collecting the user's physiological data from different dimensions and performing multimodal fusion and joint analysis on the collected data to adapt to diverse scenario requirements, thereby improving the accuracy of smart wearable devices in detecting user status and enhancing user experience.

[0087] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0088] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A user status detection method, characterized in that: The steps include: Obtain user's breathing data, EEG data and basic sensor data; Extracting features from the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data; Performing data fusion on the respiratory feature data and the EEG feature data to obtain a joint feature vector; Determining a target state detection model among various preset state detection models based on the basic sensing data; The state of the user is detected according to the target state detection model and the joint feature vector to determine the state of the user.

2. The user status detection method according to claim 1, characterized in that: The step of performing state detection on the user according to the target state detection model and the joint feature vector to determine the state of the user specifically includes: Extracting state feature key values ​​from the joint feature vector; Inputting the state feature key value into the target state detection model, and determining the state interval corresponding to the state feature key value according to the target state detection model; The user's status is determined according to the status interval.

3. The user status detection method according to claim 1, wherein: The step of determining a target state detection model in each preset state detection model based on the basic sensing data specifically includes: Determining a target scene type corresponding to the user based on the basic sensor data; According to the target scene type, a corresponding state detection model is selected from each of the preset state detection models to serve as the target state detection model.

4. The user status detection method according to claim 1, characterized in that: The step of fusing the respiratory feature data and the EEG feature data to obtain a joint feature vector specifically includes: Performing time series analysis on the respiratory characteristic data and the EEG characteristic data to determine respiratory characteristic sub-data and EEG characteristic sub-data corresponding to each time stamp; The respiratory feature sub-data and the EEG feature sub-data corresponding to the same timestamp are fused to obtain the joint feature vector.

5. The user status detection method according to claim 1, characterized in that: After the step of detecting the state of the user according to the target state detection model and the joint feature vector to determine the state of the user, the method further includes: Determining feedback timing and feedback type based on the user's status; A target feedback strategy corresponding to the user is generated according to the feedback timing and the feedback type.

6. The user status detection method according to claim 1, characterized in that: After the step of detecting the state of the user according to the target state detection model and the joint feature vector to determine the state of the user, the method further includes: Obtaining historical status data of the user; The target state detection model is updated according to the user's state and the historical state data.

7. The user status detection method according to any one of claims 1 to 6, characterized in that: The step of obtaining the user's respiratory data, EEG data, and basic sensor data specifically includes: Collecting the user's breathing signal, brain wave signal and basic sensor data; performing denoising processing on the respiratory signal to obtain the respiratory data; Perform frequency domain analysis on the brain wave signal to obtain the brain wave data.

8. A user status detection system, characterized in that: include: Acquisition module, used to obtain the user's breathing data, EEG data and basic sensor data; A feature extraction module is used to extract features from the respiratory data and the EEG data to obtain respiratory feature data and EEG feature data; A data fusion module, configured to fuse the respiratory characteristic data and the EEG characteristic data to obtain a joint characteristic vector; A first determining module, configured to determine a target state detection model from among various preset state detection models based on the basic sensing data; The second determination module is configured to perform state detection on the user according to the target state detection model and the joint feature vector to determine the state of the user.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the user status detection method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the user status detection method according to any one of claims 1 to 7.

Images