A method and apparatus for obtaining a degree of alertness

Abstract

This application relates to a method and apparatus for acquiring alertness levels in the field of artificial intelligence, used to predict a user's alertness level by combining user behavior information collected by the device. The method includes: acquiring first information, the first information including information about user behavior, the first information including information collected by a first device, the first device including a wearable device of the user, and the first information including information about the user's ingested substances; and predicting changes in the user's alertness based on the first information, the changes in alertness including the user's alertness value within a first preset time period.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method and device for obtaining alertness levels. Background Technology

[0002] In aviation, railway, and highway sectors, fatigue detection for users has a wide range of applications. For example, driver fatigue is prevalent and threatens driving safety. To reduce the phenomenon of fatigued driving and its consequences, driver fatigue monitoring systems are typically installed in motor vehicles to detect the driver's fatigue status in a timely manner.

[0003] Common fatigue detection methods include using computer vision technology (such as analyzing eye and head movements), based on features such as steering wheel movement and driving trajectory, and based on the detection of driver physiological signals (such as EEG and ECG). However, these methods often fail to detect driver fatigue in a timely manner. Summary of the Invention

[0004] This application provides a method and apparatus for obtaining alertness level, which is used to predict the alertness level of a user by combining user behavior information collected by the device.

[0005] In view of the above, in a first aspect, this application provides a method for obtaining alertness level, comprising: obtaining first information, the first information including information on user behavior, the first information including information collected by a first device, the first device including a user's wearable device, and the first information including information on the user's ingested substances; and predicting changes in user alertness based on the first information, the changes in alertness including the user's alertness value within a first preset time period.

[0006] Therefore, in this application's embodiments, the user's alertness level can be predicted based on information collected by the wearable device from the ingested substance. Compared to manually inputting information, the method provided in this application can obtain user information more efficiently and accurately, thereby improving the accuracy of the prediction. This allows for proactive handling of the user's fatigue state, improving the user experience. Furthermore, the user's alertness changes can be analyzed from the perspective of the ingested substance, enabling a more comprehensive prediction of the user's alertness level over a future period. This allows for proactive responses based on the predicted alertness level, further enhancing the user experience.

[0007] In one possible implementation, the first information may also include the user's sleep information or exercise information.

[0008] In one possible implementation, sleep information includes at least one of sleep quality score and nap information; exercise information includes at least one of exercise start time, exercise end time, exercise type or exercise amount; and intake information includes at least one of intake time, intake amount or intake type.

[0009] Therefore, in this application embodiment, in addition to information about the ingested substance, the user's movement information and sleep information can also be combined to predict changes in the user's alertness level, thereby using more comprehensive information to predict changes in the user's alertness level and improving prediction accuracy.

[0010] In one possible implementation, predicting changes in a user's alertness using first information includes: using the first information as input to a preset alertness model and outputting changes in the user's alertness. The output of the alertness model includes a fusion result of circadian rhythm and arousal rhythm. The circadian rhythm is obtained from sleep-related information in the first information, and the arousal rhythm includes changes in the user's arousal level after falling asleep.

[0011] In this application, the user's level of alertness can be predicted by a pre-set alertness model, and the user's level of alertness can be predicted more accurately by using circadian rhythms and arousal rhythms.

[0012] In one possible implementation, the alertness value is used to represent a change in the fatigue level of at least one user driving the vehicle, and the method further includes adjusting the vehicle's driving information based on the alertness value.

[0013] Therefore, in the embodiments of this application, in a driving scenario, the vehicle's driving information can be adjusted based on the predicted alert value, thereby matching the vehicle's driving state with the user's alert state and improving driving safety.

[0014] In one possible implementation, adjusting vehicle driving information based on an alert value includes sending an adjustment instruction to the vehicle, the adjustment instruction being used to instruct the vehicle driving information to be adjusted based on the alert value.

[0015] Therefore, in this embodiment of the application, if the alertness level is predicted by other devices, the driving parameters inside the vehicle can be adjusted by sending instructions to the vehicle, so that the driving state of the vehicle matches the user's alertness state, thereby improving driving safety.

[0016] In one possible implementation, the method further includes: adjusting the itinerary plan according to the type of intake of the second user to obtain an adjusted itinerary plan.

[0017] Therefore, in this embodiment, the itinerary plan can be adjusted in a timely manner based on the type of the user's intake, so that the itinerary plan matches the user's alertness state and improves the user experience.

[0018] In one possible implementation, generating a second prompt message for the second user based on the trip plan includes: generating a second prompt message for the second user based on the trip plan and information about the movement generated by the second user within a fourth preset time period, wherein the movement information includes the amount of movement or the duration of movement.

[0019] Therefore, in this embodiment, prompts can be generated based on the user's movement status, thereby prompting the user to adjust their movement status or travel plans, which can match the user's alertness with the travel plans and improve the user experience.

[0020] In one possible implementation, the method may further include: adjusting at least one of the following parameter values ​​of the vehicle based on an alert value: adjusting the alarm range of an in-vehicle eye-closing detection system, which detects whether a user closes their eyes while driving and issues an alarm when the detection result is within the alarm range; or adjusting the alarm information of an in-vehicle visual driver monitoring system (DMS), which includes a fatigue alarm range, an alert period, or an alert intensity; or adjusting the vehicle's navigation information, which includes at least one of a navigation route, a driving time along the navigation route, or a road type along the navigation route; or acquiring alert information of at least one traffic participant adjacent to the vehicle and acquiring the driving dynamics of at least one traffic participant based on the alert information of at least one traffic participant; or adjusting the detection threshold of an in-vehicle alarm system, which is related to the sensitivity of the alarm system to issue an alarm.

[0021] Therefore, in this embodiment, the parameters of various hardware or software systems in the vehicle can be adjusted based on the predicted level of user alertness, so that the various parameters in the vehicle match the level of user alertness, thereby improving the user's driving safety.

[0022] In one possible implementation, the method further includes: acquiring driving information of a user collected in the vehicle, the driving information including at least one of driving duration, road type, or traffic information; updating alertness changes based on the driving information to obtain updated alertness changes.

[0023] This can be understood as the user expending physical energy while driving the vehicle. This allows the system to update the prediction of changes in alertness in real time based on the user's actual driving conditions. This makes the changes in alertness more closely match the user's actual state, enabling the system to adjust the driving status based on the real-time updates of alertness and improve the user's driving safety.

[0024] In one possible implementation, the number of users is at least two, and the method further includes: selecting a first user from the at least two users based on the driving suitability index of the at least two users; generating a first prompt message for the first user, the first prompt message being used to prompt the first user to drive the vehicle.

[0025] Therefore, in this embodiment of the application, when there are multiple users in the vehicle, a more suitable driver can be selected from among the multiple users, thereby improving the driving safety of the vehicle.

[0026] In one possible implementation, the method further includes: acquiring input information from a second user; acquiring the second user's travel plan for a second preset time period based on the input information; and generating a second prompt message for the second user based on the travel plan and preset conditions. The second prompt message is used to remind the second user of their current movement status. This can be understood as follows: when a user has a travel plan for a future period, the method can remind the user to exercise appropriately based on their current movement status, thereby improving the user's alertness and driving safety in the future.

[0027] In one possible implementation, the method further includes: obtaining a driving suitability index based on an alertness value, wherein the driving suitability index is positively correlated with the level of alertness, and the driving suitability index is used to remind the user of the safe driving level; and playing at least one of the alertness value or the driving suitability index through a second device. Therefore, the user's current state can be represented by the driving suitability index or the alertness value, thereby reminding the user to drive cautiously and improving the user's driving safety.

[0028] In one possible implementation, the method further includes: if the first device detects that the user's behavior includes a preset behavior within a third preset time period, then the alertness change is updated based on the information of the user's preset behavior to obtain the updated alertness change, thereby updating the user's alertness change based on the user's real-time behavior, making the final predicted alertness change more accurate.

[0029] Secondly, this application provides a wearable device, comprising:

[0030] The acquisition module is used to acquire first information, which includes information about user behavior, information collected by a first device, a wearable device of the user, and information about the user's ingested substances.

[0031] The processing module is used to predict changes in the user's alertness based on the first information, which includes the user's alertness value within a first preset time period.

[0032] In one possible implementation, the first information may also include the user's sleep information or exercise information.

[0033] In one possible implementation, sleep information includes at least one of sleep quality score and nap information; exercise information includes at least one of exercise start time, exercise end time, exercise type or exercise amount; and intake information includes at least one of intake time, intake amount or intake type.

[0034] In one possible implementation, the processing module is specifically used to take the first information as input to a preset alertness model and output the user's alertness changes. The output of the alertness model includes the fusion result of the circadian rhythm and the arousal rhythm. The circadian rhythm is obtained from sleep-related information in the first information, and the arousal rhythm includes the changes in the user's arousal level after sleep.

[0035] In one possible implementation, the processing module is further configured to: adjust the alarm range of an in-vehicle eye-closing detection system, which detects whether a user closes their eyes while driving and issues an alarm when the detection result falls within the alarm range; or adjust the alarm information of an in-vehicle visual driver monitoring system (DMS), including a fatigue alarm range, a reminder period, or a reminder intensity; or adjust the vehicle's navigation information, including at least one of a navigation route, a driving time along the navigation route, or a road type along the navigation route; or acquire alertness information from at least one traffic participant adjacent to the vehicle and acquire the driving dynamics of at least one traffic participant based on the alertness information; or adjust the detection threshold of the in-vehicle alarm system, the detection threshold being related to the sensitivity of the alarm system to issue an alarm.

[0036] In one possible implementation, the acquisition module is further configured to acquire the user's driving information collected inside the vehicle, the driving information including at least one of driving duration, road type, or traffic information; the processing module is further configured to update the alertness change based on the driving information to obtain the updated alertness change.

[0037] In one possible implementation, the number of users is at least two. The processing module is further configured to select a first user from the at least two users based on the driving suitability index of the at least two users; and generate a first prompt message for the first user, which is used to prompt the first user to drive the vehicle.

[0038] In one possible implementation, the acquisition module is further configured to acquire input information from a second user;

[0039] The processing module is also used to obtain the second user's travel plan for a second preset time period in the future based on the input information; and to generate a second prompt message for the second user based on the travel plan and preset conditions. The second prompt message is used to remind the second user of their current movement status.

[0040] In one possible implementation, a playback module;

[0041] The processing module is also used to obtain a driving suitability index based on the alertness value. The driving suitability index is positively correlated with the alertness level and is used to remind the user of the safety level of driving the vehicle.

[0042] A playback module for playing at least one of alertness value or driving suitability index via a second device.

[0043] In one possible implementation, the processing module is further configured to update the alertness change status based on the information of the user-generated preset behavior if the first device detects that the user-generated behavior includes preset behavior within a third preset time period, thereby obtaining the updated alertness change status.

[0044] Thirdly, this application provides a vehicle, comprising:

[0045] The acquisition module is used to acquire first information, which includes information about user behavior, information collected by a first device, a wearable device of the user, and information about the user's ingested substances.

[0046] The processing module is used to predict changes in the user's alertness based on the first information, which includes the user's alertness value within a first preset time period.

[0047] In one possible implementation, the first information may also include the user's sleep information or exercise information.

[0048] In one possible implementation, sleep information includes at least one of sleep quality score and nap information; exercise information includes at least one of exercise start time, exercise end time, exercise type or exercise amount; and intake information includes at least one of intake time, intake amount or intake type.

[0049] In one possible implementation, the processing module is specifically used to take the first information as input to a preset alertness model and output the user's alertness changes. The output of the alertness model includes the fusion result of the circadian rhythm and the arousal rhythm. The circadian rhythm is obtained from sleep-related information in the first information, and the arousal rhythm includes the changes in the user's arousal level after sleep.

[0050] In one possible implementation, the processing module is further configured to: adjust the alarm range of an in-vehicle eye-closing detection system, which detects whether a user closes their eyes while driving and issues an alarm when the detection result falls within the alarm range; or adjust the alarm information of an in-vehicle visual driver monitoring system (DMS), including a fatigue alarm range, a reminder period, or a reminder intensity; or adjust the vehicle's navigation information, including at least one of a navigation route, a driving time along the navigation route, or a road type along the navigation route; or acquire alertness information from at least one traffic participant adjacent to the vehicle and acquire the driving dynamics of at least one traffic participant based on the alertness information; or adjust the detection threshold of the in-vehicle alarm system, the detection threshold being related to the sensitivity of the alarm system to issue an alarm.

[0051] In one possible implementation, the acquisition module is further configured to acquire the user's driving information collected inside the vehicle, the driving information including at least one of driving duration, road type, or traffic information; the processing module is further configured to update the alertness change based on the driving information to obtain the updated alertness change.

[0052] In one possible implementation, the number of users is at least two. The processing module is further configured to select a first user from the at least two users based on the driving suitability index of the at least two users; and generate a first prompt message for the first user, which is used to prompt the first user to drive the vehicle.

[0053] In one possible implementation, the acquisition module is further configured to acquire input information from a second user;

[0054] The processing module is also used to obtain the second user's itinerary plan for a second preset time period in the future based on the input information;

[0055] The processing module is also used to generate a second prompt message for the second user based on the trip plan and preset conditions. The second prompt message is used to remind the second user of the current movement status.

[0056] In one possible implementation, the vehicle further includes: a playback module;

[0057] The processing module is also used to obtain a driving suitability index based on the alertness value. The driving suitability index is positively correlated with the alertness level and is used to remind the user of the safety level of driving the vehicle.

[0058] A playback module for playing at least one of alertness value or driving suitability index via a second device.

[0059] In one possible implementation, the processing module is further configured to update the alertness change status based on the information of the user-generated preset behavior if the first device detects that the user-generated behavior includes preset behavior within a third preset time period, thereby obtaining the updated alertness change status.

[0060] Fifthly, embodiments of this application provide a wearable device, including a processor and a memory, wherein the processor and the memory are interconnected via a circuit, and the processor calls program code in the memory to execute processing-related functions in the alertness acquisition method described in any of the first aspects. Optionally, the electronic device may be a chip.

[0061] Sixthly, embodiments of this application provide a vehicle, including a processor and a memory, wherein the processor and the memory are interconnected via a circuit, and the processor calls program code in the memory to perform processing-related functions in the alertness level acquisition method described in any of the first aspects above. Optionally, the electronic device may be a chip.

[0062] In a seventh aspect, embodiments of this application provide an electronic device, which may also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform processing-related functions as described in the first aspect or any optional embodiment of the first aspect.

[0063] Eighthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any optional embodiment of the first aspect.

[0064] Ninthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect or any optional implementation of the first aspect. Attached Figure Description

[0065] Figure 1 A schematic diagram of a network architecture provided for this application;

[0066] Figure 2 A schematic diagram of the structure of a terminal provided by BenSe;

[0067] Figure 3 A schematic diagram of the structure of another terminal provided in this application;

[0068] Figure 4 A structural schematic diagram of a vehicle provided by BenSe;

[0069] Figure 5 A flowchart illustrating a method for obtaining alertness levels provided in this application;

[0070] Figure 6 A flowchart illustrating another method for obtaining alertness provided in this application;

[0071] Figure 7 This application provides a diurnal and wakefulness rhythm curve;

[0072] Figure 8 An alertness prediction curve provided for this application;

[0073] Figure 9 A flowchart illustrating another method for obtaining alertness provided in this application;

[0074] Figure 10 Another alertness prediction curve provided for this application;

[0075] Figure 11 Another alertness prediction curve provided for this application;

[0076] Figure 12 Another alertness prediction curve provided for this application;

[0077] Figure 13 Another alertness prediction curve provided for this application;

[0078] Figure 14 Another alertness prediction curve provided for this application;

[0079] Figure 15 Another alertness prediction curve provided for this application;

[0080] Figure 16 Another alertness prediction curve provided for this application;

[0081] Figure 17 This application provides an illustration of an application scenario.

[0082] Figure 18 This application provides another application scenario illustration;

[0083] Figure 19 A flowchart illustrating another method for obtaining alertness provided in this application;

[0084] Figure 20 A flowchart illustrating another method for obtaining alertness provided in this application;

[0085] Figure 21 A schematic diagram of the structure of a wearable device provided in this application;

[0086] Figure 22 A structural schematic diagram of a vehicle provided in this application;

[0087] Figure 23 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0088] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] First, the network architecture used in the alertness acquisition method provided in this application can be found in [reference needed]. Figure 1 As shown. This network architecture includes multiple devices (such as...) Figure 1 The devices described herein (1-N) can be interconnected. These devices may include various movable or non-movable electronic devices, in-vehicle equipment, or vehicles, etc.

[0090] These multiple devices can establish connections via a wireless network or a wired network. The wireless network includes, but is not limited to, any one or more combinations of: Ultra Wide Bandwidth (UWB), 5th Generation (5G) systems, Long Term Evolution (LTE) systems, Global System for Mobile Communication (GSM) or Code Division Multiple Access (CDMA) networks, Wideband Code Division Multiple Access (WCDMA) networks, Wireless Fidelity (WiFi), Bluetooth, Zigbee, Radio Frequency Identification (RFID), Long Range (Lora) wireless communication, and Near Field Communication (NFC).

[0091] The devices provided in this application may include, but are not limited to: smartphones, televisions, tablets, wristbands, head-mounted displays (HMDs), augmented reality (AR) devices, mixed reality (MR) devices, cellular phones, smartphones, personal digital assistants (PDAs), tablet computers, vehicles, in-vehicle terminals, laptop computers, and personal computers (PCs). Of course, in the following embodiments, no specific form of the device is limited. It can be understood that the first device, second device, or third device mentioned in the following embodiments of this application can be any of the aforementioned devices.

[0092] exist Figure 1 In the network architecture shown, devices can transmit data to each other based on direct or established communication connections, such as transmitting their location information to all devices or designated electronic devices in the network architecture through direct or established communication connections.

[0093] It should be understood that, Figure 1In the network architecture shown, one device can be selected as the management device to manage the devices within the system architecture, such as adding authentication, online status statistics, location statistics, bandwidth allocation, or traffic statistics. Of course, a management device can also be not selected, and each electronic device can establish a connection directly or indirectly to obtain information about other devices in the system architecture, such as location information and device attributes (such as device type or device appearance).

[0094] For ease of understanding, the devices provided in this application are categorized into various types, such as terminals, vehicles, or other devices. Different devices may have different structures. The following example uses the structures of terminals and vehicles for illustrative purposes.

[0095] The terminal can collect users' physiological and behavioral information. Taking smartwatches as an example, it can obtain information on users' sleep, activity levels, and ingested substances. Smartwatches should have multiple sensors, including but not limited to accelerometers, gyroscopes, optical heart rate sensors, body temperature sensors, and light sensors. When detecting ingested substances, in addition to smartwatches, other smart wearable devices (such as glasses and headphones) can be used to detect behaviors such as eating and drinking. Smartwatches have a feedback system that can provide feedback to the user through screen display, sound, vibration, etc. Smartwatches also have a communication system that can connect to other devices wirelessly or wiredly, including but not limited to smartphones, car intelligent main systems, independent vehicles, independent or vehicle-integrated driver monitoring systems (DMS systems), and cloud systems.

[0096] Smartphones have input systems, including cameras, touchscreens, and microphones. Smartwatches have feedback systems that can provide feedback to users through screen display, sound, vibration, etc. Smartwatches also have communication systems that can connect to other devices wirelessly or via wires, including but not limited to smartwatches, car infotainment systems, stand-alone vehicles, driver monitoring systems (DMS) integrated into stand-alone or car owner systems, and cloud systems.

[0097] Intelligent vehicles have input systems, including cameras, touchscreens, microphones, and control buttons. The cameras can be used to identify the driver. Users can control some in-vehicle devices through these input devices. Intelligent vehicles have feedback systems that provide feedback to users through screen displays, sound, vibration, etc. Intelligent vehicles have communication systems that can connect to other devices wirelessly or via wired connections, including but not limited to smartwatches, smartphones, other independent in-vehicle device systems, independent driver monitoring systems (DMS systems), and cloud systems. Intelligent vehicles have device control systems that can acquire and control the settings information of other in-vehicle devices.

[0098] The structure of the terminal and vehicle provided in this application is described below as an example.

[0099] For example, see Figure 2 The following uses a specific structure as an example to illustrate the structure of the terminal provided in this application.

[0100] Terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, a motion sensor 180N, etc.

[0101] It is understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the terminal 100. In other embodiments of this application, the terminal 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0102] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0103] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0104] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0105] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0106] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal 100.

[0107] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0108] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0109] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0110] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the shooting function of the terminal 100. The processor 110 and the display screen 194 communicate via the DSI interface to enable the display function of the terminal 100.

[0111] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0112] USB interface 130 is a USB standard compliant interface, specifically a Mini USB interface, Micro USB interface, USB Type-C interface, etc. USB interface 130 can be used to connect a charger to charge terminal 100, and can also be used for data transfer between terminal 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices. It should be understood that USB interface 130 can be replaced with other interfaces, such as Type-C or Lightning interfaces that enable charging or data transfer; USB interface 130 is used here as an example only.

[0113] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the terminal 100. In other embodiments of this application, the terminal 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0114] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the terminal 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0115] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0116] The wireless communication function of terminal 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0117] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0118] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0119] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0120] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), ultra-wideband (UWB), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0121] In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technologies mentioned may include, but are not limited to: 5th-Generation (5G) systems, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Bluetooth, the Global Navigation Satellite System (GNSS), Wireless Fidelity (WiFi), Near Field Communication (NFC), FM (Frequency Modulation Broadcasting), Zigbee, Radio Frequency Identification (RFID), and / or Infrared (IR) technologies. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS), etc.

[0122] In some implementations, terminal 100 may also include a wired communication module. Figure 1 (not shown in the image), or, the mobile communication module 150 or wireless communication module 160 here can be replaced with a wired communication module (…). Figure 1(Not shown in the image), this wired communication module enables electronic devices to communicate with other devices via a wired network. This wired network may include, but is not limited to, one or more of the following: optical transport network (OTN), synchronous digital hierarchy (SDH), passive optical network (PON), Ethernet, or flex Ethernet (FlexE), etc.

[0123] Terminal 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0124] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0125] Terminal 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0126] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0127] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, terminal 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0128] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal 100 selects a frequency point, the DSP can perform Fourier transforms on the frequency energy.

[0129] Video codecs are used to compress or decompress digital video. Terminal 100 may support one or more video codecs. Thus, terminal 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0130] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in terminals, such as image recognition, facial recognition, speech recognition, and text understanding.

[0131] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0132] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of terminal 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0133] Terminal 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0134] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0135] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal 100 can listen to music or make hands-free calls through the speaker 170A.

[0136] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal 100 receives a phone call or voice message, the receiver 170B can be brought close to the listener's ear to hear the voice.

[0137] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal 100 may have at least one microphone 170C. In some embodiments, terminal 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal 100 may have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0138] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0139] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Terminal 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, terminal 100 detects the intensity of the touch operation based on pressure sensor 180A. Terminal 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0140] The gyroscope sensor 180B can be used to determine the motion attitude of the terminal 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the terminal 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal 100's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the terminal 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0141] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0142] The magnetic sensor 180D includes a Hall sensor. The terminal 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the terminal 100 is a flip phone, the terminal 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0143] The 180E accelerometer can detect the magnitude of acceleration of terminal 100 in various directions (typically three axes). When terminal 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices, and is applied to applications such as screen orientation switching and pedometers.

[0144] A distance sensor 180F is used to measure distance. The terminal 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, the terminal 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0145] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The terminal 100 emits infrared light outward through the LED. The terminal 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal 100. When insufficient reflected light is detected, the terminal 100 can determine that there is no object near the terminal 100. The terminal 100 may use the proximity sensor 180G to detect when a user holds the terminal 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and screen locking.

[0146] The ambient light sensor 180L is used to sense the ambient light intensity. The terminal 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the terminal 100 is in a pocket to prevent accidental touches.

[0147] The fingerprint sensor 180H is used to collect fingerprints. The terminal 100 can use the characteristics of the collected fingerprints to unlock the device, access application locks, take photos with fingerprints, and answer calls with fingerprints.

[0148] Temperature sensor 180J is used to detect temperature. In some embodiments, terminal 100 uses the temperature detected by temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, terminal 100 reduces the performance of the processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, terminal 100 heats battery 142 to prevent abnormal shutdown of terminal 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, terminal 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0149] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of terminal 100, in a different position than display screen 194.

[0150] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0151] The 180N motion sensor can be used to detect moving objects within the range captured by a camera, acquiring their motion contours or trajectories. For example, the 180N motion sensor can be an infrared sensor, a laser sensor, or a dynamic vision sensor (DVS). Specifically, the DVS can include sensors such as DAVIS (Dynamic and Active-pixel Vision Sensor), ATIS (Asynchronous Time-based Image Sensor), or CeleX sensors. The DVS borrows characteristics from biological vision, with each pixel simulating a neuron, independently responding to relative changes in light intensity. When the relative change in light intensity exceeds a threshold, the pixel outputs an event signal, including the pixel's position, timestamp, and characteristic information about the light intensity.

[0152] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Terminal 100 can receive button input and generate key signal inputs related to user settings and function control of terminal 100.

[0153] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0154] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0155] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal 100. The terminal 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the terminal 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal 100 and cannot be separated from the terminal 100.

[0156] It should be noted that in some practical application scenarios, electronic devices may include more than those described above. Figure 2 The number of components may be more or less, depending on the actual application scenario, and this application does not impose any limitations on this.

[0157] The foregoing has provided an exemplary description of the hardware structure of the electronic device provided in this application. The system that can be mounted on this electronic device may include... This application does not impose any restrictions on HarmonyOS or other operating systems.

[0158] With Taking Terminal 200 of the operating system as an example, Figure 3 As shown, terminal 200 can be logically divided into a hardware layer 21, an operating system 261, and an application layer 31. Hardware layer 21 includes hardware resources such as an application processor 201, a microcontroller unit 203, a modem 207, a Wi-Fi module 211, a sensor 214, and a positioning module 150. Application layer 31 includes one or more applications, such as application 263. Application 263 can be any type of application, such as a social application, e-commerce application, browser, or a lost-and-found app, executing the alertness acquisition method provided in this application to search for the user's lost device. Operating system 261, as software middleware between hardware layer 21 and application layer 31, is a computer program that manages and controls hardware and software resources.

[0159] In one embodiment, the operating system 261 includes a kernel 23, a hardware abstraction layer (HAL) 25, libraries and runtime 27, and a framework 29. The kernel 23 provides low-level system components and services, such as power management, memory management, thread management, and hardware drivers. Hardware drivers include Wi-Fi drivers, sensor drivers, and positioning module drivers. The hardware abstraction layer 25 encapsulates the kernel drivers, providing interfaces to the framework 29 and shielding them from low-level implementation details. The hardware abstraction layer 25 runs in user space, while the kernel drivers run in kernel space.

[0160] Libraries and runtimes, also known as runtime libraries, provide the necessary library files and execution environment for executable programs at runtime. Libraries and runtimes include the Android Runtime (ART) 271 and libraries 273, among others. ART 271 is a virtual machine or virtual machine instance that translates application bytecode into machine code. Libraries 273 are libraries that provide support for executable programs at runtime, including browser engines (such as WebKit), script execution engines (such as JavaScript engines), and graphics processing engines.

[0161] Framework 27 provides various basic public components and services for applications in application layer 31, such as window management, location management, etc. Framework 27 may include a phone manager 291, a resource manager 293, a location manager 295, etc.

[0162] The functions of each component of the operating system 261 described above can be implemented by the application processor 201 executing the program stored in the memory 205.

[0163] Those skilled in the art will understand that terminal 200 may include more than Figure 3 The number of fewer or more components shown Figure 3 The electronic device shown includes only components more relevant to the various implementations disclosed in the embodiments of this application.

[0164] Furthermore, the structure of the vehicle provided in this application is as follows: Figure 4 As shown, Figure 1This is a schematic diagram of a vehicle structure provided in an embodiment of this application. The vehicle 400 can be configured in an autonomous driving mode. For example, the vehicle 400 can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine whether there are obstacles in the surrounding environment, and control the vehicle 400 based on the obstacle information. When the vehicle 400 is in autonomous driving mode, the vehicle 400 can also be set to operate without human interaction.

[0165] Vehicle 400 may include various subsystems, such as a mobility system 402, a sensor system 404, a control system 406, one or more peripheral devices 408, a power supply 410, a computer system 442, and a user interface 446. Optionally, vehicle 400 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 400 may be interconnected via wired or wireless means.

[0166] The mobility system 402 may include components that provide powered motion to the vehicle 400. In one embodiment, the mobility system 402 may include an engine 418, an energy source 419, a transmission 420, and wheels / tires 421.

[0167] Engine 418 can be an internal combustion engine, an electric motor, an air-compressed engine, or other combinations of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. Engine 418 converts energy source 419 into mechanical energy. Examples of energy source 419 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 419 can also provide energy to other systems of vehicle 400. Transmission 420 transmits mechanical power from engine 418 to wheels 421. Transmission 420 may include a gearbox, a differential, and a drive shaft. In one embodiment, transmission 420 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 421.

[0168] Sensor system 404 may include several sensors for sensing information about the environment surrounding vehicle 400. For example, sensor system 404 may include positioning system 422 (which may be a GPS system, a BeiDou system, or another positioning system), inertial measurement unit (IMU) 424, radar 426, laser rangefinder 428, and camera 430. Sensor system 404 may also include sensors for the internal systems of the monitored vehicle 400 (e.g., in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensing data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of autonomous vehicle 400. The sensors mentioned in the following embodiments of this application may be radar 426, laser rangefinder 428, or camera 430, etc.

[0169] The positioning system 422 can be used to estimate the geographical location of the vehicle 400. An IMU 424 is used to sense changes in the position and orientation of the vehicle 400 based on inertial acceleration. In one embodiment, the IMU 424 can be a combination of an accelerometer and a gyroscope. A radar 426 can use radio signals to sense objects in the surrounding environment of the vehicle 400, specifically millimeter-wave radar or lidar. In some embodiments, in addition to sensing objects, the radar 426 can also be used to sense the speed and / or direction of travel of objects. A laser rangefinder 428 can use lasers to sense objects in the environment in which the vehicle 400 is located. In some embodiments, the laser rangefinder 428 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. A camera 430 can be used to capture multiple images of the surrounding environment of the vehicle 400. The camera 430 can be a still camera or a video camera.

[0170] The control system 406 controls the operation of the vehicle 400 and its components. The control system 406 may include various components, including a steering system 432, a throttle 434, a braking unit 436, a computer vision system 440, a trajectory control system 442, and an obstacle avoidance system 444.

[0171] The steering system 432 is operable to adjust the forward direction of the vehicle 400. For example, in one embodiment, it may be a steering wheel system. The throttle 434 controls the operating speed of the engine 418 and thus the speed of the vehicle 400. The braking unit 436 controls the deceleration of the vehicle 400. The braking unit 436 may use friction to slow down the wheels 421. In other embodiments, the braking unit 436 may convert the kinetic energy of the wheels 421 into electrical current. The braking unit 436 may also take other forms to slow down the rotational speed of the wheels 421 to control the speed of the vehicle 400. The computer vision system 440 is operable to process and analyze images captured by the camera 430 to identify objects and / or features in the environment surrounding the vehicle 400. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 440 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 440 may be used to map the environment, track objects, estimate the speed of objects, etc. The route control system 442 is used to determine the driving route and speed of the vehicle 400. In some embodiments, the route control system 442 may include a lateral planning module 4421 and a longitudinal planning module 4422, which are respectively used to combine data from the obstacle avoidance system 444, GPS 422, and one or more predetermined maps to determine the driving route and speed for the vehicle 400. The obstacle avoidance system 444 is used to identify, evaluate, and avoid or otherwise traverse obstacles in the environment of the vehicle 400, which may specifically be physical obstacles and virtual moving bodies that may collide with the vehicle 400. In one instance, the control system 406 may add or alternatively include components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0172] Vehicle 400 interacts with external sensors, other vehicles, other computer systems, or users via peripheral device 408. Peripheral device 408 may include wireless data transmission system 446, on-board computer 448, microphone 450, and / or speaker 452. In some embodiments, peripheral device 408 provides a means for a user of vehicle 400 to interact with user interface 446. For example, on-board computer 448 may provide information to a user of vehicle 400. User interface 446 may also operate on-board computer 448 to receive user input. On-board computer 448 may be operated via touchscreen. In other cases, peripheral device 408 may provide a means for vehicle 400 to communicate with other devices located within the vehicle. For example, microphone 450 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 400. Similarly, speaker 452 may output audio to a user of vehicle 400. Wireless data transmission system 446 may wirelessly communicate with one or more devices, either directly or via a communication network. For example, the wireless data transmission system 446 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless data transmission system 446 may utilize a wireless local area network (WLAN) for communication. In some embodiments, the wireless data transmission system 446 may utilize an infrared link, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols, such as various vehicle data transmission systems, may also be used. For example, the wireless data transmission system 446 may include one or more dedicated short-range communications (DSRC) devices that may enable public and / or private data communication between vehicles and / or roadside stations.

[0173] Power source 440 can provide power to various components of vehicle 400. In one embodiment, power source 440 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of vehicle 400. In some embodiments, power source 440 and energy source 419 can be implemented together, as is the case in some fully electric vehicles.

[0174] Some or all of the functions of vehicle 400 are controlled by computer system 442. Computer system 442 may include at least one processor 443, which executes instructions 445 stored in a non-transitory computer-readable medium such as memory 444. Computer system 442 may also be multiple computing devices controlling individual components or subsystems of vehicle 400 in a distributed manner. Processor 443 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, processor 443 may be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although... Figure 1 The processor, memory, and other components of computer system 442 within the same block are functionally illustrated; however, those skilled in the art will understand that the processor or memory may actually include multiple processors or memories not stored in the same physical housing. For example, memory 444 may be a hard disk drive or other storage medium located in a housing different from that of computer system 442. Therefore, references to processor 443 or memory 444 will be understood to include a collection of processors or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only related to the component's specific function.

[0175] In all aspects described herein, processor 443 may be located remotely from vehicle 400 and may communicate wirelessly with vehicle 400. In other aspects, some of the processes described herein are executed on processor 443 disposed within vehicle 400 while others are executed by remote processor 443, including taking the necessary steps to perform a single operation.

[0176] In some embodiments, memory 444 may contain instructions 445 (e.g., program logic) that can be executed by processor 443 to perform various functions of vehicle 400, including those described above. Memory 444 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 402, sensor system 404, control system 406, and peripheral devices 408. In addition to instructions 445, memory 444 may also store data such as road maps, route information, vehicle position, direction, speed, and other such vehicle data, as well as other information. This information may be used by vehicle 400 and computer system 442 during operation of vehicle 400 in autonomous, semi-autonomous, and / or manual modes. A user interface 446 is provided to or receives information from a user of vehicle 400. Optionally, user interface 446 may include one or more input / output devices within the set of peripheral devices 408, such as wireless data transmission system 446, onboard computer 448, microphone 450, or speaker 452, etc.

[0177] Computer system 442 can control the functions of vehicle 400 based on input received from various subsystems (e.g., driving system 402, sensor system 404, and control system 406) and from user interface 446. For example, computer system 442 can communicate with other systems or components within vehicle 400 via a CAN bus; for instance, computer system 442 can utilize input from control system 406 to control steering system 432 to avoid obstacles detected by sensor system 404 and obstacle avoidance system 444. In some embodiments, computer system 442 is operable to provide control over many aspects of vehicle 400 and its subsystems.

[0178] Optionally, one or more of these components may be installed separately from or associated with the vehicle 400. For example, the memory 444 may exist partially or completely separately from the vehicle 400. The components may be communicatively coupled together in a wired and / or wireless manner.

[0179] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1This should not be construed as a limitation on the embodiments of this application. The data transmission method provided in this application can be executed by a computer system 442, radar 426, laser rangefinder 430, or peripheral devices such as an on-board computer 448 or other on-board terminals. For example, the data transmission method provided in this application can be executed by an on-board computer 448. The on-board computer 448 can plan a driving path and corresponding speed curve for the vehicle, and generate control commands based on the driving path. The control commands are then sent to the computer system 442, which controls the steering system 432, throttle 434, braking unit 436, computer vision system 440, path control system 442, or obstacle avoidance system 444 in the vehicle's control system 406, thereby achieving autonomous driving of the vehicle.

[0180] The aforementioned vehicle 400 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0181] Monitoring user alertness typically plays a crucial role in various scenarios. For example, when a user is driving, it's essential to monitor their fatigue level promptly and provide timely alerts based on the monitoring results to improve driving safety. However, some commonly used fatigue monitoring methods often fail to detect user fatigue levels in a timely manner.

[0182] For example, existing tools for predicting user fatigue mainly consist of "fatigue biomathematical models." Fatigue biomathematical models are tools used to study the relationship between user sleep-wake behavior, work-rest schedules, sleep-wake behavior, shift fatigue, and actual work performance. Widely used fatigue biomathematical models include "two-stage models" and "three-stage models." Common fatigue biomathematical models consider factors such as circadian rhythms, sleep homeostasis, sleep quality, task type, and task duration. Taking the three-stage model (Three-Process Model of Alertness, TPMA) as an example, this model predicts user alertness levels by analyzing the relationship between circadian rhythms and sleep homeostasis over time. However, 1) key model parameter inputs, such as sleep information (including sleep duration, sleep quality, and task duration), rely on manual reporting and collection, resulting in low information collection efficiency and the possibility of inaccurate or even false reporting, affecting the accuracy of predictions. Furthermore, individual differences are typically not considered in applications, which also affects the accuracy of fatigue level predictions.

[0183] Therefore, this application provides a method for obtaining alertness levels, which combines information collected by the user's wearable device, such as physiological and behavioral information, including sleep, exercise levels, and intake information closely related to fatigue, to predict the user's alertness level over a future period. The method provided in this application will be described in detail below, in conjunction with the aforementioned device.

[0184] First, refer to Figure 5 The flowchart of a method for obtaining alertness provided in this application is as follows.

[0185] 501. Obtain the first information.

[0186] The first information may include information about the generated behavior, which may be collected by various devices, such as referred to as the first device, which may include the aforementioned... Figure 1 Various corresponding devices, such as wearable devices, vehicles, or mobile phones.

[0187] The first piece of information may include information about the user's intake, which can be collected by the device or manually entered by the user.

[0188] Specifically, the information about the ingested substance may include: the time of ingestion, the amount ingested, or the type of ingested substance.

[0189] Optionally, the first information may also include user sleep information or nap information, etc.

[0190] The aforementioned sleep information may specifically include the user's sleep quality score or nap information. The sleep quality score can be obtained by scoring one or more of the following: sleep start time, sleep end time, deep sleep duration, light sleep duration, or sleep cycle.

[0191] The information about a user's lunch break can include details such as whether the user has a habit of taking a lunch break, the start and end times of the lunch break, or a quality score for the lunch break. This quality score can be obtained by scoring one or more of the following: the start and end times of the lunch break, the duration of deep sleep, the duration of light sleep, or the number of sleep cycles.

[0192] In addition, a user's exercise status will also affect the change in the user's alertness level. Therefore, the user's exercise information can also be collected, such as the start time of exercise, the end time of exercise, the duration of exercise, the type of exercise, and the amount of exercise. That is, the first information can also include the user's start time of exercise, the end time of exercise, the duration of exercise, the type of exercise, and the amount of exercise.

[0193] Typically, information such as a user's intake, sleep duration, sleep quality, or nap habits will influence changes in a user's fatigue level over a future period. Therefore, in this application, information such as a user's intake, sleep, or activity can be collected via a device without manual input, thus enabling efficient and accurate collection of relevant user information.

[0194] Furthermore, the method provided in this application can be executed by various devices. For example, if executed by a vehicle, the vehicle can receive user behavior information collected by other devices, or the user behavior information can be collected by devices installed inside the vehicle, such as cameras, radar, or other sensors. If the method provided in this application is executed by a wearable device, the wearable device can directly collect user behavior information, or receive user behavior information collected by other devices, and then execute subsequent steps.

[0195] 502. Predict changes in user alertness based on initial information.

[0196] After obtaining the first information, the user's changes in alertness can be predicted based on the first information. These changes in alertness can include the user's alertness values ​​in the current period and in the future for a certain period of time (referred to as the first preset period for easy distinction).

[0197] Therefore, in the embodiments of this application, the user's alertness level can be predicted based on information such as ingested substances, sleep information, or exercise information collected by the wearable device. Compared with manually inputting information, the method provided by this application can obtain user information more efficiently and accurately, thereby improving the accuracy of prediction. This allows for advance processing of the user's fatigue state and improves the user experience.

[0198] In one specific implementation, the first information can be used as the input to a preset alertness model, and the output can be the user's alertness changes. The output of the alertness model includes the fusion result of the circadian rhythm and the arousal rhythm. The circadian rhythm is obtained from the sleep-related information in the first information, and the arousal rhythm includes the changes in the user's arousal level after sleep.

[0199] Therefore, in this embodiment, an alertness model can be pre-set. This model can be calculated from a large amount of historical user data or manually set by the user, and can be adjusted according to the actual application scenario. After obtaining the user's behavioral information, the user's behavioral information can be used as input to the alertness model. The alertness model can then predict changes in the user's alertness level over a future period, thereby predicting the user's fatigue level in advance and providing timely predictions of the user's state.

[0200] Optionally, in a driving scenario, user driving information collected inside the vehicle can also be obtained, including at least one of driving duration, road type, or traffic information; and the updated alertness changes can be obtained based on the driving information.

[0201] This can be understood as the user expending physical energy while driving the vehicle. This allows the system to update the prediction of changes in alertness in real time based on the user's actual driving conditions. This makes the changes in alertness more closely match the user's actual state, enabling the system to adjust the driving status based on the real-time updates of alertness and improve the user's driving safety.

[0202] Optionally, in one scenario, if the first device detects user behavior including preset behavior within a third preset time period, it can update the alertness change based on the information of the preset behavior, thus obtaining the updated alertness change. For example, the user's historical lunch break time periods can be collected. If the user's lunch break behavior is detected within that lunch break time period on the current day, the alertness change can be adjusted based on the lunch break information. This allows the alertness change to be updated based on the user's real-time behavior, making the final predicted alertness change more accurate.

[0203] Optionally, after obtaining information on changes in the user's alertness level, a driving suitability index can be generated for the user in a specific driving scenario. This index can be used to represent the user's level of safety while driving or the user's level of fatigue.

[0204] Then, a driving suitability index or the user's current alertness value can be played inside the vehicle or on a device with a display screen (such as a second device). This driving suitability index or alertness value can indicate the user's current state, thereby reminding the user to drive cautiously and improving the user's driving safety.

[0205] Optionally, after obtaining information on changes in the user's alertness level, the vehicle's driving information can be adjusted based on the user's alertness level for different driving scenarios. For example, in a user's driving scenario, vehicle driving information such as the vehicle's maximum speed, interior temperature, and ventilation mode can be adjusted to better match the vehicle's driving status with the user's alertness level, thereby improving the user's driving safety.

[0206] Typically, the method for adjusting vehicle driving information can be tailored to the specific application scenario. For example, if the method provided in this application is executed by a terminal, such as a user's wearable device, mobile phone, or tablet, an adjustment command can be sent to the vehicle. This command can include information about changes in alertness levels, allowing the vehicle to adaptively adjust its various parameters based on these changes. Alternatively, the specific vehicle parameters requiring adjustment can be determined directly based on the changes in alertness levels, and then this parameter can be included in the adjustment command, instructing the vehicle to adjust its parameters accordingly. If the method provided in this application is executed by the vehicle itself, the vehicle can directly adjust its various parameters based on these changes in alertness levels.

[0207] Optionally, one or more of the following parameters of the vehicle can be adjusted: the alarm range of the in-vehicle eye-closing detection system, which is used to detect whether at least one user closes their eyes while driving the vehicle and triggers an alarm when the detection result is within the alarm range; the alarm information of the in-vehicle visual driver monitoring system (DMS), including the fatigue alarm range, reminder period, or reminder intensity; or, the vehicle's navigation information, including at least one of the following: navigation route, driving time along the navigation route, or road type in the navigation route; or, acquiring the alertness information of at least one traffic participant adjacent to the vehicle and acquiring the driving dynamics of at least one traffic participant based on the alertness information of at least one traffic participant; or, adjusting the detection threshold of the in-vehicle alarm system, the detection threshold being related to the sensitivity of the alarm system to trigger an alarm.

[0208] Therefore, in this embodiment, the parameters of various hardware or software systems in the vehicle can be adjusted based on the predicted level of user alertness, so that the various parameters in the vehicle match the level of user alertness, thereby improving the user's driving safety.

[0209] Optionally, before driving the vehicle, if there are at least two users in the vehicle, the optimal user can be selected as the first user based on the driving suitability index of these two users, and a first prompt message can be generated for the first user to prompt the first user to drive the vehicle. Therefore, in this embodiment, when there are multiple users in the vehicle, a more suitable user can be selected from among the multiple users, thereby improving the driving safety of the vehicle.

[0210] Optionally, the system can obtain the input information of a second user, and then determine the user's travel plan for a second preset time period in the future based on the input information. Based on the travel plan and preset conditions, a second prompt message is generated for the second user. The second prompt message is used to remind the second user of their current movement status, that is, to make suggestions on the user's current movement status, so that the user's alertness level in the second preset time period is more suitable for driving the vehicle, thereby improving the user's driving safety.

[0211] Optionally, when generating the second prompt message for the second user, the message can be generated based on the trip plan and the second user's activity information during the fourth preset time period, such as the amount or duration of the activity. This can be understood as follows: when a user has a trip planned for the future, the system can remind the user to exercise appropriately based on their current activity level, thereby increasing the user's alertness and improving driving safety.

[0212] The foregoing has described the process of the method provided in this application. For ease of understanding, the following section will describe it in the context of specific application scenarios.

[0213] The method provided in this application can be executed by a terminal, such as a wearable device or other mobile terminal, or by a vehicle. The following describes different scenarios using wearable devices, mobile phones, and vehicles as examples.

[0214] I. Wearable devices

[0215] See Figure 6 The flowchart of another alertness procedure acquisition method provided in this application is as follows.

[0216] 601. Wearable devices collect user behavior information.

[0217] Wearable devices can include wristbands, smartwatches, head-mounted displays, headphones, smart glasses, and other similar devices. These devices can use sensors such as accelerometers, gyroscopes, optical heart rate sensors, body temperature sensors, and light sensors to collect user behavior information, i.e., primary information.

[0218] Furthermore, the number of wearable devices can be one or more. Some behaviors can be detected by multiple wearable devices. For example, a user's eating or drinking behaviors can be detected by devices such as smartwatches, smart glasses, headphones, or cameras. When a smartwatch detects a user's behavior, it can make a preliminary judgment on the user's movement type based on the direction and speed of the movement. Then, other devices, such as smart glasses or cameras, can collect the user's specific behavior. In this way, multiple devices can be combined to accurately detect the user's specific behavior.

[0219] User behavior information can specifically include information such as the time period, type, or exercise expenditure of the user's behavior. For example, it can include information such as the user's exercise status, sleep start time, sleep end time, deep sleep period, light sleep period, nap period, food intake time, food intake type, food intake amount, exercise time, exercise type, or exercise intensity.

[0220] For example, behavioral information can include multiple types:

[0221] (1) Sleep information:

[0222] Sleep start time, sleep end time, sleep quality, whether or not a nap is taken, nap start time, nap end time, or nap quality, etc.

[0223] Among them, the user's sleep quality can be evaluated in a pre-set way, such as the longer the user's deep sleep duration, the better the sleep quality; or the higher the ratio of deep sleep to light sleep, the better the user's sleep quality; or within a certain range, the longer the sleep duration, the better the user's sleep quality; or the fewer times the user moves during sleep, the better the user's sleep quality, etc. The specifics can be adjusted according to the actual application scenario.

[0224] (2) Motion Information

[0225] Exercise information can include the user's start and end times, exercise type, or exercise volume. For example, if a user runs during a certain period, the wristband worn by the user can collect information such as the start and end times of the run, running speed, and heart rate changes.

[0226] (3) Dietary information

[0227] Dietary information can include information related to the food a user consumes. This may include whether the user ate lunch, the time of lunch, the amount of lunch, whether they took medication, the time of medication, whether they consumed alcohol or alcoholic beverages, the time of drinking, and the alcohol content.

[0228] The number of wearable devices can be one or more. When there are multiple wearable devices, the multiple wearable devices can collect the user's behavior information and then send the collected behavior information to one of the wearable devices in real time or periodically, so that one of the wearable devices can execute the method process provided in this application.

[0229] In addition, in scenarios where users are driving, wearable devices can also receive information from other devices in the vehicle, such as cameras, microphones, radar, or infrared sensors that can reflect user behavior. For example, cameras installed in the vehicle can collect information about the user's specific movements; users can actively input specific behavioral information through microphones; radar can detect the user's limb movements or gestures; or infrared sensors can detect changes in the user's position. The specifics can be adjusted according to the actual application.

[0230] 602. Wearable devices predict changes in user alertness.

[0231] After the wearable device obtains the user's behavioral information, it can predict the user's changes in alertness over a future period (i.e., a first preset time period) or longer. This means predicting the user's changes in alertness value within the first preset time period. The alertness value reflects the user's level of alertness. Typically, it can be set to have a positive correlation between alertness value and alertness level; for example, a higher alertness value indicates a higher level of alertness and a lower level of fatigue. Alternatively, it can be set to have a negative correlation between alertness value and alertness level; for example, a higher alertness value indicates a lower level of alertness and a higher level of fatigue. The specific setting can be determined based on the actual application scenario.

[0232] Furthermore, steps 601 and 602 can be executed iteratively. For example, the wearable device can collect the user's behavioral information in real time or at a preset period. During the collection of behavioral information, the changes in alertness can be updated in real time, thereby adjusting the changes in alertness according to the actual application scenario to better match the user's current alertness state.

[0233] For example, a prediction cycle can be preset, such as predicting changes in user alertness every 30 minutes based on the collected information. The collected information can then be used as input to a pre-set alertness model to predict the user's alertness level over a future period.

[0234] For example, based on a three-stage arousal model, information related to the user's physiology and behavior can be used as adjustment parameters to calculate the user's current alertness prediction value. The different components, adjustment parameters, and calculation formulas of the alertness prediction model are as follows.

[0235] First, the basic circadian rhythm curve can be represented as:

[0236]

[0237] Where t represents time (e.g., decimal hours can be used); p represents half rhythm, such as 12 (hours); and m represents rhythm amplitude (e.g., the amplitude can be 2.5 by default in a three-stage model).

[0238] The first half of the arousal rhythm, S, refers to the continuous decline in alertness after arousal, which can be represented as:

[0239] S(t)=(S a -L)e H(t) +L,

[0240] where H(t) = h1(t) + h2(t) + h3(t) + ... (Formula 2)

[0241] Where Sa represents the user's alertness value after waking up, which is also related to sleep quality; L represents the default minimum alertness value of 2.4 in the three-stage model; h1(t) represents the impact of a midday nap; h2(t) represents the impact of exercise; h3(t) represents the impact of food-induced drowsiness; in addition, other influencing factors can be included, which will not be listed here.

[0242] Typically, if factors such as lunch break, physical activity, and post-meal drowsiness are absent, or if these factors are not detected, then S(t) can degenerate into...

[0243]

[0244] where t2=(t+24-W)%24 (Formula 3)

[0245] Where W represents the moment of wakefulness during the day.

[0246] The latter half of the arousal rhythm, S', which is the gradual recovery of alertness after falling asleep, can be represented as:

[0247]

[0248] where t3=(t+24-V)%24 (Formula 4)

[0249] Where t represents the current time; t3 represents the duration of the sleep state (usually in decimal hours); U represents the default maximum arousal value of the three-stage model, such as 14.3 by default; and Sr represents the last alertness value before falling asleep.

[0250] For example, based on the above formula "going to bed at 10 pm and waking up at 6 am, without being significantly affected by other factors, the diurnal and wakefulness rhythm curve can be as follows..." Figure 7 As shown.

[0251] When making alertness predictions, the curves of C and S (including S') can be combined to obtain the alertness prediction curve A.

[0252] For example, it can be represented as:

[0253] A(t) = C(t) + S(t) (Formula 5)

[0254] For example, such as Figure 8 The figure shows the alertness prediction curve for users who "wake up at 6 a.m., go to bed at 10 p.m., and have good sleep quality without being significantly affected by other factors".

[0255] Typically, when making alertness predictions, predictions can be made based on the information already detected. When the wearable device detects an event that affects alertness, the alertness prediction curve can be updated in a timely manner, thereby achieving personalized alertness predictions based on the user's state.

[0256] For example, the execution flow of a wearable device can be as follows: Figure 9 As shown.

[0257] First, wearable devices can detect user behavior in real time or receive information sent by other devices.

[0258] When information is received or collected, it is determined whether there are any events that affect alertness. If not, the user's alertness curve can be predicted according to the methods provided by Formula 1, Formula 3, Formula 4 and Formula 5.

[0259] If there are events that affect alertness, the alertness prediction curve can be adjusted based on the detected events and the aforementioned Formulas 1, 3, 4 and 5, and then the adjusted alertness prediction curve can be output.

[0260] For example, the following section introduces some events that may affect alertness and the corresponding methods for adjusting the alertness prediction curve.

[0261] (1) Sleep time

[0262] "Wake-up time" and "sleep time," i.e., changing the parameters W and V in the model, will result in a shift in the curve. For example... Figure 10 As shown, the solid line represents the alertness prediction curve for users who "wake up at 6 a.m., go to bed at 10 p.m., and have good sleep quality," while the dashed line represents the alertness prediction curve for users who "wake up at 8 a.m., go to bed at 12 a.m., and have good sleep quality."

[0263] (2) Sleep quality

[0264] This means changing the parameter Sa, which represents the highest arousal value after awakening. For example... Figure 11As shown, the solid line represents the alertness prediction curve for "waking up at 6 a.m., going to bed at 10 p.m., and having good sleep quality", while the dashed line represents the alertness prediction curve for "waking up at 6 a.m., going to bed at 10 p.m., but having poor sleep quality".

[0265] (3) Has the habit of taking a midday nap and takes a midday nap

[0266] "Having a habit of taking a midday nap, and napping for one hour," means changing the model parameter h1(t). The assumption behind this parameter change is that for the group with a midday nap habit, the rate of decline in alertness accelerates before the nap; however, after the nap, this downward trend in alertness is interrupted, and alertness recovers rapidly. For example... Figure 12 As shown, the solid line represents the alertness prediction curve for "waking up at 6 am, going to bed at 10 pm, sleeping for 8 hours with good sleep quality, and not having a habit of taking a nap at noon", while the dashed line represents the alertness prediction curve for "waking up at 6 am, going to bed at 10 pm, sleeping for 8 hours with good sleep quality, having a habit of taking a nap at noon, and taking a one-hour nap from 12 pm to 1 pm".

[0267] (4) Has the habit of taking a midday nap, but does not take one.

[0268] This involves changing the model parameter h1(t). The assumption behind this parameter change is that for groups with a habit of taking a midday nap, the rate of decline in alertness accelerates as the nap approaches; however, because they do not take a midday nap, the recovery of the alertness curve is slow, resulting in an overall decrease in alertness. For example... Figure 13 As shown, the solid line represents the alertness prediction curve for someone who "wakes up at 6 a.m., goes to bed at 10 p.m., gets 8 hours of sleep with good quality, and has a habit of taking a nap for one hour between 12 p.m. and 1 p.m.". The dashed line represents the alertness prediction curve for someone who "wakes up at 6 a.m., goes to bed at 10 p.m., gets 8 hours of sleep with good quality, and has a habit of taking a nap, but does not take one".

[0269] (5) Exercise expenditure

[0270] "Exercise expenditure (including long-distance driving)" refers to changing the model parameter h2(t). The assumption behind this parameter change is that the alertness curve declines rapidly during exercise expenditure and recovers slowly after exercise. For example... Figure 14 As shown, the solid line represents the alertness prediction curve for "waking up at 6 am, going to bed at 10 pm, getting 8 hours of sleep with good sleep quality, and engaging in moderate daily activities." The dashed line represents the alertness prediction curve for "waking up at 6 am, going to bed at 10 pm, getting 8 hours of sleep with good sleep quality, and engaging in one hour of high-intensity physical exercise at 3 pm."

[0271] (6) Diet

[0272] "Lunchtime overeating / drinking alcohol / taking medication" refers to changing the model parameter h3(t). Taking lunch as an example, the assumption behind this parameter change is that the alertness curve declines more rapidly after lunch, then slowly recovers after about one or two hours. For example... Figure 15 As shown, the solid line represents the alertness prediction curve for "good sleep quality the night before and moderate food intake at noon." The dashed line represents the alertness prediction curve for "good sleep quality the night before and overeating at noon."

[0273] 603. Wearable devices calculate the driving suitability index and send it to the vehicle.

[0274] The driving suitability index can be positively correlated with the current level of alertness. This positive correlation can be linear or non-linear. For example, the higher the level of alertness, the higher the driving suitability index, indicating that the user is safer driving the vehicle.

[0275] Step 603 is optional. The wearable device can calculate the driving suitability index or it can choose not to perform step 603. The specific steps can be adjusted according to the actual application scenario, and this application does not impose any restrictions.

[0276] Specifically, the driving suitability index is determined based on the calculated alertness value at the current moment. For example, based on the range of alertness levels, 15 is the maximum value of Sa and 7 is the sleep threshold of Sr, which can be used to identify three levels of suitability index: (1) when the alertness value is >= 13, the driving suitability index is "suitable"; (2) when the alertness value is between 13 and 8, the driving suitability index is "relatively suitable"; (3) when the alertness value is < 8, the driving suitability index is "unsuitable". For example Figure 16 The alertness curves of users who "go to bed at 10 pm, wake up at 6 am, and have good sleep quality" show that the suitable time range is 6-14 pm, the more suitable time ranges are 14-20 pm and 23-5 am, and the unsuitable time range is 20-23 pm.

[0277] 604. Vehicle plays driving suitability index.

[0278] The vehicle can be equipped with a display screen or speaker. When the vehicle receives the driving suitability index sent by the wearable device, it can play the driving suitability index through the display screen or speaker, so that users can know the driving suitability index in time and improve the user's driving safety.

[0279] Furthermore, if there are multiple users in the vehicle, the system can filter out the more suitable driver based on each user's driving suitability index and generate a prompt message to remind the more suitable driver to take the lead, thereby improving driving safety.

[0280] For example, when there are multiple drivers, the vehicle can suggest which driver is more suitable to drive. If the vehicle obtains information about two drivers / passengers who have boarded the vehicle, and obtains the driving suitability index of user A as "suitable" and user B as "unsuitable" from their wearable devices, the vehicle can provide feedback through voice or vision, such as "A did not take a lunch break and has a low driving suitability index. It is recommended that B drive."

[0281] The feedback can also include suggestions for what the user should consume, such as during rest breaks on long drives. When the vehicle stops at a service station, the system can provide feedback via voice or visual means, such as "Please eat xx food at noon, it will help you with your driving."

[0282] 605. Wearable devices send adjustment commands to the vehicle.

[0283] In driving scenarios, after receiving information about changes in the user's alertness, wearable devices can adjust vehicle driving information based on these changes, such as adjusting the alarm threshold for the vehicle's closed-eye detection, alarm information from the DMS (Driver Monitoring System), the driving status of other road users, navigation information, or the detection threshold of the alarm system.

[0284] The adjustment command can carry an alert value or directly instruct the vehicle how to adjust parameters during driving. For example, when the adjustment command carries an alert value, the vehicle, upon receiving the command, can determine how to adjust various parameters based on that alert value.

[0285] 606. Adjust vehicle driving information.

[0286] The following examples illustrate some application scenarios.

[0287] (1) Eyes closed test

[0288] Among them, the closed-eye detection is to detect whether the driver's eyes are closed or the degree of eye closure during driving. When the driver's eyes are closed or the degree of eye closure is high, an alarm can be triggered to remind the user to drive safely or to take a rest.

[0289] When a user's alertness level is lower than a preset value, the detection threshold for the closed-eye detection can be adjusted to detect the degree of eye closure in a timely manner, issue an early warning, and improve the user's driving safety.

[0290] (2) DMS alarm range

[0291] The vehicle can adjust the alarm thresholds and feedback settings of the visual DMS system based on alertness and driving suitability index. The visual DMS system can be integrated into the vehicle or a standalone DMS system paired with it.

[0292] If the driving suitability index is "relatively suitable" or "unsuitable," the DMS system will lower the critical threshold for fatigue alarms while increasing the density and intensity of fatigue warnings. If the driving suitability index is in the "suitable" range, the critical threshold, warning density, or intensity of fatigue alarms will remain unchanged.

[0293] (3) Navigation adjustment

[0294] The vehicle outputs its alertness level and driving suitability index to the navigation system. After initiating route navigation, the navigation system adjusts relevant information based on the driving suitability index, which may include, but is not limited to, the following:

[0295] A. Estimated Trip Duration

[0296] If the driver's alertness level is low and their driving suitability index is in the "relatively suitable" or "unsuitable" range, the estimated driving time will be extended accordingly. Otherwise, no adjustment is needed.

[0297] For example, such as Figure 17 As shown, when the driver's alertness is normal and the driving suitability index is "suitable", the estimated travel time from the starting point "my location" to the destination "xx city" is 2 hours and 23 minutes. When the driver's alertness is low and the driving suitability index is "relatively suitable", the estimated travel time from the starting point "my location" to the destination "xx city" is extended to 2 hours and 53 minutes.

[0298] B. Path Recommendation

[0299] If the driver's alertness level is low and their driving suitability index is in the "relatively suitable" or "unsuitable" range, then routes with less traffic and avoiding highways are recommended. Figure 18 As shown, the predicted driver alertness value is low, which is in the "relatively suitable" zone. The navigation actively recommends non-highway routes (such as those marked by solid lines, which have a high recommendation index) and does not recommend routes that are entirely on highways (such as those marked by dashed lines, which have a low recommendation index).

[0300] C. Environmental Monitoring

[0301] It can dynamically predict the safety of vehicles traveling together on a road segment, or predict the safety of that road segment itself. It can monitor driver alertness in real time and upload the data to the cloud. Driver alertness information for the same road segment can be displayed on a (third-person view) navigation system or HUD, or stored for later analysis and prediction of road segment safety.

[0302] Alternatively, users can plan their trips in advance. For example, if a user plans to go out the next day, they can search for routes on their phone the night before. The phone can then prompt the user, either by providing a map or through a service card, with messages like, "You'll be driving tomorrow, so please eat xx food tonight and go to bed at xx o'clock!" When the user starts driving at the planned time, the phone or in-car system may prompt them with, "You ate xx last night, so you'll be easily fatigued today. We recommend taking a break after driving for an hour."

[0303] Optionally, when users plan their trips in advance, the wearable device can also provide feedback since the data is acquired from it. For example, according to the model calculation of the present invention, if a user is going to take a long-distance driving trip the next day, and the amount of exercise (in steps, for example) exceeds 20,000 steps or the exercise time exceeds 21:00, the driving suitability index will be "unsuitable". Then, when the step count on the wearable device is about to reach 20,000 steps, or when the exercise time exceeds 21:00, the wearable device will prompt the user, "You will be driving tomorrow, so it is recommended not to do a lot of exercise." Another example is that if the food intake exceeds xx, the driving suitability index will be "unsuitable". Then, when it is determined that the food intake exceeds xx, the user will be prompted, "You will be driving tomorrow, so it is recommended not to eat too much."

[0304] In some scenarios, when a user's intake contains alcohol, a wearable device or mobile service card will prompt the user with "You drank alcohol at x o'clock and xx minutes. If you need to drive, please do so after xx minutes."

[0305] (4) Alarm System

[0306] Other intelligent adjustment systems in the vehicle include an adjustment warning system, such as a collision detection system. The lower the driving suitability index, the higher the threshold of the collision detection system. For example, when the driving suitability index is "suitable", the vehicle will play an alarm sound when other vehicles or pedestrians appear within 1 meter of the vehicle. When the vehicle index is "relatively suitable", the vehicle will play an alarm sound when other vehicles or pedestrians appear within 2 meters of the vehicle, in order to increase the driver's reaction time.

[0307] (5) Other equipment

[0308] If the driving suitability index is in the "relatively suitable" or "unsuitable" range, the vehicle can proactively recommend more suitable air conditioning temperatures (e.g., lower temperatures), seat angles (e.g., a backrest angle closer to 90 degrees), and interior lighting brightness and color (e.g., higher brightness, using blue or orange), for the user to select or confirm. If the driving suitability index is in the "suitable" range, no recommendations or changes will be made.

[0309] 607. The vehicle collects driving records and sends the driving records to the wearable device.

[0310] While the user is driving, the vehicle can record the user's driving data and then periodically send it to the wearable device.

[0311] For example, during vehicle operation, every short interval (e.g., 5 minutes), the vehicle's infotainment system will synchronize the recorded driving information, such as driving time, to the wearable device, which will then calculate the latest alertness value and driving suitability index.

[0312] 608. Updates on wearable device alert status.

[0313] The process of updating the changes in alertness is similar to step 602 described above, and will not be repeated here.

[0314] Therefore, in this embodiment, a parameterized fatigue biomathematical model can be constructed using data acquired by wearable devices as input parameters. Unlike manual information collection methods, acquiring user physiological and activity information through sensors on wearable devices results in high efficiency and accuracy in information collection. Furthermore, through the interconnection and information exchange between wearable devices and vehicles, cross-scenario applications for predicting and warning of user fatigue can be achieved. Information acquired from non-driving scenarios is used for fatigue prediction and warning in driving scenarios. Moreover, intelligent control of in-vehicle equipment or systems is implemented based on predictions of the driver's alertness and driving suitability index.

[0315] II. Mobile Terminals

[0316] Typically, wearable devices have limited storage capacity and computing power, so they need to rely on mobile terminals as intermediaries for data storage and computation. In driving scenarios, mobile terminals also connect with vehicles for information exchange.

[0317] like Figure 19 As shown in the figure, a flowchart of another method for obtaining alertness provided in this application is described below.

[0318] 1901. Wearable devices collect user behavior information and send it to mobile terminals.

[0319] The difference between step 1901 and the aforementioned step 601 is that in step 601, the wearable device does not need to send the behavior information to the mobile terminal, while in step 1901, the wearable device needs to send the collected user behavior information to the mobile terminal. Similarities will not be elaborated further.

[0320] 1902. Terminal prediction of changes in user alertness.

[0321] 1903. The mobile terminal calculates the driving suitability index and sends it to the vehicle.

[0322] 1904. Vehicle plays driving suitability index.

[0323] 1905. The mobile terminal sends adjustment instructions to the vehicle.

[0324] 1906. Adjust vehicle driving information.

[0325] 1907. The vehicle collects driving records and sends the driving records to the mobile terminal.

[0326] 1908. Mobile terminal updates alert status.

[0327] The difference between steps 1902-1908 and the aforementioned steps 602-608 is that the steps performed by the wearable device in the aforementioned steps 602-608 can be replaced by the steps performed by the mobile terminal. For details, please refer to the description of the aforementioned steps 602-608, which will not be repeated here.

[0328] Therefore, in this embodiment, the terminal can predict changes in the user's level of alertness, and through the interconnection and information exchange between the wearable device and the vehicle, cross-scenario applications for predicting and warning of user fatigue can be realized.

[0329] III. Vehicles

[0330] The method provided in this application can also be performed by a vehicle.

[0331] like Figure 20 As shown in the figure, a flowchart of another method for obtaining alertness provided in this application is described below.

[0332] 2001. Wearable devices collect user behavior information and send it to mobile vehicles.

[0333] The difference between step 2001 and the aforementioned step 601 is that in step 601, the wearable device does not need to send the behavior information to the mobile terminal, while in step 1901, the wearable device needs to send the collected user behavior information to the mobile terminal. Similarities will not be elaborated further.

[0334] 2002, predicting changes in user alertness in vehicles.

[0335] 2003, Calculate the driving suitability index for vehicles.

[0336] 2004, Vehicles play driving suitability index

[0337] 2005, vehicle driving information was adjusted.

[0338] 2006. Vehicles collect driving records and update alert status changes.

[0339] The difference between steps 2002-2006 and the aforementioned steps 602-608 is that the steps performed by the wearable device in the aforementioned steps 602-608 can be replaced by the vehicle. For details, please refer to the description of the aforementioned steps 602-608, which will not be repeated here.

[0340] Therefore, in this embodiment of the application, the vehicle can predict changes in the user's alertness level, reducing the data interaction process. The vehicle can directly predict the user's alertness state, thus improving the efficiency of alertness prediction.

[0341] The foregoing has provided a detailed description of the method flow provided in this application. The following section describes the structure of the device provided in this application.

[0342] See Figure 21 The present application provides a schematic diagram of the structure of a wearable device, including:

[0343] The acquisition module 2101 is used to acquire first information, which includes information about user behavior, information collected by a first device, a wearable device of the user, and information about the user's ingested substances.

[0344] The processing module 2102 is used to predict the user's changes in alertness based on the first information, which includes the user's alertness value within a first preset time period.

[0345] In one possible implementation, the first information may also include the user's sleep information or exercise information.

[0346] In one possible implementation, sleep information includes at least one of: sleep quality score and nap information;

[0347] Exercise information includes at least one of the following: exercise start time, exercise end time, exercise type, or exercise volume;

[0348] Information on ingested substances includes at least one of the following: the time of ingestion, the amount ingested, or the type of ingested substance.

[0349] In one possible implementation, the processing module 2102 is specifically used to take the first information as input to a preset alertness model and output the user's alertness changes. The output of the alertness model includes the fusion result of the circadian rhythm and the arousal rhythm. The circadian rhythm is obtained from the sleep-related information in the first information, and the arousal rhythm includes the changes in the user's arousal level after sleep.

[0350] In one possible implementation, the processing module 2102 is further configured to determine the alarm range of an eye-closing detection system configured in the vehicle. This system detects whether the user has closed their eyes while driving and issues an alarm when the detection result falls within the alarm range. Alternatively...

[0351] Adjust the alarm information of the visual driver monitoring system (DMS) installed in the vehicle, including the fatigue alarm range, reminder cycle, or reminder intensity; or,

[0352] Adjust the vehicle's navigation information, which includes at least one of the following: navigation route, driving time along the navigation route, or road type along the navigation route; or,

[0353] Acquire alertness information from at least one traffic participant adjacent to the vehicle, and obtain the driving dynamics of at least one traffic participant based on the alertness information of at least one traffic participant; or,

[0354] Adjust the detection threshold of the alarm system installed in the vehicle. The detection threshold is related to the sensitivity of the alarm system in triggering an alarm.

[0355] In one possible implementation, the acquisition module 2101 is further configured to acquire the user's driving information collected in the vehicle, the driving information including at least one of: driving duration, road type or traffic information;

[0356] The processing module 2102 is also used to update the alertness change based on the driving information and obtain the updated alertness change.

[0357] In one possible implementation, the number of users is at least two, and the processing module 2102 is further configured to:

[0358] The first user is selected from at least two users based on the driving suitability index of at least two users;

[0359] Generate a first prompt message for the first user, which is used to prompt the first user to drive the vehicle.

[0360] In one possible implementation, the acquisition module 2101 is further configured to acquire input information from the second user;

[0361] The processing module 2102 is also used to obtain the second user's itinerary plan for a second preset time period in the future based on the input information;

[0362] The processing module 2102 is also used to generate a second prompt message for the second user based on the trip plan and preset conditions. The second prompt message is used to remind the second user of the current movement status.

[0363] In one possible implementation, the playback module 2103;

[0364] The processing module 2102 is also used to obtain a driving suitability index based on the alertness value. The driving suitability index is positively correlated with the alertness level and is used to remind the user of the safety level of driving the vehicle.

[0365] Playback module 2103 is used to play at least one of alertness value or driving suitability index via a second device.

[0366] In one possible implementation, the processing module 2102 is further configured to update the alertness change status based on the information of the user's preset behavior if the first device detects that the user's behavior includes preset behavior within a third preset time period, thereby obtaining the updated alertness change status.

[0367] See Figure 22 This application provides a schematic diagram of the structure of a vehicle, including:

[0368] The acquisition module 2201 is used to acquire first information, which includes information about user behavior, information collected by a first device, a wearable device of the user, and information about the user's ingested substances.

[0369] The processing module 2202 is used to predict the user's changes in alertness based on the first information, which includes the user's alertness value within a first preset time period.

[0370] In one possible implementation, the first information may also include the user's sleep information or exercise information.

[0371] In one possible implementation, sleep information includes at least one of: sleep quality score and nap information;

[0372] Exercise information includes at least one of the following: exercise start time, exercise end time, exercise type, or exercise volume;

[0373] Information on ingested substances includes at least one of the following: the time of ingestion, the amount ingested, or the type of ingested substance.

[0374] In one possible implementation, the processing module 2202 is specifically used to take the first information as input to a preset alertness model and output the user's alertness changes. The output of the alertness model includes the fusion result of the circadian rhythm and the arousal rhythm. The circadian rhythm is obtained from the sleep-related information in the first information, and the arousal rhythm includes the changes in the user's arousal level after sleep.

[0375] In one possible implementation, the processing module 2202 is further configured to determine the alarm range of an eye-closing detection system configured in the vehicle. This system detects whether the user's eyes are closed while driving and issues an alarm when the detection result falls within the alarm range. Alternatively...

[0376] Adjust the alarm information of the visual driver monitoring system (DMS) installed in the vehicle, including the fatigue alarm range, reminder cycle, or reminder intensity; or,

[0377] Adjust the vehicle's navigation information, which includes at least one of the following: navigation route, driving time along the navigation route, or road type along the navigation route; or,

[0378] Acquire alertness information from at least one traffic participant adjacent to the vehicle, and obtain the driving dynamics of at least one traffic participant based on the alertness information of at least one traffic participant; or,

[0379] Adjust the detection threshold of the alarm system installed in the vehicle. The detection threshold is related to the sensitivity of the alarm system in triggering an alarm.

[0380] In one possible implementation, the acquisition module 2201 is further configured to acquire the user's driving information collected in the vehicle, the driving information including at least one of: driving duration, road type or traffic information;

[0381] The processing module 2202 is also used to update the alertness change based on the driving information and obtain the updated alertness change.

[0382] In one possible implementation, the number of users is at least two, and the processing module 2202 is further configured to:

[0383] The first user is selected from at least two users based on the driving suitability index of at least two users;

[0384] Generate a first prompt message for the first user, which is used to prompt the first user to drive the vehicle.

[0385] In one possible implementation, the acquisition module 2201 is further configured to acquire input information from the second user;

[0386] The processing module 2202 is also used to obtain the second user's itinerary plan for a second preset time period in the future based on the input information;

[0387] The processing module 2202 is also used to generate a second prompt message for the second user based on the trip plan and preset conditions. The second prompt message is used to remind the second user of the current movement status.

[0388] In one possible implementation, the vehicle further includes: a playback module 2203;

[0389] The processing module 2202 is also used to obtain a driving suitability index based on the alertness value. The driving suitability index is positively correlated with the alertness level and is used to remind the user of the safety level of driving the vehicle.

[0390] The playback module 2203 is used to play at least one of alertness value or driving suitability index via a second device.

[0391] In one possible implementation, the processing module 2202 is further configured to update the alertness change status based on the information of the user-generated preset behavior if the first device detects that the user-generated behavior includes preset behavior within a third preset time period, thereby obtaining the updated alertness change status.

[0392] Please see Figure 23 The following is a schematic diagram of another electronic device provided in this application.

[0393] The electronic device may include the aforementioned wearable device, terminal, or vehicle, and may include a processor 2301, a memory 2302, and a transceiver 2303. The processor 2301 and the memory 2302 are interconnected via a circuit. The memory 2302 stores program instructions and data.

[0394] The aforementioned are stored in memory 2302 Figures 5-20 The steps in the code include the corresponding program instructions and data.

[0395] Processor 2301 is used to perform the aforementioned Figures 5-20 The method steps performed by the first device or electronic device shown in any of the embodiments.

[0396] Transceiver 2303 is used to perform the aforementioned Figures 5-20 The steps of receiving or transmitting data performed by the first device or electronic device shown in any of the embodiments.

[0397] Optionally, the device may also include a display screen 2304 for displaying the aforementioned... Figures 5-20 The interface of the first device or electronic device in the system is used for display.

[0398] Optionally, the device may also include a speaker 2305 for playing the aforementioned... Figures 5-20 The voice played by the first device or electronic device in the system.

[0399] This application also provides a computer-readable storage medium storing a program for generating vehicle speed, which, when run on a computer, causes the computer to perform the aforementioned... Figures 5-20 The steps in the method described in the illustrated embodiment.

[0400] Alternatively, the aforementioned Figure 23 The electronic device shown is a chip.

[0401] This application also provides an electronic device, which may also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform the aforementioned... Figures 5-20 The method steps performed by the electronic device shown in any of the embodiments.

[0402] This application also provides a digital processing chip. This digital processing chip integrates circuitry for implementing the processor 2301 described above, or the functions of processor 2301, and one or more interfaces. When the digital processing chip integrates a memory, it can complete the method steps of any one or more of the foregoing embodiments. When the digital processing chip does not integrate a memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the actions performed by the electronic device in the foregoing embodiments based on the program code stored in the external memory.

[0403] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 5-20 The steps performed by the electronic device in the method described in the illustrated embodiment.

[0404] The electronic device provided in this application embodiment can be a chip, which includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in a storage unit to cause the chip within the server to perform the aforementioned operations. Figures 5-20 The device search method described in the illustrated embodiment. Optionally, the storage unit is a storage unit within the chip, such as a register, cache, etc. The storage unit can also be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), etc.

[0405] Specifically, the aforementioned processing unit or processor can be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0406] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more processors used to control the above. Figures 5-20 The method of program execution of integrated circuits.

[0407] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0408] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0409] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0410] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0411] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0412] Finally, it should be noted that the above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of obtaining a degree of alertness, characterized by, The method comprises: obtaining first information, the first information comprising information of a user-generated behavior, the first information comprising information collected by a first device, the first device comprising a wearable device of the user, the first information comprising information of an intake of the user; predicting a vigilance change of the user based on the first information, the vigilance change comprising a vigilance value of the user within a first preset time period; adjusting at least one of the following of a vehicle according to the vigilance value: adjusting an alarm range of an eye closure detection system configured in the vehicle, the eye closure detection system being configured to detect whether the user closes eyes when driving the vehicle and to alarm when the detection result is within the alarm range; or adjusting alarm information of a driver monitoring system (DMS) configured in the vehicle, the alarm information comprising a fatigue alarm range, a reminding period or a reminding intensity; or adjusting navigation information of the vehicle, the navigation information comprising at least one of a navigation path, a driving time according to the navigation path or a road type in the navigation path; or obtaining vigilance information of at least one traffic participant adjacent to the vehicle and obtaining driving dynamics of the at least one traffic participant according to the vigilance information of the at least one traffic participant; or adjusting a detection threshold of an alarm system configured in the vehicle, the detection threshold being related to a sensitivity of the alarm system. The first information further comprises sleep information or motion information of the user.

3. The method of claim 2, wherein: the sleep information comprises at least one of a sleep quality score or nap information; 2. The method of claim 1, wherein, the motion information comprises at least one of a motion start time, a motion end time, a motion type or a motion amount; the information of the intake comprises at least one of an intake time, an intake amount or an intake type. The prediction of the vigilance change of the user based on the first information comprises: inputting the first information into a preset vigilance model, the vigilance model outputting the vigilance change of the user, the output of the vigilance model comprising a fusion result of a circadian rhythm and an arousal rhythm, the circadian rhythm being obtained based on sleep-related information in the first information, the arousal rhythm comprising a change of an arousal degree of the user after sleep. The method further comprises:

4. The method according to any one of claims 1-3, characterized in that, obtaining driving information of the user collected in the vehicle, the driving information comprising at least one of a driving time, a road type or traffic information; updating the vigilance change based on the driving information to obtain an updated vigilance change.

5. The method according to any one of claims 1-3, characterized in that, The number of users is at least two users, and the method further comprises: selecting a first user from the at least two users based on a driving suitability index of the at least two users; generating first prompt information for the first user, the first prompt information being configured to prompt the first user to drive the vehicle.

6. The method according to any one of claims 1-3, characterized in that, The method further comprises: obtaining input information of a second user; obtaining a trip plan of the second user within a second preset time period in the future based on the input information.

7. The method according to any one of claims 1-3, characterized in that, ​ ​ ​ According to the travel plan and a preset condition, second prompt information for the second user is generated, the second prompt information being used to remind a current motion state of the second user.

8. The method of any one of claims 1-3, wherein, The method further comprises: According to the vigilance value, a driving suitability index is obtained, the driving suitability index being in positive correlation with the vigilance value, the driving suitability index being used to remind a safety degree of the user driving the vehicle; At least one of the vigilance value or the driving suitability index is played through a second device.

9. The method of any one of claims 1-3, wherein, The method further comprises: If the first device detects that the behavior generated by the user in a third preset time period includes a preset behavior, the vigilance change condition is updated according to information of the preset behavior generated by the user, to obtain an updated vigilance change condition.

10. A wearable device, comprising: Comprise: An acquisition module is configured to acquire first information, the first information including information of a behavior generated by a user, the first information including information collected by a first device, the first device including a wearable device of the user, and the first information including information of an intake of the user; A processing module is configured to predict a vigilance change condition of the user through the first information, the vigilance change condition including a vigilance value of the user in a first preset time period; The processing module is further configured to adjust at least one of the following of a vehicle according to the vigilance value: Adjust an alarm range of an eye closure detection system configured in the vehicle, the eye closure detection system being used to detect whether the user closes eyes when driving the vehicle, and to alarm when a detection result is in the alarm range; Or, Adjust alarm information of a vision driver monitoring system (DMS) configured in the vehicle, the alarm information including a fatigue alarm range, a reminding period, or a reminding intensity; Or, Adjust navigation information of the vehicle, the navigation information including at least one of a navigation path, a driving time length according to the navigation path, or a road type in the navigation path; Or, Obtain vigilance information of at least one traffic participant adjacent to the vehicle, and obtain driving dynamics of the at least one traffic participant according to the vigilance information of the at least one traffic participant; or Adjust a detection threshold of an alarm system configured in the vehicle, the detection threshold being related to a sensitivity of the alarm system to alarm.

11. The apparatus of claim 10, wherein, The first information further includes sleep information or motion information of the user.

12. The device according to claim 11, wherein The sleep information includes at least one of a sleep quality score or lunch break information; The motion information includes at least one of a motion start time, a motion end time, a motion type, or a motion amount; The information of the intake includes at least one of an intake time, an intake amount, or an intake type.

13. The device according to any one of claims 10-12, wherein The processing module is specifically configured to take the first information as an input of a preset alertness model, and output the alertness change of the user, wherein an output of the alertness model includes a fusion result of a circadian rhythm and an arousal rhythm, the circadian rhythm is obtained from information related to sleep in the first information, and the arousal rhythm includes a change of an arousal degree of the user after sleep.

14. The device of any one of claims 10-12, wherein, The acquisition module is further configured to acquire driving information of the user collected in the vehicle, the driving information including at least one of driving duration, road type, or traffic information. The processing module is further configured to update the alertness change according to the driving information to obtain an updated alertness change.

15. The apparatus of any one of claims 10-12, wherein, The number of users is at least two users, and the processing module is further configured to: select a first user from the at least two users according to a driving suitability index of the at least two users; and generate first prompt information for the first user, the first prompt information being used to prompt the first user to drive the vehicle.

16. The device of any one of claims 10-12, wherein, The acquisition module is further configured to acquire input information of a second user. The processing module is further configured to acquire a trip plan of the second user in a second preset time period in the future according to the input information. The processing module is further configured to generate second prompt information for the second user according to the trip plan and a preset condition, the second prompt information being used to remind a current motion state of the second user.

17. The apparatus of any one of claims 10-12, wherein, The device further includes a playing module. The processing module is further configured to acquire a driving suitability index according to the alertness value, the driving suitability index being in a positive correlation with the alertness value, and the driving suitability index being used to remind a safety degree of the user driving the vehicle. The playing module is configured to play at least one of the alertness value or the driving suitability index through a second device.

18. The device of any one of claims 10-12, wherein, The processing module is further configured to update the alertness change according to information of a preset behavior generated by the user if the first device detects that the behavior generated by the user includes the preset behavior in a third preset time period, to obtain an updated alertness change.

19. A vehicle characterized by comprising: includes: an acquisition module configured to acquire first information, the first information including information of a behavior generated by a user, the first information including information collected by a first device, the first device including a first device of the user, and the first information including information of an intake of the user; a processing module configured to predict an alertness change of the user through the first information, the alertness change including an alertness value of the user in a first preset time period; The processing module is further configured to adjust at least one of the following of a vehicle according to the alertness value: adjusting an alarm range of an eye-closed detection system arranged in the vehicle, the eye-closed detection system being configured to detect whether the user closes eyes when driving the vehicle, and to alarm when the detection result is within the alarm range; or adjusting alarm information of a visual driver monitoring system DMS arranged in the vehicle, the alarm information including a fatigue alarm range, a reminding period, or a reminding strength; or adjusting navigation information of the vehicle, the navigation information including at least one of a navigation path, a driving time along the navigation path, or a road type in the navigation path; or obtaining alert information of at least one traffic participant adjacent to the vehicle, and obtaining driving dynamics of the at least one traffic participant according to the alert information of the at least one traffic participant; or adjusting a detection threshold of an alarm system arranged in the vehicle, the detection threshold being related to a sensitivity of the alarm system.

20. The vehicle of claim 19, wherein, The first information further includes sleep information or motion information of the user.

21. The vehicle of claim 20, wherein the sleep information includes at least one of a sleep quality score or nap information; the motion information includes at least one of a motion start time, a motion end time, a motion type, or a motion amount; the information of the intake includes at least one of an intake time, an intake amount, or an intake type.

22. The vehicle of any one of claims 19-21, wherein the processing module is specifically configured to input the first information as an input of a preset alert model, and output the alert change condition of the user, the output of the alert model including a fusion result of a circadian rhythm and an arousal rhythm, the circadian rhythm being obtained through information related to sleep in the first information, and the arousal rhythm including a change condition of an arousal degree of the user after sleep.

23. The vehicle of any one of claims 19-21, wherein the obtaining module is further configured to obtain driving information of the user collected in the vehicle, the driving information including at least one of a driving time, a road type, or traffic information; the processing module is further configured to update the alert change condition according to the driving information, to obtain an updated alert change condition.

24. The vehicle of any one of claims 19-21, wherein, The number of users is at least two users, and the processing module is further configured to: select a first user from the at least two users according to a driving suitability index of the at least two users; generate first prompt information for the first user, the first prompt information being used to prompt the first user to drive the vehicle.

25. The vehicle of any one of claims 19-21, wherein the obtaining module is further configured to obtain input information of a second user; the processing module is further configured to obtain a trip plan of the second user in a second preset time period in the future according to the input information. The processing module is further configured to generate second prompt information for the second user according to the trip plan and a preset condition, the second prompt information being used to remind a current motion state of the second user.

26. The vehicle of any one of claims 19-21, wherein, The vehicle further comprises a playing module; The processing module is further configured to obtain a driving suitability index according to the alertness value, the driving suitability index being positively correlated with the alertness value, and the driving suitability index being used to remind a safety degree of driving the vehicle by the user. The playing module is configured to play at least one of the alertness value or the driving suitability index through a second device.

27. The vehicle of any one of claims 19-21, wherein, The processing module is further configured to update the alertness change condition according to information of a preset behavior generated by the user, if the first device detects that the behavior generated by the user in a third preset time period comprises the preset behavior, to obtain an updated alertness change condition.

28. A wearable device, comprising: A computer program product including one or more computer-readable non-transitory storage media having instructions stored thereon that, when executed by one or more processors, implement the steps of any of the methods of claims 1-9.

29. A vehicle characterized by A computer program product including one or more computer-readable non-transitory storage media having instructions stored thereon that, when executed by one or more processors, implement the steps of any of the methods of claims 1-9.

30. A computer-readable storage medium, characterized in that, A computer program product including one or more computer-readable non-transitory storage media having instructions stored thereon that, when executed by one or more processors, implement the steps of any of the methods of claims 1-9.

31. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by a processor, implement the steps of any of the methods of claims 1-9.

32. A chip, comprising: The chip includes a processing unit and a communication interface, the processing unit obtains program instructions through the communication interface, the program instructions are executed by the processing unit, and the processing unit is configured to execute the steps of any of the methods of claims 1-9.

Images