Vehicle perception and attitude recognition system and method in vehicles with smart helmets
By integrating an inertial measurement unit and vehicle sensors into a smart helmet, the rider's posture is identified and a vehicle status interface is output, solving the problem of slow voice interaction during high-speed riding and achieving an efficient and low-load interaction method.
Patent Information
- Application Number
- CN202011524131.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-12-23
- Filing Date
- 2020-12-22
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2040-12-22
AI Technical Summary
Existing smart helmets suffer from slow voice interaction and high cognitive load during high-speed riding, making it difficult to provide an efficient interaction method to supplement voice interaction.
By integrating an inertial measurement unit (IMU) and sensors into the helmet, combined with sensor data from the vehicle, the rider's posture is identified, and a vehicle-related status interface is output on the display. Interaction is achieved using head posture, reducing reliance on walkie-talkies.
It enables efficient interaction without manual operation during high-speed riding, reducing cognitive load and improving rider safety and convenience.
Smart Images

Figure CN113080560B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to smart helmets, such as helmets for motorcycles or other vehicles including off-road bicycles, tricycles or four-wheeled vehicles such as all-terrain vehicles. Background Technology
[0002] Two-wheeled riders can utilize smart helmets to access information about head-mounted see-through displays. Helmets can provide an unobstructed view of the world while overlaying alarms, notifications, and status information on the user or rider. Interaction with such information can be crucial for an immersive user experience. Current helmets allow for voice interaction to navigate and select displayed elements. However, voice interaction can be relatively slow and limited when riders are moving at high speeds. In most cases, voice interaction may be multiplexed with a rider's walkie-talkie, which imposes functional limitations. Furthermore, voice interaction can impose a higher cognitive load on riders compared to other forms of interaction. Utilizing additional forms of interaction to supplement voice interaction may be beneficial. Summary of the Invention
[0003] According to one embodiment, the helmet includes: a transceiver configured to receive vehicle data from one or more sensors located on the vehicle; an inertial measurement unit (IMU) configured to collect motion data from the helmet worn by a vehicle rider; and a processor communicating with the transceiver and the IMU and programmed to receive vehicle data from the one or more sensors located on the vehicle via the transceiver, determine an attitude in response to the vehicle data from the one or more sensors located on the vehicle and the helmet motion data from the IMU, and output a vehicle-related status interface on the helmet's display in response to the attitude.
[0004] According to a second embodiment, a system including a helmet and a vehicle having at least two wheels includes: a helmet transceiver configured to receive vehicle data from one or more sensors located on the vehicle; a helmet inertial measurement unit (IMU) configured to collect helmet motion data associated with the helmet; a rider-facing camera located on the vehicle and configured to monitor the vehicle rider and collect rider image data; and a processor in the helmet communicating with the helmet transceiver and the helmet IMU. The processor is further programmed to receive attitude data from the helmet transceiver via the transceiver and vehicle motion data from the vehicle IMU, determine an attitude in response to the attitude data and compensate for it using the vehicle motion data, and output an action on a display in response to the attitude.
[0005] According to a third embodiment, the helmet includes: a transceiver configured to receive vehicle data from one or more sensors located on the vehicle; an inertial measurement unit (IMU) configured to collect helmet motion data associated with the helmet and the vehicle rider; and a processor communicating with the transceiver and the IMU and programmed to receive vehicle data from one or more sensors located on the vehicle via the transceiver, determine an attitude in response to the vehicle data from the one or more sensors located on the vehicle and the helmet motion data from the IMU, and execute commands in response to the attitude. Attached Figure Description
[0006] Figure 1 This is an example of a system design 100 that includes a smart helmet 101 and a motorcycle 103; Figure 2 These are system design examples for smart helmets and motorcycles that allow for gesture recognition; Figure 3 This is an example flowchart of a smart helmet that allows for gesture recognition. Detailed Implementation
[0007] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The figures are not necessarily to scale; some features may be enlarged or reduced to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but rather as a representative basis for teaching those skilled in the art to employ the embodiments in different ways. As will be understood by those skilled in the art, various features illustrated and described with reference to any of the figures may be combined with features illustrated in one or more other figures to produce embodiments that are not explicitly illustrated or described. The combinations of illustrated features provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of this disclosure are desired for particular applications or implementations.
[0008] This disclosure relates to helmets and saddle-riding vehicles. It should be understood that "saddle-riding vehicle" typically refers to a motorcycle, but can include any type of motor vehicle in which the driver is typically seated and typically wears a helmet due to the lack of a cab for rider protection. In addition to motorcycles, this can also include motorized two-wheeled (PTW) vehicles such as off-road bicycles and scooters. It can also include motorized tricycles or motorized four-wheeled vehicles such as all-terrain vehicles (ATVs). Unless otherwise stated, any specific reference to motorcycles or bicycles may also apply to any other saddle-riding vehicle.
[0009] A system can be used to identify head postures on a moving two-wheeled vehicle (or more) for interaction with a smart helmet display. The detected postures can be used to control the display layout on the smart helmet. Common head postures, such as nodding, pitching, looking up, yawing, and head tilting, can be used to interact with the smart helmet. Therefore, riders may not need to use their hands, which might be gripping the handlebars, to operate the system. Motion measurements from the motorcycle can be used to compensate for motion measurements from the rider's head, allowing the use of existing posture recognition modules.
[0010] The following disclosure describes a system that enables riders to interact with a smart helmet based on head posture. The system uses sensors on the smart helmet in conjunction with sensors on the motorcycle to robustly detect posture. The detected posture is used to control the display layout on the smart helmet. Common head postures, such as nodding, pitching, looking up, yawing, and head tilting, are used to interact with the smart helmet. These postures can be performed while the rider keeps their attention on the road. Because these interactions are natural, they can be performed while using a walkie-talkie.
[0011] Pose recognition can be performed using measurements from a vision camera or an inertial measurement unit (IMU) attached to a helmet. The IMU produces time-series measurements of linear acceleration and rotational rate. End-to-end methods can learn to recognize poses based on raw measurements from the IMU. However, to robustly recognize poses under dynamic conditions caused by vehicle motion, the system may need to sample the pose as the rider experiences vehicle motion. For robust head pose recognition, the system may need to collect the pose as the vehicle undergoes motion, such as wheel bounce, vehicle tilt, etc.
[0012] As disclosed below, in addition to IMU measurements from the helmet, the system can also utilize the IMU on the motorcycle to perform robust pose recognition. Visual camera measurements can be used to support the system. Motorcycle motion estimated based on IMU observations is used to stabilize IMU motion from the helmet. This allows pose recognition trained on the rider when stationary to be used in moving conditions. While one embodiment may not utilize visual measurements from a camera, the same approach can be extended to use additional sensors, such as cameras.
[0013] Figure 1This is an example of a system design 100 including a smart helmet 101 and a motorcycle 103. The smart helmet 101 and motorcycle 103 may include various components and sensors that interact with each other. The smart helmet 101 may focus on collecting data related to the rider's body and head movements. In one example, the smart helmet 101 may include a camera 102. The camera 102 of the helmet 101 may include a main sensor for positioning and orientation identification in moving vehicles. Therefore, the camera 102 may be oriented outwards from the helmet 101 to track other vehicles and objects around the rider. The camera 102 may have difficulty capturing the dynamics of such objects and vehicles. In another example, in addition to or as an alternative to the camera 102, the helmet 101 may also include a radar or lidar sensor.
[0014] Helmet 101 may also include a helmet inertial measurement unit (IMU) 104. The helmet IMU 104 can be used to track the high-dynamic movements of the rider's head. Therefore, the helmet IMU 104 can be used to track the direction the rider is facing or the direction the rider is observing. Furthermore, the helmet IMU 104 can be used to track sudden movements and other possible movements. The IMU may include one or more motion sensors.
[0015] An inertial measurement unit (IMU) uses a combination of accelerometers and gyroscopes (sometimes called magnetometers) to measure and report specific forces and angular velocities on a vehicle body, and sometimes even the Earth's magnetic field. IMUs are typically used to maneuver aircraft (including unmanned aerial vehicles (UAVs) among many other types of aircraft), as well as spacecraft, including satellites and landers. IMUs can be used as components of inertial navigation systems in various vehicle systems. Data collected from IMU sensors allows computers to track motor positioning.
[0016] An IMU operates by detecting current acceleration using one or more axes, and by detecting changes in rotational properties such as pitch, roll, and yaw using one or more axes. A typical IMU also includes a magnetometer, which can help calibrate orientation drift by using measurements of the Earth's magnetic field. Inertial navigation systems contain IMUs with angular and linear accelerometers (for changes in positioning); some IMUs include gyroscope elements (for maintaining an absolute angular reference). An angular accelerometer measures how the vehicle rotates in space. At least one sensor can be present for each of the following three axes: pitch (front up and down), yaw (front left and right), and roll (clockwise or counterclockwise from the cockpit). A linear accelerometer measures the vehicle's non-gravitational acceleration. Because it can move along three axes (up and down, left and right, forward and backward), a linear accelerometer can be present for each axis. The three gyroscopes are typically placed in a similar orthogonal pattern, measuring rotational positioning with reference to an arbitrarily chosen coordinate system. A computer can continuously calculate the vehicle's current positioning. For each of the six degrees of freedom (x, y, z and Ox, Oy, and Oz), it can integrate the sensed acceleration and an estimate of gravity over time to calculate the current velocity. It can also integrate the velocity to calculate the current location. Some of the measurements provided by the IMU are as follows: These are the original measurements from the IMU within the vehicle's chassis frame. a w ω B It is the expected correct acceleration and gyroscope rate measurement. b a 、b g It refers to the bias offset in accelerometers and gyroscopes. η a η g It is noise from the accelerometer and gyroscope.
[0017] Helmet 101 may also include an eye tracker 106. The eye tracker 106 can be used to determine the direction the rider of motorcycle 103 is looking. The eye tracker 106 can also be used to identify drowsiness and fatigue in the PTW rider. The eye tracker 106 can identify different parts of the eye (e.g., the retina, cornea, etc.) to determine where the user is glancing. The eye tracker 106 may include a camera or other sensors to help track the rider's eye movements.
[0018] Helmet 101 may also include a helmet processor 108. The helmet processor 108 may be used for sensor fusion of data collected by various cameras and sensors of the motorcycle 103 and helmet 101. In other embodiments, the helmet may include one or more transceivers for short-range and long-range communication. Short-range communication of the helmet may include communication with the motorcycle 103 or other vehicles and objects in the vicinity. In another embodiment, long-range communication may include communication with non-vehicle servers, the Internet, the "cloud," cellular communication, etc. Helmet 101 and motorcycle 103 may communicate with each other using wireless protocols implemented by transceivers located on both helmet 101 and motorcycle 103. Such protocols may include Bluetooth, Wi-Fi, etc. Helmet 101 may also include a head-up display (HUD) for outputting graphic images on the visor of helmet 101.
[0019] Motorcycle 103 may include a forward-facing camera 105. The forward-facing camera 105 may be located in the headlight or other similar area of motorcycle 103. The forward-facing camera 105 can be used to help identify where the PTW is moving. Furthermore, the forward-facing camera 105 can identify various objects or vehicles in front of motorcycle 103. Therefore, the forward-facing camera 105 can contribute to various safety systems, such as intelligent cruise control or collision detection systems.
[0020] Motorcycle 103 may include bicycle IMU 107. Bicycle IMU 107 may be attached to the headlight or other similar area of the PTW. Bicycle IMU 107 may collect inertial data, which can be used to understand the movement of the bicycle. Bicycle IMU 107 typically has multi-axis accelerometers on three orthogonal axes. Similarly, bicycle IMU 107 may also include multiple gyroscopes.
[0021] Motorcycle 103 may include a rider camera 109. The rider camera 109 can be used to maintain tracking of the rider of motorcycle 103. The rider camera 109 can be mounted at different locations along the handlebars of the motorcycle, or at other locations facing the rider. The rider camera 109 can be used to capture images or videos of the rider, which are then used for various calculations, such as identifying various body parts or movements of the rider. The rider camera 109 can also be used to focus on the rider's eyes. In this way, eye gaze movement can be determined to determine where the rider is looking.
[0022] Motorcycle 103 may include an electronic control unit 111. The ECU 111 can be used to process data collected by sensors on the motorcycle and by sensors on the helmet. The ECU 111 can utilize data received from various IMUs and cameras to process and calculate various localizations or perform object recognition. The ECU 111 can communicate with rider camera 109 and forward-facing camera 105. For example, data from the IMUs can be fed into the ECU 111 to identify localization and orientation relative to a reference point. When image data is combined with such calculations, the direction the rider is facing or focusing on can be identified using the movement of the bicycle. Image data from both the forward-facing camera on the bicycle and the camera on the helmet are compared to determine the relative orientation between the bicycle and the rider's head. Image comparison can be performed based on sparse features extracted from the two cameras (e.g., rider camera 109 and forward-facing camera 105). Motorcycle 103 may include a bicycle central processing unit 113 supporting the ECU. Therefore, the system can continuously monitor the rider's attention, posture, positioning, orientation, contact (e.g., gripping the handlebars), rider slippage (e.g., contact between the rider and the seat), the rider's relationship with the vehicle, and the rider's relationship with the world.
[0023] Figure 2 The diagram illustrates a block diagram of a system for gesture interaction on a smart helmet. To improve the robustness of the gesture recognition system and compensate for vehicle motion, the system can use measurements from the bicycle IMU 203. However, measurements from the motorcycle IMU 203 attached to the motorcycle are captured at different time instances on devices with different clock domains. (For example, the movement of a bicycle while riding at a certain point in time) and measurements from an IMU attached to the helmet. (For example, the riding movement of the helmet at a certain point in time). Therefore, in the synchronization block timestamp, the data received from the helmet and motorcycle are compared to perform clock domain synchronization. Using the result of the synchronization, IMU measurements from the motorcycle (e.g., the bicycle IMU) can be time-shifted to the helmet clock domain. The output of the synchronization block can be motion data (IMU measurements) from the time-series synchronization of the helmet and motorcycle: .
[0024] Synchronization block 205 may include software for synchronizing all data collected to help identify posture, including image data and other sensor data. Synchronization block 205 can collect motion data not only from the helmet IMU 201 and bicycle IMU 203, but also image data collected from cameras located on the motorcycle or helmet. Therefore, camera data (e.g., image data) can also be used to help identify whether a posture is being performed. For example, the camera may monitor the rider's stable positioning during normal riding periods; however, it may include a detection mechanism to determine when the rider's helmet moves up or down in a particular manner.
[0025] Acceleration and rotational rates measured from an IMU on the helmet (e.g., helmet IMU 201) may need to be compensated for by removing the effects of vehicle motion. For example, the system may need to understand the riding environment to understand when the user provides a posture and to distinguish that posture from head movements that occur in response to road bumps or other natural riding events. This can be achieved by using time-synchronized measurements from the motorcycle (e.g., bicycle IMU, bicycle camera, etc. 203). A time-synchronization block can compare data from the helmet and motorcycle at specific times to see if there are any correlations caused by the vehicle environment relative to posture. Relative rigid body transformations between the vehicle and helmet IMUs. (THB(t)) This is used to transform measurements from the motorcycle IMU to the helmet IMU's coordinate system. The transformed values can be subtracted to estimate the compensated helmet measurements. (zc(t)) Rigid body transformations are time-dependent and need to be continuously estimated. Nonlinear filters can be used to minimize the error between the helmet's predicted pose and the pose observed via an integrated helmet IMU. The helmet's predicted pose is derived from a human skeletal model. The result of the nonlinear filter is a relative transformation. THB(t) : .
[0026] The pose recognizer 209 may include software for pose recognition. Pose recognition can be achieved by adjusting the window... w Current compensated helmet IMU measurement sequence The comparison is performed against a database of helmet-mounted IMU motion sequences for each posture. This window can be a time interval comparing data points from different times across a set time period. Various postures, such as nodding, yaw, looking up, tilting, pitching, and stationary postures, can be pre-trained in a static environment. Therefore, the system can include options to activate a learning mode that requires the user to mimic such postures (nodding, yaw, looking up, tilting, pitching, and stationary postures) to record IMU data as a baseline while the motorcycle is in a static environment. Each posture can be encoded using a different internal representation. Thus, each posture can have associated IMU measurements within a time interval to be associated with that posture. The posture recognizer can provide a likelihood coefficient for each class of postures during runtime. Bayesian learning, deep learning, or other learning methods can be used to create the learned posture recognizer model. Inference can be performed on this model to detect postures and identify them with associated likelihood coefficients. The class with the highest likelihood coefficient is selected as the output. This output can command the action associated with each posture. For example, tilting left or right can be associated with activating a steering signal. The helmet can wirelessly transmit this data to the motorcycle to request commands from the motorcycle. Filters based on the detector's output history are used to reduce noise and perform anomaly detection.
[0027] Event Manager 211 can synchronize the output from the rider posture recognition block with information from the bicycle ECU 213. The bicycle ECU 213 can be used to aggregate information from different parts of the motorcycle, as well as collect data and other information from other sensors on the bicycle. Event Manager 211 can contain trigger messages for notifications, alarms, and vehicle information displayed to the rider or provided via audio output. Event Manager 211 can use a finite state machine, whose transitions are determined by the trigger messages from the ECU and the recognized posture. Therefore, different postures can be timed based on messages retrieved from the bicycle, helmet, or non-vehicle server by Event Manager. Thus, a nodding posture from the helmet can include different output commands when a low fuel warning occurs, compared to when the motorcycle is using cruise control. Similarly, posture manipulation can be used to control a walkie-talkie if the rider is using it; in other situations, posture manipulation may be used to play audio / music.
[0028] Event manager 211 can drive or operate display controller 215. Display controller 215 can contain information about the layout of various information. For example, display controller 215 can determine what status information is displayed on the helmet display, and how said status information must be provided. Furthermore, display controller 215 can also store graphical user interface (GUI) elements and animations displayed on the smart helmet display. Control and navigation GUI elements can be changed based on input from a posture recognizer. For example, nodding left or right using the smart helmet can change the navigation display shown on the smart helmet, or, for example, change the mode from audio mode to navigation mode. Display controller 215 can also contain layout profiles for different riding scenarios, which can be changed using posture. For example, the layout can include interfaces for audio screens, hands-free communication, vehicle information (e.g., speed, engine speed, tire pressure, temperature, fuel economy, fuel mileage, fuel level, etc.), navigation information, etc.
[0029] The smart helmet display 217 can be used to output objects on a HUD. The helmet may include, but is not limited to, a forward-facing camera, a rear-facing camera, an accelerometer, volatile memory, non-volatile memory, a power supply, a data connector, a HUD, a Global Positioning System (GPS), a processor, a network transceiver, and a Bluetooth transceiver. In this embodiment, electronic components cooperate to perform various tasks, including but not limited to displaying vehicle instrument information, automatically executing emergency procedures (e.g., calling EMS), presenting pre-collision warnings, and recording sensor information (e.g., IMU data). The smart display 217 may include a HUD, which can be a transparent display that can present data without requiring the rider to look away from a selected viewpoint. The smart display 217 may be displayed within a visor (e.g., an embedded display) or within a display close to the helmet. Display technology 217 may include liquid crystal displays (LCDs), liquid crystal on silicon (LCoS), digital micromirrors (DMDs), organic light-emitting diodes (OLEDs), optical waveguides, and scanning lasers. The display 217 can present relevant information in real-time or near real-time.
[0030] Figure 3This is an example flowchart of a smart helmet that allows for posture recognition. At step 301, the system can collect both helmet data and bicycle data. Such data may include IMU and image data from the helmet as well as from the motorcycle or electric two-wheeler. The system can aggregate this data to later use it to determine whether the user has commanded a posture, or to determine how to compensate for motorcycle movement to identify the posture. Therefore, when road conditions or motorcycle movement cause the helmet to bounce or otherwise move suddenly, the system will want to avoid identifying the posture during unexpected situations. As explained, motorcycle movement data and / or image data collected by the motorcycle IMU can be used to compensate for bicycle movement.
[0031] At step 303, the system can analyze helmet and motorcycle data collected from various sensors, cameras, and other components. The system can analyze the data offline at a remote server (e.g., the cloud), at the helmet via the helmet's processor, or at the motorcycle via the motorcycle's processor. Thus, the system can collect motion data from both the helmet and the motorcycle to determine if the user intends to create a gesture to activate a command or action. This analysis may include compensation for vehicle movement, as described above regarding... Figure 2 The subject of discussion.
[0032] At step 305, the system can determine whether a pose has been identified. Therefore, the movement can match or nearly match a baseline pose measurement, or exceed a movement threshold to be identified as a pose. The system can include baseline measurements for each pose based on the helmet-mounted IMU. The system can also utilize camera data from the motorcycle or helmet to help identify the pose using images that can be compared to it. Furthermore, it can compensate for any movement from the motorcycle, as shown by the IMU.
[0033] At step 307, the system can identify the posture and the associated commands. The system can have a lookup table that stores each associated command as associated with each posture. For example, tilting the head to the right can activate a right turn signal, and tilting the head to the left can activate a left turn signal. The lookup table can include information about which command must be sent to which controller. Furthermore, the lookup table can also map different commands to different postures based on the smart display's display interface. For example, nodding during audio mode can change the audio track, but nodding during hands-free calling can change the call volume.
[0034] At step 309, the system can then output a posture-related command. This command includes instructing the processor to execute posture-related commands, followed by displaying any relevant information on the smart helmet's display. The command can be associated with features on the vehicle, helmet, or motorcycle itself. Furthermore, the command can request the exchange of wireless data or other information between the helmet, bicycle, non-vehicle server, other bicycles, etc. Information can also be displayed on the motorcycle's instrument cluster to indicate the execution or update status of the command.
[0035] The processes, methods, or algorithms disclosed herein may be transferable to / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, the processes, methods, or algorithms may be stored in various forms as data and instructions executable by a controller or computer, including but not limited to information permanently stored on non-writable storage media such as ROM devices and information reproducibly stored on writable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. The processes, methods, or algorithms may also be implemented as software executable objects. Alternatively, the processes, methods, or algorithms may be embodied wholly or partially using suitable hardware components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices, or a combination of hardware, software, and firmware components.
[0036] While exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms covered by the claims. The language used in this specification is descriptive and not restrictive, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As described above, features of various embodiments may be combined to form further embodiments of the invention that may not be explicitly described or illustrated. Although various embodiments may have been described as providing advantages in one or more desired characteristics, or being preferred over other embodiments or prior art implementations, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system properties, depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, merchantability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. Accordingly, any embodiment is described to a certain extent as having one or more characteristics less desirable than other embodiments or prior art implementations that are outside the scope of this disclosure and may be desirable for a particular application.
Claims
1. A helmet, comprising: The transceiver in the helmet is configured to receive vehicle data from one or more sensors located on the vehicle, wherein the vehicle data includes image data associated with the vehicle's rider-facing camera; The inertial measurement unit (IMU) in the helmet is configured to collect motion data of the helmet in relation to the rider of the vehicle; and The processor in the helmet communicates with the transceiver and IMU, and is programmed to: Receive vehicle data from one or more sensors located on the vehicle via a transceiver; In response to vehicle data from one or more sensors located on the vehicle and helmet motion data from the IMU, an attitude is determined, wherein the processor is further programmed to utilize the vehicle data to compensate for vehicle motion when determining the attitude; and In response to the posture, a status interface related to the vehicle is output on the helmet's display. The processor is further programmed to determine the posture by comparing the current compensated helmet IMU measurement sequence z¬¬c(tH-w: tH+w) on window w with a database of helmet IMU motion sequences for each posture.
2. The helmet of claim 1, wherein each of the vehicle data and helmet motion data includes a timestamp identifying when the vehicle data or helmet motion data was collected, wherein the timestamp is used to perform clock domain synchronization.
3. The helmet of claim 1, wherein the processor is further programmed to compensate for helmet motion data using at least vehicle data.
4. The helmet of claim 1, wherein the processor is further programmed to use timestamps from vehicle data and helmet motion data for compensation.
5. The helmet of claim 1, wherein the processor is further configured to determine the posture in response to nodding, yaw, looking up, tilting, or pitching.
6. The helmet of claim 1, wherein the helmet includes a head-up display configured to output graphic images on the visor of the helmet.
7. The helmet of claim 1, wherein the helmet includes a camera.
8. The helmet of claim 1, wherein, The processor is further programmed to remove the effects of vehicle motion by utilizing one or more time-synchronized measurements from the vehicle, wherein the time-synchronized measurements determine the correlation caused by the vehicle environment by comparing data from the helmet and the vehicle at a specific time.
9. A system comprising a helmet and a vehicle having at least two wheels, comprising: A helmet transceiver is configured to receive vehicle data from one or more sensors located on the vehicle. The helmet inertial measurement unit (IMU) is configured to collect helmet motion data associated with the helmet; A rider-facing camera is located on the vehicle and configured to monitor the rider of the vehicle and collect rider image data. and The processor in the helmet communicates with the helmet transceiver and the helmet IMU, and is programmed to: The transceiver receives attitude data from the helmet transceiver and vehicle motion data from the vehicle's IMU. The attitude is determined in response to the attitude data, and the helmet motion data is compensated using vehicle data, wherein the processor is further programmed to compensate for vehicle motion using vehicle data, wherein the attitude is determined by comparing the current compensated helmet IMU measurement sequence z¬¬c(tH-w: tH+w) on window w with a database of helmet IMU motion sequences for each attitude; and In response to the posture, the action is output on the helmet display.
10. The system of claim 9, wherein the processor in the helmet is further programmed to activate a learning mode to identify one or more postures in a static vehicle environment when the vehicle is not in operation and in response to associated IMU measurements from the helmet IMU over a period of time.
11. The system of claim 9, wherein the display of the helmet includes a head-up display configured to output graphic images on the visor of the helmet.
12. The system of claim 9, wherein the vehicle is a motorcycle or an electric two-wheeled unit.
13. The system of claim 9, wherein the processor is further configured to determine the posture using at least the rider image data.
14. The system of claim 9, wherein the vehicle data includes vehicle motion data collected from a vehicle inertial measurement unit.
15. A helmet, comprising: The transceiver is configured to receive vehicle data from one or more sensors located on the vehicle, including image data near the rider of the vehicle, wherein the image data comes from the vehicle's rider-facing camera. An inertial measurement unit (IMU) is configured to collect helmet motion data associated with the rider of the vehicle. and The processor communicates with the transceiver and IMU, and is programmed to: Receive vehicle data from one or more sensors located on the vehicle via a transceiver; and In response to vehicle data from one or more sensors located on the vehicle and helmet motion data from the IMU, an attitude is determined, wherein the processor is further programmed to use the vehicle data to compensate for vehicle motion; and Execute commands in response to the stated posture. The processor is further programmed to determine the posture by comparing the current compensated helmet IMU measurement sequence z¬¬c(tH-w: tH+w) on window w with a database of helmet IMU motion sequences for each posture.
16. The helmet of claim 15, wherein the processor is further programmed to output the state of the vehicle on the helmet's display in response to the posture.
17. The helmet of claim 16, wherein the processor is further programmed to execute the command to activate a function at the vehicle based on the posture.
18. The helmet of claim 15, wherein the processor is further programmed to activate a learning mode to identify one or more postures in a static vehicle environment when the vehicle is not in operation.
Citation Information
Patent Citations
Helmet attitude measuring method, device and system
CN108917746A
Image display system
US20170032195A1
