Method and system for calculating head movement in vehicles

Abstract

The present invention provides a method and apparatus for calculating head movement in a vehicle, which does not require attaching sensors to the occupant's head and can calculate head movement by considering the movement of the vehicle itself and the relative movement of the head to the vehicle. [Solution] A calculation method for calculating the movement of the head 2a of a subject 2 in a vehicle 10 includes: acquiring the movement of the vehicle 10 in a predetermined vehicle coordinate system; calculating the head position movement acting on the position of the head 2a in the vehicle coordinate system based on the acquired movement of the vehicle 10; acquiring image data of the head 2a in the vehicle 10; calculating the head relative movement, which is the relative movement of the head 2a with respect to the vehicle 10, based on the acquired image data; and calculating the movement of the head 2a in the vehicle coordinate system based on the head position movement and the head relative movement.

Description

[Technical Field]

[0001] This invention relates to a method and system for calculating head acceleration in a vehicle.

[0002] Patent Document 1 discloses the use of information on the movement of the occupant's head caused by the movement of the vehicle to estimate the occupant's motion sickness state. The occupant's head movement information is measured by an acceleration sensor or gyro sensor attached to the head, or by capturing an image of the occupant's face and measuring it from the position and orientation of the face in the image. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] WO2020 / 170640 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] In the actual usage environment of a vehicle, attaching sensors to the occupant's head to measure their head movements is cumbersome.

[0005] On the other hand, head movements measured based on captured images only represent the relative movement of the head to the vehicle, and do not take into account the movement of the vehicle itself. In other words, there is room for improvement in calculating the head movements of the occupants by considering both the movement of the vehicle itself and the relative movement of the head to the vehicle.

[0006] The present invention aims to provide a method and apparatus for calculating head movement in a vehicle that does not require attaching sensors to the occupant's head and can calculate head movement by considering the movement of the vehicle itself and the relative movement of the head to the vehicle. [Means for solving the problem]

[0007] One aspect of this disclosure is, A calculation method for calculating the head movement of a subject in a vehicle, To acquire the movement of the vehicle in a predetermined vehicle coordinate system, Based on the acquired movement of the vehicle, calculate the head position movement acting on the position of the head in the vehicle's coordinate system. To acquire image data of the head in the aforementioned vehicle, Based on the acquired image data, the relative movement of the head relative to the vehicle is calculated. Based on the head position movement and the head relative movement, the movement of the head in the vehicle coordinate system is calculated. This provides a calculation method that includes [the following].

[0008] Other aspects of this disclosure are: A calculation system for calculating the head movements of a subject in a vehicle, A vehicle motion acquisition unit acquires the movement of the vehicle in a predetermined vehicle coordinate system, The vehicle includes an image data acquisition unit that acquires image data of the subject's head, Control unit and It has, The control unit, A head position movement calculation process calculates the head position movement acting on the head position in the vehicle coordinate system based on the acquired movement of the vehicle, Based on the acquired image data, a head relative motion calculation process is performed to calculate the head relative motion, which is the movement of the head relative to the vehicle. A head movement calculation process calculates the movement of the subject's head in the vehicle coordinate system based on the head position movement and the relative head movement. We provide a calculation system that performs this task. [Effects of the Invention]

[0009] According to this disclosure, it is possible to more accurately calculate the head movements of a subject, including head movements relative to the vehicle, without requiring the attachment of sensors to the subject's head. For example, it is easier to estimate a subject's motion sickness based on their head movements. [Brief explanation of the drawing]

[0010] [Figure 1] A schematic diagram showing a calculation system 1 for calculating the head movements of a subject in a vehicle according to one embodiment of the present disclosure. [Figure 2] A diagram showing the schematic configuration of the control unit. [Figure 3A] A schematic diagram showing the translational acceleration acting on the head position. [Figure 3B] A schematic diagram showing the centrifugal acceleration acting on the head position. [Figure 3C] A schematic diagram showing the tangential acceleration acting on the head position. [Figure 4] A flowchart showing the process for calculating head movements. [Figure 5] A subroutine that shows the process for setting the head coordinate system. [Figure 6] This subroutine shows the process for calculating the orientation of the head coordinate system. [Figure 7] A subroutine that shows the process for calculating head position acceleration. [Figure 8] This subroutine shows the process for calculating head position angular velocity. [Figure 9] A graph showing the measured and calculated results of head movement in the example. [Figure 10] Figure 9 shows a graph illustrating the correlation between the measured results and the correlation results. [Modes for carrying out the invention]

[0011] A head motion calculation system and calculation method according to one embodiment of the present disclosure will be described below with reference to the attached drawings. The following description is essentially illustrative and is not intended to limit the present invention, its applications, or its uses.

[0012] Figure 1 schematically shows a calculation system 1 for calculating the head movements of a subject in a vehicle according to this embodiment. The calculation system 1 calculates the head movements of the subject 2 in conjunction with the movement of the vehicle 10 and estimates the degree of motion sickness of the subject 2 based on the calculated head movements. In this embodiment, the vehicle 10 is an automobile. The vehicle 10 may be any vehicle other than an automobile, for example, a motorcycle, a railway car, a ship, an airplane, etc.

[0013] The calculation system 1 includes an inertial measurement unit (IMU), a camera (22), an electronic control unit (ECU), and an output device (40). The IMU (21), camera (22), control device (30), and output device (40) are mounted on the vehicle (10). In this embodiment, the calculation system 1 is mounted on the vehicle (10) on which the subject (2) is riding, but some parts of it may not be mounted on the vehicle (10).

[0014] The IMU 21 measures the inertia acting on the vehicle 10 as the movement of the vehicle 10. Therefore, the IMU 21 constitutes the vehicle motion acquisition unit according to this disclosure. In this embodiment, the IMU 21 takes the vehicle origin O, with the vicinity of the center of gravity of the vehicle 10 as the vehicle origin, and measures the inertia acting on the vehicle along a vehicle coordinate system having an X-axis along the vehicle longitudinal direction, a Y-axis along the vehicle width direction, and a Z-axis along the vehicle vertical direction. Specifically, the IMU 21 measures the vehicle acceleration a acting on the vehicle 10 along the X-axis, Y-axis, and Z-axis. car And the vehicle angular velocity ω acting on vehicle 10 around the X, Y, and Z axes. car Equations (1) and (2) are used to measure the vehicle acceleration a. car and vehicle angular velocity ω car These are represented as column vectors having scalar components in the X, Y, and Z axes.

[0015]

number

[0016]

number

[0017] IMU21 is electrically connected to control unit30. Measured vehicle acceleration a car and vehicle angular velocity ω car This information is input to the control unit 30, for example, via CAN communication with the control unit 30.

[0018] Camera 22 captures images of the area around the head of subject 2 and acquires image data. Therefore, camera 22 constitutes the image data acquisition unit according to this disclosure. Camera 22 may shoot video or take still images at predetermined time intervals. Camera 22 is electrically connected to control unit 30. The acquired image data is input to control unit 30, for example, via CAN communication.

[0019] The control unit 30 receives the vehicle acceleration a from the IMU 21. car and vehicle angular velocity ω car Based on the image data acquired from camera 22, the movement of subject 2's head 2a is calculated to estimate the degree of motion sickness of subject 2. The control unit 30 outputs the estimated degree of motion sickness of subject 2 to output device 40.

[0020] The output device 40 notifies the occupants of the vehicle 10 based on the degree of motion sickness input from the control unit 30. For example, the output device 40 may notify the person of the degree of motion sickness and / or prompt the driver of the vehicle 10 to drive in a way that reduces motion sickness (such as suppressing sudden changes in acceleration and / or angular velocity, including sudden acceleration, sudden braking, and sudden steering).

[0021] The configuration and operation of the control unit 30 will be described below.

[0022] Figure 2 is a block diagram showing a schematic configuration of the control unit 30. As shown in Figure 2, the control unit 30 is a well-known computer having a storage unit 31 such as a hard disk, an arithmetic processing unit 32, and a memory (not shown). The control unit 30 executes various processes by executing programs stored in the storage unit 31.

[0023] The processes executed by the control unit 30 include a head coordinate system setting process 33, a head orientation calculation process 34, a head position motion calculation process 35, a head relative motion calculation process 36, a head motion calculation process 37, a head perceived motion calculation process 38, and a motion sickness determination process 39. The arithmetic processing unit 32 may be configured by one or a plurality of processing circuits that execute each process. The arithmetic processing unit 32 may execute each process by a single processing circuit, or may execute each process by a plurality of processing circuits.

[0024] In the head coordinate system setting process 33, a head coordinate system is set for the head 2a of the subject 2 based on the image data acquired by the camera 22. In the head coordinate system, a head origin o is set at a predetermined position of the subject's head 2a, and the x-axis, y-axis and z-axis corresponding to respective directions of the head 2a in a predetermined posture are set to be along the X-axis, Y-axis and Z-axis of the vehicle coordinate system. The head origin o is the head position P in the vehicle coordinate system head (P head_X ,P head_Y ,P head_Z ) and may vary with the movement of the head 2a.

[0025] Specifically, in the head coordinate system setting process 33, feature points such as the position of the eyes and nose of the head 2a are extracted based on the acquired image data, and a head coordinate system is set for the head 2a of the subject 2 based on these feature points. In this embodiment, in the head coordinate system setting process 33, the head coordinate system is set for the head 2a with respect to the direction of travel of the vehicle 10, where the head 2a is facing straight in the direction of travel (a predetermined posture), that is, not facing left or right or up or down, as the reference axis. Various algorithms can be used to extract feature points of the head 2a; for example, MediaPipe, provided by Google as a live media processing solution, can be used.

[0026] In the head coordinate system setting process 33, a gain G related to the size of the image data is calculated based on the image data. For example, in the head coordinate system setting process 33, the number of pixels in the image data between the two eyes is compared with the standard distance between the two eyes, and a gain G showing the correlation between the number of pixels and the distance is calculated.

[0027] In the head orientation calculation process 34, the orientation of the head 2a is calculated based on the image data acquired by the camera 22. Specifically, in the head orientation calculation process 34, the orientation of the subject 2's head 2a is calculated based on feature points extracted from the acquired image data.

[0028] In this embodiment, the orientation of the head 2a is calculated as a unit rotation matrix R, which is described as a scalar quantity representing the degree to which each axis of the head coordinate system is oriented with respect to the X, Y, and Z axes of the vehicle coordinate system.

[0029] Equation (3) shows the unit rotation matrix R as unit vectors for each of the three axes, each having scalar components in the X, Y, and Z directions of the vehicle coordinate system. The first column shows the x-axis of the head coordinate system as a column vector having scalar components in the X, Y, and Z directions of the vehicle coordinate system. Similarly, the second and third columns show the y and z axes of the head coordinate system as unit column vectors having scalar components in the X, Y, and Z directions of the vehicle coordinate system, respectively.

[0030]

number

[0031] In the head orientation calculation process 34, the transpose matrix R of the unit rotation matrix R is also calculated. T The transpose matrix R is calculated. T When applied to a vector described in vehicle coordinates, the vector is converted to a description in head coordinates.

[0032] The head position motion calculation process 35 calculates the head position motion, which is the movement of the head 2a in the vehicle coordinate system, based on the movement of the vehicle 10 acquired by the IMU 21. The head position motion includes the head position P, which is the head origin o of the head coordinate system in the vehicle coordinate system. head The head position acceleration a acts on it. phead and head position angular velocity ω phead This includes the following: In other words, the head position motion calculation process 35 calculates the head position acceleration a based on the movement of the vehicle 10. phead and head position angular velocity ω phead Calculate.

[0033] Head position acceleration a phead This includes the acceleration a in the axial direction of the head position. phead_A And, head position centrifugal acceleration a phead_C And, head position tangential acceleration a phead_T It includes.

[0034] Figure 3A shows the head position axial acceleration a phead_AThis is conceptually illustrated. As shown in Figure 3A, the head position axial acceleration a phead_A is the vehicle acceleration a car It is equal to the origin o of the head and the vehicle acceleration a. car This acts directly. Equation (4) is the acceleration a in the axial direction of the head position. phead_A This is represented as a column vector having scalar components in the X, Y, and Z axes.

[0035]

number

[0036] Figure 3B shows the vehicle angular velocity ω acting around the X-axis of the vehicle coordinate system. car_X The centrifugal acceleration a of the head position acting on the head origin o is caused by the following phead_CX This conceptually illustrates the head position and centrifugal acceleration a. phead_CX At the head origin o, the vehicle's angular velocity ω around the X-axis is... car_X This is the acceleration acting centrifugally from the vehicle's origin O towards the head's origin O.

[0037] Equation (5) represents the distance D between the vehicle origin O and the head origin o when viewed from the X-axis direction. _x and the vehicle's angular velocity ω around the X-axis car_X Using this, the vehicle angular velocity ω around the X-axis. car_X Head position centrifugal acceleration a phead_CX This shows that equation (5) is a well-known formula for calculating centrifugal acceleration.

[0038]

number

[0039] Vehicle angular velocity ω around the X-axis car_X Head position centrifugal acceleration a phead_CX This is the Y-axis component a phead_CX_Y and the Z-axis component a phead_CX_Z It is decomposed into and . Similarly, the vehicle angular velocity ω around the Y axis car_YThe head position is at the origin o, and the centrifugal acceleration a phead_CY This is what acts on it, and this is the X-axis component a phead_CY_X and the Z-axis component a phead_CY_Z It is decomposed into and . Similarly, the vehicle angular velocity ω around the Z axis car_Z The head position is at the origin o, and the centrifugal acceleration a phead_CZ This is what acts on it, and this is the X-axis component a phead_CZ_X and the Y-axis component a phead_CZ_Y It is broken down into two parts.

[0040] Head position centrifugal acceleration a acting on vehicle angular velocity ωcar around the X, Y, and Z axes phead_c When we organize the X-axis component, Y-axis component, and Z-axis component separately, we obtain equation (6). Equation (6) is the centrifugal acceleration a of the head position. phead_C This is represented as a column vector having scalar components in the X, Y, and Z axes.

[0041]

number

[0042] Figure 3C shows the vehicle angular velocity ω acting around the X-axis of the vehicle coordinate system. car_X The tangential acceleration a of the head position acting on the head origin o due to this phead_TX This conceptually illustrates the head position tangential acceleration a phead_TX At the head origin o, the vehicle's angular velocity ω around the X-axis is... car_X This is the acceleration acting tangentially, perpendicular to the centrifugal direction from the vehicle's origin O towards the head's origin O.

[0043] Equation (7) represents the distance D between the vehicle origin O and the head origin o when viewed from the X-axis direction. _x and the vehicle's angular velocity ω around the X-axis car_X Using this, the vehicle angular velocity ω around the X-axis. car_X Tangential acceleration a due to head position phead_TX This shows that equation (7) is the vehicle angular velocity ω around the X axis. car_XThe angular acceleration is calculated by differentiating it with respect to time, and the distance D between the vehicle's origin O and the head's origin O is added to this. _X By multiplying by this, the vehicle angular velocity ω around the X-axis is obtained. car_X Tangential acceleration a due to head position phead_TX It is calculating this.

[0044]

number

[0045] Vehicle angular velocity ω around the X-axis car_X Tangential acceleration a due to head position phead_TX This is the Y-axis component a phead_TX_Y and the Z-axis component a phead_TX_Z It is decomposed into and . Similarly, the vehicle angular velocity ω around the Y axis car_Y The head position is tangential to the head origin o, and the acceleration a phead_TY This is what acts on it, and this is the X-axis component a phead_TY_X and the Z-axis component a phead_TY_Z It is decomposed into and . Similarly, the vehicle angular velocity ω around the Z axis car_Z The head position is tangential to the head origin o, and the acceleration a phead_TZ This is what acts on it, and this is the X-axis component a phead_TZ_X and the Y-axis component a phead_TZ_Y It is broken down into two parts.

[0046] Vehicle angular velocity ω around the X, Y, and Z axes car The centrifugal acceleration a of the head position acting as a phead_T When we organize the X-axis component, Y-axis component, and Z-axis component separately, we obtain equation (8). Equation (8) is the tangential acceleration a of the head position. phead_T This is represented as a column vector having scalar components in the X, Y, and Z axes.

[0047]

number

[0048] Head position acceleration a pheadis the head position axial acceleration a phead_A , the head position centrifugal acceleration a phead_C , and the head position tangential acceleration a phead_T , is calculated by summing these, and is described in Equation (9).

[0049] [Mathematical formula]

[0050] The head position angular velocity ω phead is equal to the vehicle angular velocity ω car . That is, the vehicle angular velocity ω acts directly on the head origin o car . Equation (10) expresses the head position angular velocity ω phead as a column vector having scalar components in the X-axis, Y-axis, and Z-axis directions. ω phead_X denotes the angular velocity acting on the head origin o by the vehicle angular velocity ω about the X-axis car_X . ω phead_Y denotes the angular velocity acting on the head origin o by the vehicle angular velocity ω about the Y-axis car_Y . ω phead_Z denotes the angular velocity acting on the head origin o by the vehicle angular velocity ω about the Z-axis car_z .

[0051] [Mathematical formula]

[0052] In the head relative motion calculation process 36, the head relative motion, which is the relative motion of the head 2a with respect to the vehicle 10, is calculated based on the acquired image data. Specifically, in the head relative motion calculation process 36, a head relative acceleration a that is the relative acceleration of the head 2a with respect to the vehicle 10 is calculated as the head relative motion head_to_car , and a head relative angular velocity ω that is the relative angular velocity of the head 2a with respect to the vehicle 10 head_to_car are calculated.

[0053] Head relative acceleration a head_to_caris calculated based on the motion of the head origin o. Specifically, the head relative acceleration a head_to_car is the head position P head (P head_X , P head_Y , P head_Z ) multiplied by a gain G and then time-differentiated. Therefore, the head relative acceleration a head_to_car is expressed as an acceleration in the vehicle coordinate system. Equation (11) expresses the head relative acceleration a head_to_car as a column vector having scalar components in the X-axis, Y-axis, and Z-axis directions.

[0054] [Mathematical formula]

[0055] The head relative angular velocity ω head_to_car is calculated based on the change in orientation of the head coordinate system. The head relative angular velocity ω head_to_car may be calculated using a unit rotation matrix R. The head relative angular velocity ω head_to_car may be calculated using the temporal change of the unit rotation matrix R. Therefore, the head relative angular velocity ω head_to_car is expressed as an angular velocity in the vehicle coordinate system. The head relative angular velocity ω head_to_car is described as a column vector having scalar components in the X-axis, Y-axis, and Z-axis directions.

[0056] In the head motion calculation process 37, the head acceleration A acting on the head origin o is obtained by adding the head relative motion to the head position motion head and the head angular velocity W head are calculated. Specifically, as shown in Equation (12), the head acceleration A head is calculated by adding the head position acceleration a phead and the head relative acceleration a head_to_car . As shown in Equation (13), the head angular velocity W head is calculated by adding the head position angular velocity ω phead and the head relative angular velocity ω head_to_car .

[0057]

number

[0058]

number

[0059] The head perception motion calculation process 38 calculates head acceleration A, which is expressed in the vehicle coordinate system. head and head angular velocity W head Therefore, head perceptual acceleration A described in the head coordinate system head_on_head and head perceived angular velocity W head_on_head The following is calculated: Specifically, as shown in equation (14), the head perceptual acceleration A head_on_head Head acceleration A head The transpose matrix R T It is calculated by converting using the following formula. As shown in equation (15), the head's perceived angular velocity W head_on_head This is the head angular velocity W head The transpose matrix R T It is calculated by converting using [this method].

[0060]

number

[0061]

number

[0062] Motion sickness detection process 39 uses the calculated head perception acceleration A head_on_head and / or head perceived angular velocity W head_on_head The degree of motion sickness in subject 2 is determined using this. For example, in motion sickness determination process 39, the calculated head perception acceleration A head_on_head and / or head perceived angular velocity W head_on_head The amount may be compared to a predetermined threshold, and if it exceeds the threshold, it may be determined that subject 2 is experiencing motion sickness. Furthermore, the degree of motion sickness may be determined from the amount exceeding the threshold.

[0063] Furthermore, if the motion sickness detection process 39 determines that subject 2 is experiencing motion sickness, it causes the output device 40 to notify the system that subject 2 is experiencing motion sickness. Alternatively or in addition, the motion sickness detection process 39 may also encourage the driver of vehicle 10 to drive in a way that reduces motion sickness (such as avoiding sudden acceleration, sudden braking, and sudden steering, or other sudden changes in acceleration and / or angular velocity).

[0064] Next, the processing flow in the calculation system 1 according to this embodiment will be explained with reference to the flowchart shown in Figure 4 and the subroutines shown in Figures 5 to 8.

[0065] As shown in Figure 4, according to the calculation system 1, processing begins when the vehicle 10 is in operation. First, in step S010, the camera 22 acquires image data of the area around the head 2a of the subject 2. In this embodiment, for example, the image data is captured as still images at predetermined intervals. The predetermined interval is set to a time interval that is short enough to acquire the movement of the head 2a from the sequentially acquired image data.

[0066] Next, in step S020, the head coordinate system is set. Referring to Figure 5, the subroutine S020 for setting the head coordinate system will be explained. The head coordinate system is set by the control unit 30 as the head coordinate system setting process 33. First, in step S021, facial feature points are extracted for each acquired image data. Next, in step S022, the gain G is calculated based on the extracted facial feature points. Next, in step S023, the head origin o is set at a predetermined position, in this embodiment, at the center position between the left and right semicircular canals 3, based on the extracted facial feature points.

[0067] Finally, in step S024, the head coordinate axes are set. As described above, the head coordinate axes are set with the head origin o as the origin, and based on the state in which the head 2a is in a predetermined posture, each coordinate axis (i.e., x axis, y axis, and z axis) is set to align with the predetermined X axis, Y axis, and Z axis of the vehicle coordinate system and fixed to the head 2a. In this embodiment, when the head 2a is in the predetermined posture, the subject 2 is seated in the seat with the direction of travel facing forward, so the front-to-back, left-to-right, and up-and-down directions of the head 2a at this time are parallel to the front-to-back, left-to-right, and up-and-down directions of the vehicle 10. Therefore, when the head 2a is in the predetermined posture, the x axis, y axis, and z axis of the head coordinate system are parallel to the X axis, Y axis, and Z axis of the vehicle coordinate system, respectively.

[0068] For each of the multiple image data, a head coordinate setting process is performed. Therefore, depending on the position and orientation of head 2a, the position and orientation of the head origin o in the vehicle coordinate system may change in the head coordinate system.

[0069] Returning to the flowchart in Figure 4, in step S030, the orientation of the head coordinate system is calculated. Referring to Figure 6, the subroutine S030 related to the calculation of the orientation of the head coordinate system will be explained. The calculation of the orientation of the head coordinate system is performed by the control unit 30 as the head orientation calculation process 34.

[0070] As shown in Figure 6, first, in step S031, the change in the head coordinate system set in step S020 is acquired. Specifically, the change in the head coordinate system is acquired for each of the multiple image data. Next, in step S032, the orientation of each coordinate axis (x axis, y axis, and z axis) of the head coordinate system is calculated using unit vectors corresponding to each coordinate axis (X axis, Y axis, and Z axis) of the vehicle coordinate system. This calculates a unit rotation matrix R that describes the three-axis unit vectors indicating the direction of each coordinate axis of the head coordinate system, serving as an index indicating the orientation of the head coordinate system relative to the vehicle coordinate system.

[0071] Finally, in step S033, the transpose matrix R of the unit rotation matrix R T The transpose matrix R is calculated. TThis is used to transform vectors described in the vehicle coordinate system into vectors described in the head coordinate system.

[0072] Returning to the flowchart in Figure 4, in step S040, the movement of the vehicle 10 is acquired. The movement of the vehicle 10 is determined by the IMU 21, which then acquires the vehicle acceleration a acting on the vehicle 10. car and vehicle angular velocity ω car It is obtained as follows: Vehicle acceleration a of vehicle 10 car This includes the vehicle acceleration a in the longitudinal direction (X-axis direction). car_X , vehicle acceleration a in the vehicle width direction (Y-axis direction) car_Y , vehicle acceleration a in the vertical direction (Z-axis direction) car_Z It is obtained as follows: Vehicle angular velocity ω of vehicle 10 car ω is the vehicle's angular velocity around the X-axis. car_X , the angular velocity of the vehicle around the Y axis ω car_Y and the vehicle angular velocity ω around the Z axis car_Z It will be acquired as follows.

[0073] Next, in steps S050 and S060, the head position P is where the head 2a is located in the vehicle coordinate system. head (P head_X ,P head_Y ,P head_Z ) The head movement acting based on the movement of the vehicle 10 is calculated. Head movement includes head acceleration A head and head angular velocity W head This includes the following. The calculation of head movement is performed by the control unit 30, which first executes the head position movement calculation process 35 and the head relative movement calculation process 36 to calculate the head position movement and the head relative movement. Next, based on the calculated head position movement and head relative movement, the head movement calculation process 37 is executed to calculate the head movement.

[0074] Refer to Figure 7, head acceleration A head The subroutine S050 related to the calculation of [the value] will be explained.

[0075] As shown in Figure 7, first in step S051 the movement of the vehicle 10 is acquired. Furthermore, in step S052 the position of the head origin o is calculated based on the acquired image data. Next, in step S053 the vehicle acceleration a car Based on the head position axial acceleration a phead_A The following is calculated. Furthermore, in steps S054 and S055, the vehicle acceleration a car , vehicle angular velocity ω car Based on the distance D between the vehicle origin O and the head origin O, the centrifugal acceleration a of the head position phead_C and head position tangential acceleration a phead_T This is calculated.

[0076] On the other hand, in step S056, based on the change in the position of the head origin o, the head relative acceleration a head_to_car The following is calculated. The calculation of relative head movement is performed by the control unit 30 as a relative head movement calculation process 36.

[0077] Next, in step S057, head acceleration A head The following is calculated: Head acceleration A head The calculation of this is performed by the control unit 30 as a head motion calculation process 37. Head position axial acceleration a phead_A And, head position centrifugal acceleration a phead_C And, head position tangential acceleration a phead_T These are added together to determine the head position acceleration a phead The following is calculated. Finally, the calculated head position acceleration a phead and head relative acceleration a head_to_car These are added together, and the head acceleration A head The following is calculated: Head acceleration A head It is described in vehicle coordinate system.

[0078] Next, referring to Figure 8, the head angular velocity W head The subroutine S060 related to the calculation of [the value] will be explained.

[0079] As shown in Figure 8, first in step S061, the movement of the vehicle 10 is acquired. Furthermore, in step S062, the change in the orientation of the head coordinate system is acquired. Next, in step S063, the acquired vehicle angular velocity ω car Based on the head position angular velocity ω phead The head position angular velocity ω is calculated. phead The calculation is performed by the control unit 30 as a head position movement calculation process 35.

[0080] Next, in step S064, based on the acquired change in the orientation of the head coordinate system, the head relative angular velocity ω head_to_car The following is calculated: Head relative angular velocity ω head_to_car The calculation is performed by the control unit 30 as a head relative motion calculation process 36.

[0081] Finally, in step S065, head angular velocity W head The following is calculated: Head angular velocity W head The calculation is performed as head movement calculation process 37. The calculated head position angular velocity ω phead and head relative angular velocity ω head_to_car These are added together, and the head angular velocity W head The following is calculated: Head angular velocity W head It is described in vehicle coordinate system.

[0082] That is, head acceleration A head and head angular velocity W head This describes the movement of the vehicle 10 by summing up the movement of the vehicle and the resulting relative movement of the head 2a with respect to the vehicle 10, which acts on a predetermined position on the head 2a of the subject 2, i.e., the head origin o of the head coordinate system, which in this embodiment is the center position between the left and right semicircular canals 3.

[0083] Returning to the flowchart in Figure 4, in step S070, the perceived head movement is calculated. The calculation of the perceived head movement is performed by the control unit 30 as the perceived head movement calculation process 38. The perceived head movement includes the perceived head acceleration A head_on_head and head perceived angular velocity W head_on_head It includes [this].

[0084] Head Perceptual Acceleration A head_on_head Head acceleration A head The transpose matrix R T It is calculated by converting using the following: Head perceptual angular velocity W head_on_head This is the head angular velocity W head The transpose matrix R T It is calculated by converting using [this method].

[0085] Head Perceptual Acceleration A head_on_head and head perceived angular velocity W head_on_head It is described in the head coordinate system. That is, head perceptual acceleration A head_on_head and head perceived angular velocity W head_on_head This describes, in accordance with the movement of the vehicle 10, the movement of the vehicle and the resulting relative movement of the head 2a with respect to the vehicle 10, which acts on a predetermined position on the head 2a of the subject 2, i.e., the head origin o of the head coordinate system, which in this embodiment is the central position between the left and right semicircular canals 3, as a movement that can be perceived at the head origin o depending on the orientation of the head 2a, and is expressed in the head coordinate system.

[0086] Next, in step S080, motion sickness of subject 2 is determined. Motion sickness determination is performed by the control unit 30 as motion sickness determination process 39. Motion sickness determination is performed by comparing the calculated head perception movement with a predetermined threshold. Based on the motion sickness determination result, the content to be output to the output device 40 is output.

[0087] The calculation system 1 and calculation method described herein will produce the following effects.

[0088] (1) A calculation method for calculating the movement of the head 2a that can be perceived by a subject 2 in a vehicle 10, To obtain the movement of vehicle 10 in a predetermined vehicle coordinate system, Based on the acquired movement of vehicle 10, calculate the head position movement acting on the position of head 2a in the vehicle coordinate system. To acquire image data of the head 2a in vehicle 10, Based on the acquired image data, the relative movement of the head 2a relative to the vehicle 10 is calculated. Calculate the movement of head 2a in the vehicle coordinate system based on head position movement and head relative movement. Includes.

[0089] As a result, it is possible to more accurately calculate the movement of subject 2's head 2a, including not only the head position movement but also the relative head movement relative to the vehicle 10, without requiring the attachment of a sensor to subject 2's head 2a. For example, it is easier to estimate the degree of motion sickness of subject 2 based on the movement of their head 2a.

[0090] (2) To obtain the movement of the vehicle 10, the vehicle acceleration a of the vehicle 10 car This may further include obtaining it.

[0091] As a result, the vehicle acceleration a of acquired vehicle 10 car Based on this, the head position axial acceleration a that may act on the head 2a of subject 2. phead-A It can be calculated.

[0092] (3) Obtaining the movement of vehicle 10 is the vehicle angular velocity ω of vehicle 10. car This may further include obtaining it.

[0093] As a result, the vehicle angular velocity ω of acquired vehicle 10 car Based on this, the head position angular velocity ω that may act on the head 2a of subject 2. phead Furthermore, the vehicle angular velocity ω that may act on the head 2a of subject 2 can be estimated. car Based on this, the centrifugal acceleration a of the head position may act on the head 2a of subject 2. phead_C and head position tangential acceleration a phead_T This allows us to calculate the movement of subject 2's head 2a in more detail.

[0094] (4) Calculating the head position movement is the vehicle angular velocity ω of the acquired vehicle 10. carThis may include calculating the acceleration that may occur in the head 2a.

[0095] As a result, the vehicle's angular velocity ω car As a result, it is possible to calculate the accelerations in the pitch direction (X direction), yaw direction (Y direction), and roll direction (Z direction) that may occur in the head 2a.

[0096] (5) For example, the acceleration that may occur in the head 2a due to the angular velocity of the vehicle 10 is from the origin O of the vehicle coordinate system to a predetermined position P of the head. head Head position acting centrifugally towards the direction of centrifugal acceleration a phead_C It may include.

[0097] (6) For example, the acceleration that may occur in the head due to the angular velocity of the vehicle 10 is centered at a predetermined position P of the head 2a with respect to the origin O of the vehicle coordinate system. head The predetermined position P relative to the arc passing through the arc head Tangential acceleration a of the head position acting in the tangential direction phead_T It may include.

[0098] (7) Obtaining head relative motion is head relative acceleration a, which is the acceleration of the head 2a relative to the vehicle 10. head_to_car This may further include obtaining it.

[0099] As a result, the relative acceleration of the head a head_to_car Based on this, the movement of head 2a can be calculated more accurately.

[0100] (8) The relative head motion is obtained by the relative angular velocity of the head 2a relative to the vehicle 10, which is the head relative angular velocity ω head_to_car This may further include obtaining it.

[0101] As a result, the head relative angular velocity ω head_to_car Based on this, the movement of head 2a can be calculated more accurately.

[0102] (9) The calculation system 1 for calculating the movement of the head 2a of the subject 2 in the vehicle 10 is: The IMU21 acquires the movement of vehicle 10 in a predetermined vehicle coordinate system, A camera 22 acquires image data of the head 2a of subject 2 in vehicle 10, Control unit 30 and It has, The control unit 30 is Based on the acquired movement of vehicle 10, the head position P in the vehicle coordinate system is determined. head A head position movement calculation process 35 calculates the head position movement that acts upon it, Based on the acquired image data, a head relative motion calculation process 36 calculates the head relative motion, which is the relative movement of the head 2a with respect to the vehicle 10. Based on head position movement and head relative movement, a head movement calculation process 37 calculates the movement of the subject 2's head 2a in the vehicle coordinate system. Execute this.

[0103] (10) Based on the acquired image data, calculate the orientation of the head 2a in the vehicle coordinate system. Based on the head position and the orientation of head 2a, the perceptual head movement that subject 2 can perceive as movement of head 2a in accordance with the orientation of head 2a is calculated. It may also include the following.

[0104] As a result, without requiring the attachment of sensors to the subject's head 2a, the perceived head movement that subject 2 may perceive according to the orientation of their head 2a can be calculated based on the head position and head orientation. For example, it is easier to estimate subject 2's intoxication state based on perceived head movement.

[0105] (11) This may further include setting up a head coordinate system, The head coordinate system has its origin at a predetermined position on the head 2a, and has coordinate axes that align with each direction of the vehicle 10 when the head is in a predetermined position, and is fixed to the head 2a, and as the position of the head 2a changes from the predetermined position, the orientation of each coordinate axis changes with respect to each direction of the vehicle. Calculating head perceptual movement may include representing head positional movement in a head coordinate system.

[0106] As a result, perceived head movement can be represented as the movement of head 2a in the head coordinate system, making it easier to calculate and more intuitively represent the movement of head 2a that subject 2 can perceive.

[0107] (12) Representing head position and movement in a head coordinate system is, To calculate an index that shows the relationship between the orientation of each coordinate axis in the vehicle coordinate system and the orientation of the corresponding coordinate axis in the head coordinate system, and Using the calculated indicators, calculate the perceived head movement from the head position movement. It may include.

[0108] As a result, head perceptual movement can be calculated from head position movement using an index that shows the relationship between the vehicle coordinate system and the head coordinate system.

[0109] (13) The index may be an identity rotation matrix R, described as a scalar quantity representing the degree to which each coordinate axis of the head coordinate system is oriented relative to each coordinate axis of the vehicle coordinate system.

[0110] As a result, the relationship between the vehicle coordinate system and the head coordinate system can be represented using the unit rotation matrix R.

[0111] (14) Representing head movement in a head coordinate system is the transpose of the unit rotation matrix R T This may include the use of [a specific method / tool].

[0112] As a result, the transpose matrix R of the unit rotation matrix R T Using this method, perceptual head movement can be calculated from head movement.

[0113] (15) The head origin o of the head coordinate system may be set at the center between the left and right semicircular canals 3 of the subject 2.

[0114] As a result, by calculating the movement that can be perceived at the central position between the left and right semicircular canals, which is considered to have a high correlation with intoxication, it is possible to estimate the degree of intoxication of subject 2 with higher accuracy.

[0115] (16) The calculation system 1 for estimating the movement of the head 2a that the subject 2 can perceive in the vehicle 10 is: The IMU21 acquires the movement of vehicle 10 in a predetermined vehicle coordinate system, A camera 22 acquires image data of the head 2a in the vehicle 10, Control unit 30 and It has, The control unit 30 is Based on the acquired movement of the vehicle 10, a head position movement calculation process 35 calculates the head position movement, which is the movement of the subject 2's head 2a in the vehicle coordinate system. Based on the acquired image data, a head orientation calculation process 34 calculates the orientation of the head 2a in the vehicle coordinate system, Based on the head position and head orientation, a head perception movement calculation process 38 calculates the head perception movement that subject 2 can perceive as the movement of head 2a in accordance with the orientation of head 2a. Execute this.

[0116] The calculation system 1 and calculation method relating to this disclosure are not limited to the configuration described in the above embodiment, and various modifications are possible.

[0117] In the above embodiment, the head origin o of the head coordinate system was set to the center position between the left and right semicircular canals 3, but it is not limited to this. The origin of the head coordinate system may be set to any position on the head 2a, for example, to the center position of the head 2a.

[0118] In the above embodiment, the head coordinate system is set based on the case where the head 2a is facing the direction of travel of the vehicle 10 as a predetermined posture, but it is not limited to this. For example, if the predetermined posture of the head 2a is facing left or right with respect to the direction of travel of a railway vehicle, the x-axis of the head coordinate system is set to align with the X-axis of the vehicle coordinate system in that predetermined posture. As a result, the x-axis of the head coordinate system extends left or right with respect to the head 2a. Alternatively, each coordinate axis of the head coordinate system may be set according to the orientation of the head 2a. For example, if the predetermined posture of the head 2a is facing left or right with respect to the direction of travel, the x-axis of the head coordinate system may be set to align with the Y-axis of the vehicle coordinate system in that predetermined posture.

[0119] In the above embodiment, the head relative angular velocity ω is described in the vehicle coordinate system. head Head angular velocity W calculated using head Therefore, the head perceptual angular velocity W described in the head coordinate system head_on_head The above is the calculation, but it is not limited to this. The head relative angular velocity of head 2a with respect to vehicle 10 may also be calculated in the head coordinate system using the unit rotation matrix R for the head coordinate system. Specifically, as shown in equation (16), the head relative angular velocity ω in the head coordinate system is calculated. head Using Poisson's differential equation, we obtain the unit rotation matrix R and its transpose R. T It may also be calculated by multiplying by .

[0120]

number

[0121] In the above embodiment, the motion sickness detection process 39 is described in the head coordinate system, and the head perceptual acceleration A head_on_head and head perceived angular velocity W head_on_head The motion sickness detection process 39 is performed based on, but is not limited to, the vehicle coordinate system, head acceleration A head and head angular velocity W headIt may also be performed based on this. Even in this case, it is easier to determine the degree of intoxication of subject 2 compared to when it is based on either head position movement or head relative movement alone.

[0122] In the above embodiment, the example described was one in which head movement includes both head position movement and head relative movement, but it is not limited to this. Only head position movement may be used as the head movement, and only this head position movement may be described in the head coordinate system using the unit rotation matrix R. This allows for the calculation of the head position movement that can be perceived by head 2a, depending on the orientation of head 2a. Even in this case, it is easier to determine the degree of intoxication of subject 2 compared to the case where the orientation of head 2a is not considered. [Examples]

[0123] The movement of the head 2a of subject 2, who was riding in vehicle 10, was calculated using calculation system 1, and the calculated results were compared with the measured results.

[0124] The calculation system 1 calculates the head position movement and head relative movement, and based on these, it calculates the head acceleration A, which is the movement of head 2a in the vehicle coordinate system. head and head angular velocity W head Next, head acceleration A was calculated. head and head angular velocity W head The transpose matrix R T Using this, the head's perceived acceleration A is the movement of head 2a in the head coordinate system. head_on_head and head perceived angular velocity W head_on_head The following were calculated for each.

[0125] The measured values ​​were obtained by attaching an IMU to the head 2a of subject 2 and measuring the movement acting on the subject's head 2a. Since the IMU is attached to the head 2a of subject 2, the movement of the head 2a in the head coordinate system was measured.

[0126] As an example, head perceptual acceleration A acts on head 2a in the anterior-posterior direction. head_on_head_xFigure 9 shows the results of comparing the calculated value with the measured anterior-posterior acceleration. In the graph in Figure 9, the horizontal axis is time and the vertical axis is acceleration, showing the calculated and measured anterior-posterior acceleration acting on the head 2a. In Figure 9, the thin dotted line shows the anterior-posterior acceleration related to the measured value, and the thick solid line shows the head perceived acceleration A related to the calculated value. head_on_head_x This is shown in Figure 9. As shown, the calculation results closely reproduce the measured results.

[0127] Figure 10 is a graph summarizing the correlation between the calculated results and the measured results, with the measured values ​​plotted on the horizontal axis and the calculated values ​​on the vertical axis. The correlation equation derived from the plot in Figure 10 showed that the slope of the correlation equation between the two was 0.8167, indicating a high correlation with the measured values, and the coefficient of determination R² of the correlation equation was 0.7738, indicating low variability. Therefore, the calculation system 1 and calculation method described herein allowed for the calculation of values ​​closer to actual measurements without requiring the attachment of sensors or other devices to the head 2a of the subject 2.

[0128] [Note] The calculation system and calculation method relating to this disclosure provide the following aspects.

[0129] [Aspect 1] A calculation method for calculating the head movement of a subject in a vehicle, To acquire the movement of the vehicle in a predetermined vehicle coordinate system, Based on the acquired movement of the vehicle, calculate the head position movement acting on the position of the head in the vehicle's coordinate system. To acquire image data of the head in the aforementioned vehicle, Based on the acquired image data, the relative movement of the head relative to the vehicle is calculated. Based on the head position movement and the head relative movement, the movement of the head in the vehicle coordinate system is calculated. A calculation method that includes this.

[0130] [Aspect 2] Obtaining the motion of the vehicle further includes obtaining the acceleration of the vehicle. The calculation method described in Embodiment 1.

[0131] [Aspect 3] Obtaining the motion of the vehicle further includes obtaining the angular velocity of the vehicle. The calculation method described in embodiment 1 or 2.

[0132] [Aspect 4] Calculating the head position movement includes calculating the acceleration that may occur in the head based on the acquired angular velocity of the vehicle. The calculation method described in Embodiment 3.

[0133] [Aspect 5] The acceleration that may occur in the head due to the angular velocity of the vehicle includes centrifugal acceleration acting in the centrifugal direction from the origin of the vehicle's coordinate system toward a predetermined position of the head. The calculation method described in Embodiment 4.

[0134] [Aspect 6] The acceleration that may occur in the head due to the angular velocity of the vehicle includes an acceleration acting tangentially at a predetermined position to a circular arc passing through a predetermined position of the head, with respect to the origin of the vehicle's coordinate system. The calculation method described in aspect 4 or 5.

[0135] [Aspect 7] Obtaining the head relative motion further includes obtaining the acceleration of the head relative to the vehicle. The calculation method described in any one of embodiments 1 to 6.

[0136] [Aspect 8] Obtaining the head relative motion further includes obtaining the angular velocity of the head relative to the vehicle. The calculation method described in any one of embodiments 1 to 7.

[0137] [Aspect 9] A calculation system for calculating the head movements of a subject in a vehicle, A vehicle motion acquisition unit acquires the movement of the vehicle in a predetermined vehicle coordinate system, The vehicle includes an image data acquisition unit that acquires image data of the subject's head, Control unit and It has, The control unit, A head position movement calculation process calculates the head position movement acting on the head position in the vehicle coordinate system based on the acquired movement of the vehicle, Based on the acquired image data, a head relative motion calculation process is performed to calculate the head relative motion, which is the movement of the head relative to the vehicle. A head movement calculation process calculates the movement of the subject's head in the vehicle coordinate system based on the head position movement and the relative head movement. A calculation system that performs this task.

[0138] [Aspect 10] Based on the acquired image data, the orientation of the head in the vehicle coordinate system is calculated. Based on the head position movement and the head orientation, the perceptual head movement that the subject can perceive as head movement in accordance with the head orientation is calculated. including, The calculation method described in any one of embodiments 1 to 8.

[0139] [Aspect 11] This further includes setting the head coordinate system, The head coordinate system has an origin at a predetermined position on the head, has coordinate axes that align with each direction of the vehicle when the head is in a predetermined posture, and is fixed to the head, and as the posture of the head changes from the predetermined posture, the orientation of each of the coordinate axes changes with respect to each direction of the vehicle. Calculating the aforementioned head perceptual movement includes representing the aforementioned head position movement in the aforementioned head coordinate system. The calculation method described in aspect 10.

[0140] [Aspect 12] Representing the aforementioned head position movement in the aforementioned head coordinate system means that To calculate an index that shows the relationship between the orientation of each coordinate axis of the vehicle coordinate system and the orientation of the corresponding coordinate axis of the head coordinate system, and Using the calculated index, the perceived head movement is calculated from the head position movement. The calculation method described in embodiment 11, including the method described in embodiment 11.

[0141] [Aspect 13] The aforementioned index is a unit rotation matrix, described as a scalar quantity representing the degree to which each coordinate axis of the head coordinate system is oriented relative to each coordinate axis of the vehicle coordinate system. The calculation method described in Embodiment 12.

[0142] [Aspect 14] Representing the head position movement in the head coordinate system includes using the transpose of the unit rotation matrix. The calculation method described in Embodiment 13.

[0143] [Aspect 15] The origin of the head coordinate system is set at the center position between the left and right semicircular canals of the subject. The calculation method described in any one of embodiments 11 to 14.

[0144] [Aspect 16] A calculation system for estimating the head movements that a person in a vehicle may perceive, A vehicle motion acquisition unit acquires the movement of the vehicle in a predetermined vehicle coordinate system, The vehicle includes an image data acquisition unit that acquires image data of the head, Control unit and It has, The control unit, A head position movement calculation process calculates the head position movement, which is the movement of the subject's head in the vehicle's coordinate system, based on the acquired movement of the vehicle. a head orientation calculation process of calculating the orientation of the head in the vehicle coordinate system based on the acquired image data; and a head perceived motion calculation process of calculating a head perceived motion that the subject can perceive as the motion of the head according to the orientation of the head, based on the head position motion and the orientation of the head to execute the above processes, a calculation system.

Description of Reference Signs

[0145] 1 Calculation system 2 Subject 2a Head 3 Semicircular canal 10 Vehicle 21 IMU 22 Camera 30 Control unit 33 Head coordinate system setting process 34 Head orientation calculation process 35 Head position motion calculation process 36 Head relative motion calculation process 37 Head motion calculation process 38 Head perceived motion calculation process 39 Motion sickness determination process 40 Output device O Vehicle origin o Head origin a car Vehicle acceleration ω car Vehicle angular velocity P head Head position R Unit rotation matrix R T Transposed matrix a phead Head position acceleration a phead_A Head position axial acceleration a phead_C Head position centrifugal direction acceleration a phead_T Head position tangential direction acceleration Head position angular velocity ω phead Head relative acceleration a head_to_car Head relative angular velocity ω head_to_car Head acceleration A head Head angular velocity W head Head Perceptual Acceleration A head_on_head Head Perceptual Angular Velocity W head_on_head

Claims

1. A calculation method for calculating the head movement of a subject in a vehicle, To acquire the movement of the vehicle in a predetermined vehicle coordinate system, Based on the acquired movement of the vehicle, calculate the head position movement acting on the position of the head in the vehicle's coordinate system. To acquire image data of the head in the aforementioned vehicle, Based on the acquired image data, the relative movement of the head relative to the vehicle is calculated. Based on the head position movement and the head relative movement, the movement of the head in the vehicle coordinate system is calculated. A calculation method that includes this.

2. Obtaining the motion of the vehicle further includes obtaining the acceleration of the vehicle. The calculation method according to claim 1.

3. Obtaining the motion of the vehicle further includes obtaining the angular velocity of the vehicle. The calculation method according to claim 1 or 2.

4. Calculating the head position movement includes calculating the acceleration that may occur in the head based on the acquired angular velocity of the vehicle. The calculation method described in claim 3.

5. The acceleration that may occur in the head due to the angular velocity of the vehicle includes centrifugal acceleration acting in the centrifugal direction from the origin of the vehicle's coordinate system toward a predetermined position of the head. The calculation method described in claim 4.

6. The acceleration that may occur in the head due to the angular velocity of the vehicle includes an acceleration acting tangentially at a predetermined position to a circular arc passing through the predetermined position of the head, with respect to the origin of the vehicle's coordinate system. The calculation method described in claim 4.

7. Obtaining the head relative motion further includes obtaining the acceleration of the head relative to the vehicle. The calculation method according to claim 1 or 2.

8. Obtaining the head relative motion further includes obtaining the angular velocity of the head relative to the vehicle. The calculation method according to claim 1 or 2.

9. A calculation system for calculating the head movements of a subject in a vehicle, A vehicle motion acquisition unit acquires the movement of the vehicle in a predetermined vehicle coordinate system, The vehicle includes an image data acquisition unit that acquires image data of the subject's head, Control unit and It has, The control unit, A head position movement calculation process calculates the head position movement acting on the head position in the vehicle coordinate system based on the acquired movement of the vehicle, Based on the acquired image data, a head relative motion calculation process is performed to calculate the head relative motion, which is the movement of the head relative to the vehicle. A head movement calculation process calculates the movement of the subject's head in the vehicle coordinate system based on the head position movement and the relative head movement. A calculation system that performs this task.

Images