Simulation device and server device

The simulation device addresses the burden of setting gaze areas by using a recognition and evaluation unit to automatically assess driving performance, improving realism and accuracy through gaze area recognition and feedback.

JP2024158534AInactive Publication Date: 2024-11-08CLARION LIFE CYCLE SOLUTIONS CO LTD

Patent Information

Application Number
JP2023073798
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Setting the areas of focus for a test subject in a simulation device is time-consuming and burdensome for operators, hindering effective driving evaluation.

Method used

A simulation device that includes a display control unit, recognition unit, and evaluation unit to automatically recognize the gaze areas through image recognition and evaluate driving based on the subject's line of sight direction using a server device and simulation device configuration.

Benefits of technology

The device reduces operator burden by accurately evaluating driving performance by recognizing gaze areas and providing real-time feedback, enhancing simulation realism and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158534000001_ABST
    Figure 2024158534000001_ABST
Patent Text Reader

Abstract

To provide a simulation device capable of further appropriately evaluating driving while reducing a burden on an entrepreneur.SOLUTION: A simulation device 20 comprises: a display control section 34 which controls an image display section 60 so as to display a photographic image which is picked up by an on-vehicle camera 1; a recognition section 32 which recognizes a gaze region that an examinee who performs a simulation should gaze at, through an image recognition on the basis of a photographic image G1; and an evaluation section 36 which evaluates driving of the examinee based on whether or not a line of sight of the examinee is directed toward the gaze region of the photographic image G.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a simulation device and a server device. [Background technology]

[0002] Conventionally, simulation devices that allow subjects to have a simulated driving experience are known (see, for example, Patent Documents 1 to 5). Patent Document 1 discloses that real video data and real driving data captured and collected during driving of a real vehicle on a real road surface are taken in and reproduced in a video generating device and a tremor generating device. Patent Document 2 discloses that a plurality of meshes are set in a driving video to detect the subject's gaze position and gaze retention time, and further, on this mesh, the values ​​are tallied by area such as the left mirror area, and the values ​​of accident-free drivers and the subject are statistically compared and scored to automatically evaluate the driving aptitude of the subject.

[0003] Patent Document 3 discloses a method for acquiring and generating a target image for arbitrary viewpoint movement by using a panoramic image obtained by capturing a 360-degree periphery of a moving body such as a vehicle and expanding the panoramic image in a plane in the viewpoint direction. Patent Document 4 discloses a method for detecting and recording the head or eye movement of a subject, and evaluating whether the subject responded appropriately based on the recorded data. Patent Document 5 discloses a method for displaying an image on a VR head-mounted display and vibrating a chair to reproduce the vibration of an actual vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2005-316004 A [Patent Document 2] JP 2008-139553 A [Patent Document 3] JP 2020-8664 A [Patent Document 4] U.S. Patent Application Publication No. 2022 / 051589 [Patent Document 5] Korean Patent Publication No. 10-2120553 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, to evaluate the gaze position of a subject, the company providing the simulation device had to set the area the subject should gaze at while looking at the captured image in advance. Setting the area the subject should gaze at required a huge amount of time and effort, which was a huge burden for the company.

[0006] Therefore, an object of the present disclosure is to provide a simulation device that can more appropriately evaluate driving while reducing the burden on businesses. [Means for solving the problem]

[0007] In order to achieve the above-mentioned objective, the simulation device disclosed herein includes a display control unit that controls a display unit to display an image captured by a camera mounted on a vehicle, a recognition unit that recognizes by image recognition a gaze area that a subject performing a simulation should gaze at based on the captured image, and an evaluation unit that evaluates the subject's driving based on whether the subject's gaze is directed toward the gaze area of ​​the captured image. Effect of the Invention

[0008] In the simulation device of the present disclosure configured as above, the recognition unit recognizes the area that the subject should focus on from the captured image by image recognition, which makes it possible to more appropriately evaluate driving while reducing the burden on the business operator. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a conceptual diagram showing a schematic configuration of a simulation device in a simulation system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a functional configuration of a simulation system according to a first embodiment. [Diagram 3] 11 is a diagram for explaining a process in which a recognition unit detects a gaze area on which a subject should gaze. FIG. [Figure 4] FIG. 13 is a diagram showing an example of a display image displayed on a VR display unit of an image display unit worn by a subject. [Diagram 5] 11 is a graph showing an example of impact information detected by a G sensor. [Figure 6] FIG. 13 is a diagram showing a score table and a graph of impact information. [Figure 7] 5 is a flowchart for explaining an example of an operation of a simulation device of the simulation system according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] (First embodiment) A simulation system 100 according to a first embodiment of the present disclosure will be described below with reference to the drawings. Fig. 1 is a diagram showing a schematic configuration of a simulation device 20 in the simulation system 100 according to the first embodiment. Fig. 2 is a block diagram showing a functional configuration of the simulation system 100 according to the first embodiment.

[0011] As shown in FIGS. 1 and 2, a simulation system 100 according to the first embodiment mainly includes a server device 10 and a simulation device 20.

[0012] The server device 10 is a device that transmits images G1 (see FIG. 3) captured by an on-board camera 1 mounted on a vehicle (not shown) to the simulation device 20. The server device 10 is configured from a cloud server or the like. As shown in FIG. 2, the server device 10 is capable of communicating with a plurality of on-board cameras 1 via a communication network L1. The server device 10 is also capable of communicating with the simulation device 20 via a communication network line L2 such as the Internet.

[0013] As shown in Fig. 2, the server device 10 mainly includes a receiving unit 11, a transmitting unit 12, and an event information database 13. Hereinafter, the event information database 13 will be simply referred to as "event information DB13". The receiving unit 11 receives event information including a captured image G1 from the vehicle-mounted camera 1 via the communication network L1. The event information is information acquired by the vehicle-mounted camera 1 when an accident or an event that does not result in an accident but has a high risk of causing an accident occurs.

[0014] The receiving unit 11 stores the event information received from the vehicle-mounted camera 1 in the event information DB 13. At this time, the receiving unit 11 may appropriately classify the event information according to, for example, the vehicle type, the type of accident, etc., and store the event information in the event information DB 13. Examples of vehicle types are trucks, taxis, buses, company cars, etc. The receiving unit 11 may distinguish and classify accidents from events that do not result in an accident but have a high risk of causing an accident, and store the event information in the event information DB 13.

[0015] The transmission unit 12 transmits (outputs) one or more pieces of captured image data from various pieces of event information stored in the event information DB 13 to the simulation device 20 in response to a request from the simulation device 20.

[0016] In the first embodiment, the vehicle on which the vehicle-mounted camera 1 is mounted is a so-called commercial vehicle such as a truck, a taxi, a bus, a company car, etc. However, the vehicle is not limited to a commercial vehicle, and may be a so-called personal vehicle owned by an individual.

[0017] The vehicle-mounted camera 1 can be, for example, a "video recording type drive recorder." As shown in FIG. 2, the vehicle-mounted camera 1 mainly includes a camera unit 2, a G sensor 3, a GPS 4, a communication unit 5, a memory 6, and the like. The camera unit 2 is composed of a 360-degree camera and can capture images of the vehicle's surroundings in all directions at 360 degrees. The G sensor 3 is also called an acceleration sensor. For example, the G sensor 3 is a sensor that detects the acceleration detected by the vehicle-mounted camera 1 with three degrees of freedom. Examples of the three degrees of freedom are the acceleration in the X-axis direction, which is the forward / backward direction based on the vehicle-mounted camera 1, the acceleration in the Y-axis direction, which is the left / right direction, and the acceleration in the Z-axis direction, which is the up / down direction.

[0018] The in-vehicle camera 1 records in the memory 6 as event information various data such as the time of a predetermined period before and after the G-sensor 3 detects an acceleration equal to or greater than a predetermined value, location information acquired by the GPS 4, captured images which are moving images captured by the camera unit 2, impact information detected by the G-sensor, blinker operation, brake operation, etc. The impact information detected by the G-sensor 3 is, for example, acceleration in three axial directions of X, Y, and Z. The communication unit 5 of the in-vehicle camera 1 transmits the event information recorded in the memory 6 to the server device 10 via the communication network L1.

[0019] The simulation device 20 is a device that allows the subject P to have a simulated driving experience, and is also called a "driving simulator" or the like. As shown in FIG. 2, the simulation device 20 of the first embodiment mainly includes a control unit (information processing unit) 30, a storage unit 40, a vibration generating unit 50, an image display unit 60, and an operation unit 70.

[0020] The control unit 30 and the storage unit 40 are configured, for example, by a personal computer. The control unit 30 is mainly configured by a CPU, a RAM, etc. The storage unit 40 is mainly configured by a storage device such as a ROM, a HDD, etc., but may also include an external server or database.

[0021] The memory unit 40 temporarily or non-temporarily stores a control program for operating the simulation device 20, various data and parameters used during various operations in the control unit 30, position information of the gaze area extracted based on event information, periods related to events, etc.

[0022] The storage unit 40 also includes an event information storage unit 41 and a score table storage unit 42. The event information storage unit 41 temporarily or non-temporarily stores event information from the server device 10. The score table storage unit 42 temporarily or non-temporarily stores the score table created by the evaluation unit 36.

[0023] The control unit 30 loads a predetermined program stored in the ROM into the RAM and executes it to control the overall operation of the simulation device 20. In addition, the control unit 30 functions as a communication unit 31, a recognition unit 32, a seat angle information transmission unit 33, a display control unit 34, a motion detection unit 35, an evaluation unit 36, and a driving information display processing unit 37, as shown in FIG.

[0024] The communication unit 31 accesses the server device 10, receives the event information from the server device 10, and stores it in the event information storage unit 41. At this time, the communication unit 31 may be configured to receive all the event information stored in the event information DB 13, or may be configured to receive one or more pieces of event information corresponding to the user's designation of the vehicle type, the type of accident, and the type of event that does not result in an accident but has a high risk of causing an accident.

[0025] The recognition unit 32 acquires predetermined event information from the event information storage unit 41. The recognition unit 32 analyzes the event information to identify a period related to the event, and classifies the identified period into a motion period of the subject P described later. The recognition unit 32 recognizes a gaze area, which is an area to be gazed upon by the subject P, by image recognition based on a captured image included in the event information. Specifically, as shown in FIG. 3, the recognition unit 32 divides the captured image G1 into a plurality of regions R in a lattice shape for each frame, and detects an area R (see the shaded portion in FIG. 3) in which a mirror, a monitor, etc. are captured, by image recognition. Members such as mirrors and monitors are visual recognition members for visually recognizing the periphery of the vehicle in the captured image G1. The left mirror, right mirror, rearview mirror, and a monitor displaying an image captured around the vehicle are included in the visual recognition members for visually recognizing the periphery of the vehicle. On the other hand, a monitor that does not display an image captured around the vehicle is not included in the visual recognition members for visually recognizing the periphery of the vehicle. Next, the recognition unit 32 determines whether or not a moving object such as another vehicle or a pedestrian is captured in the detected visual component through image recognition. This moving object is an example of an object that the subject P should gaze at. The recognition unit 32 determines that the moving object is captured in the detected visual component as a gaze area A. The gaze area A is an area in which the subject P can indirectly visually recognize the moving object through the visual component.

[0026] Furthermore, the recognition unit 32 determines whether an object to be watched, for example, another vehicle, a moving object such as a pedestrian, an obstacle, etc., is captured in an area other than the area where the visual member is captured, by image recognition. The recognition unit 32 determines that an object to be watched is captured in an area other than the area where the visual member is captured as a gaze area B. The gaze area B is an area where the subject P can directly view a moving object, etc., without using a visual member.

[0027] 3 is a part of the captured image G1 captured by the camera unit 2 consisting of a 360-degree camera. In reality, the recognition unit 32 detects the gaze area A or the gaze area B by image recognition for each frame of the captured image G1 captured in all 360-degree directions. The recognition unit 32 stores the position information of the detected gaze area A or the gaze area B, for example, the coordinates of the four corner points of the gaze area A or the gaze area B, in the storage unit 40 in association with the shooting time.

[0028] Furthermore, the recognition unit 32 acquires the time when the event occurred based on the impact information detected by the G sensor 3, which is included in the event information. Hereinafter, this time may be referred to as the "event occurrence time." FIG. 5 is a graph showing an example of impact information against the time axis detected by the G sensor 3. These graphs, from the top of the page in FIG. 5, are graphs showing the change in acceleration in the X-axis direction (front-back direction of the vehicle), the Y-axis direction (left-right direction), and the Z-axis direction (up-down direction). The horizontal axis of each graph is time, and the vertical axis is acceleration (G).

[0029] In addition, in the graph along the X-axis, a positive value is an acceleration in the backward direction, and a negative value is an acceleration in the forward direction. In the graph along the Y-axis, a positive value is an acceleration in the left direction, and a negative value is an acceleration in the right direction. In the graph along the Z-axis, a positive value is an acceleration in the downward direction, and a negative value is an acceleration in the upward direction. Note that, in this embodiment, the X-axis direction is defined as the front-back direction, the Y-axis direction as the left-right direction, and the Z-axis direction as the up-down direction, but this is not limited thereto, and any direction may be defined as the X-, Y-, or Z-axis direction.

[0030] In the example of the impact information shown in FIG. 5, a large acceleration is detected in the negative direction (forward direction) in the X-axis direction (front-rear direction) (see the part surrounded by a square in FIG. 5). From this, the recognition unit 32 recognizes that an event has occurred in which the vehicle has suddenly braked and collided with another vehicle or the like, or has almost collided with another vehicle or the like. This event is an event that does not result in an accident or an accident occurring, but has a high risk of causing an accident. The recognition unit 32 stores the time when the large acceleration occurred in the storage unit 40 as the "event occurrence time" (T3). Furthermore, the recognition unit 32 identifies the time when the driver performed an action, such as the time when the driver performed a steering operation or a braking operation, based on the impact information.

[0031] FIG. 6 is a diagram showing a score table and a graph of impact information. The graph of impact information shown in FIG. 6 is a compilation of the graphs in the X-axis direction, the Y-axis direction, and the Z-axis direction shown in FIG. 5. In FIG. 6, T is the time when the subject P performed an action, such as gazing at the gaze area A or operating the operation unit 70. This time is also the time when the action detection unit 35 described later detects action information related to the action performed by the subject P, so hereinafter, this time may be referred to as the "action detection time". T3 is the event occurrence time when an accident or an event that does not lead to an accident but has a high risk of causing an accident occurs, but in the score table, it is set to a time before the time when the driver operates the brakes, etc. in the actual accident, in order to perform a more appropriate evaluation. T2 is the time immediately before the accident or the start of the period of an event that does not lead to an accident but has a high risk of causing an accident. Specifically, for example, T2 is the time when the driver starts to brake suddenly. T1 is a predetermined time immediately before the accident or before the occurrence of an event that does not lead to an accident but has a high risk of causing an accident. Specifically, for example, T1 is the time when the vehicle switches from low speed driving to normal speed driving.

[0032] The recognition unit 32 classifies the period into events based on the identified time, etc. In Fig. 6, the period less than T1 is the period when the vehicle is traveling at a low speed. The period from T1 to T2 is the period when another vehicle is approaching, that is, when another vehicle is captured in the gaze area A or the gaze area B, and the vehicle is traveling at a normal speed. The period from T2 to T3 is the period immediately before an accident, or the period when an event that has a high risk of causing an accident occurs, but does not result in an accident. The period from T3 onwards is the period after the accident occurs.

[0033] Next, the recognition unit 32 creates a score table (table) to be used for driving evaluation as shown in Fig. 6 based on time information such as the event occurrence time and impact information. The recognition unit 32 stores the created score table in the score table storage unit 42 of the storage unit 40. Note that the recognition unit 32 can be configured to execute a classification process of the event information when the simulation device 20 is started and event information is selected, and to store the created score table in the score table storage unit 42. Alternatively, the recognition unit 32 can be configured to execute a classification process in advance for all event information stored in the event information storage unit 41, for example, when event information is received from the server device 10, and to create score tables for all event information and store them in the score table storage unit 42.

[0034] The recognition unit 32 sets the score to be given to the subject P based on the operation detection time T at which the operation of the subject P is detected and the classification result of the above period. Examples of the operations to be detected are that the subject P has gazed at the gaze area or the gaze area B, that the subject P has operated the steering, that the subject P has operated the brake, and that the subject P has operated the accelerator. For example, according to the score table in FIG. 6, when the time T at which the operation that the subject P has gazed at the gaze area A or the gaze area B is detected is in the period before T1 (T < T1) and the direction of the line of sight is within the gaze area A or the gaze area B, 2 points are given. Note that since the tip of the line of sight is also the position at which the subject P is gazing, the direction of the line of sight may be referred to as the "gaze position". When the time T at which the subject P has operated the steering is T < T1 and the direction of the operation matches the correct steering direction preset based on the event information, 2 points are given. For example, when the time T at which the brake operation is detected is T < T1, 2 points are given. For example, when the event is "collision", the "correct steering direction preset in advance" is the direction opposite to the direction in which the collided obstacle exists.

[0035] When the time T at which the subject P has gazed at the gaze area A or the gaze area B is in the period from T1 or more to less than T2 (T1 ≤ T < T2) and the direction of the line of sight is within the gaze area A or the gaze area B, 5 points are given on the assumption that the other vehicle has been properly gazed at. When the time T of the brake operation is T1 ≤ T < T2, 5 points are given, and when the time T of the steering operation is T1 ≤ T < T2 and the steering direction matches the correct steering direction preset in advance, 5 points are given.

[0036] When the time T at which the subject P gazes at the gaze area A or the gaze area B is in the period of T2 or more and less than T3 (T2 ≤ T < T3), and the direction of the line of sight is within the gaze area A or the gaze area B, although it does not reach immediately before the accident or the occurrence of the accident, if the subject appropriately gazes at the gaze area A or the gaze area B in the event of a high-risk event of an accident, 10 points are given. Similarly, when the time T of the braking operation is T2 ≤ T < T3, 10 points are given, and when the time T of the steering operation is T2 ≤ T < T3 and the steering direction coincides with the preset correct steering direction, 10 points are given. On the other hand, when the time at which the subject P gazes at the gaze area A or the gaze area B, the time of the braking operation, and the time of the steering operation are in the period of T3 or more (T3 ≤ T), that is, when these operations are performed after the accident has occurred, they are not subject to scoring and no score is given. Even if the subject P performs a steering operation, if the direction of the subject P's line of sight is outside the gaze area A or the gaze area B, no score is given. This is because it is considered that the subject P performs a steering operation without looking at the area to be gazed at.

[0037] In addition, in the example shown in FIG. 6, the recognition unit 32 creates a score table based on the direction of the line of sight, the steering operation, and the braking operation, but it is not limited to this. The recognition unit 32 can also create a score table based on the accelerator operation, the traveling speed, the operation amount of the accelerator operation or the braking operation, etc.

[0038] The seat angle information transmission unit 33 acquires the accelerations in the X-axis direction, the Y-axis direction, and the Z-axis direction as seat angle information based on the impact information from the G sensor 3 included in the event information. The seat angle information transmission unit 33 transmits the acquired seat angle information to the seat angle reproduction unit 53 of the vibration generation unit 50. The seat angle reproduction unit 53 drives and controls the seat drive unit 52 based on the received seat angle information to move the seat body 51 in six degrees of freedom combined with the X, Y, and Z-axis directions and the rotation angles in each direction. The subject P can experience vibrations as if actually driving a vehicle.

[0039] The display control unit 34 controls the image reproducing unit 63 of the image display unit 60 to display the display image G3 on the VR display unit 61. More specifically, the display control unit 34 transmits the captured image G1 captured by the camera unit 2 of the vehicle-mounted camera 1 and the evaluation result image G2 generated by the driving information display processing unit 37 described later, which are included in the event information, to the image reproducing unit 63. Then, the display control unit 34 controls the image reproducing unit 63 to display (play) the display image G3 including the captured image G1 and the evaluation result image G2 based on the direction of the subject P's line of sight. At this time, it is preferable that the display control unit 34 controls the image reproducing unit 63 so that the camera unit 2 displays the captured image G1 including a viewing member for viewing the periphery of the vehicle, such as a mirror.

[0040] The motion detection unit 35 acquires motion information related to the motion of the subject P, such as the operation of the operation unit 70 and the movement of the head. For example, the motion detection unit 35 acquires a detection result from a steering angle detection unit 71a provided in the steering unit 71 of the operation unit 70, and detects the operation state of the steering unit 71 of the subject P as steering operation information. The operation state of the steering unit 71 includes the presence or absence of operation, the time when the operation by the subject P was detected, and the steering direction based on the steering angle (steering angle). The motion detection unit 35 acquires a detection result from an accelerator detection unit 72a provided in the accelerator unit 72, and detects the operation state of the accelerator unit 72 of the subject P as accelerator operation information. The operation state of the accelerator unit 72 includes the presence or absence of operation, the time when the operation by the subject P was detected, the amount of operation, and the like. The motion detection unit 35 acquires a detection result from a brake detection unit 73a provided in the brake unit 73, and detects the operation state of the brake unit 73 of the subject P as brake operation information. The operation state of the brake unit 73 includes the presence or absence of operation, the time when the operation by the subject P was detected, the amount of operation, etc. Furthermore, the action detection unit 35 detects the direction of the subject P's line of sight as line of sight information based on the line of sight detection result by the line of sight detection unit 62 provided in the image display unit 60. The action detection unit 35 transmits the detected steering operation information, accelerator operation information, brake operation information, and line of sight information to the evaluation unit 36 ​​as action information of the subject P. Note that the action detection unit 35 may regard the timing when the device detects the operation by the subject P as the time when the subject P performed the operation.

[0041] The evaluation unit 36 ​​evaluates the driving of the subject P based on the motion information detected by the motion detection unit 35 and the score table in the score table storage unit 42. For example, the evaluation unit 36 ​​evaluates the driving of the subject P based on the gaze information among the motion information of the subject, based on whether the direction of the gaze of the subject P is directed to the gaze area A or the gaze area B of the image, and whether the time T when the gaze of the subject P is directed to the gaze area A or the gaze area B of the image is appropriate, and gives a score. The evaluation unit 36 ​​may weight the gaze areas A and B so that a larger score is given to the area in which an object to be watched, such as an obstacle, is captured first than to the area in which the obstacle is captured later. The evaluation unit 36 ​​evaluates the driving of the subject P based on the steering operation information, based on whether the time T when the subject P operated the steering unit 71 is appropriate, and gives a score. The evaluation unit 36 ​​evaluates the driving of the subject P based on the brake operation information, based on whether the time T when the subject P operated the brake unit 73 is appropriate, and gives a score. The evaluation unit 36 ​​transmits the evaluation result to the driving information display processing unit 37. When the evaluation unit 36 ​​evaluates that the motion detection time T at which the motion detection unit 35 detects the motion of the subject P, for example, the time T at which the braking operation is detected, is after the event occurrence time T3, the evaluation unit 36 ​​transmits the evaluation result to the driving information display processing unit 37 as warning information to notify the subject P, for example, by turning on a warning flag.

[0042] The driving information display processing unit 37 generates an evaluation result image G2 based on the evaluation result and the captured image received from the evaluation unit 36. In the evaluation result image G2, the evaluation results such as the score, the steering operation state, the steering angle, and the brake operation state are expressed by letters, numbers, etc. In addition, when the driving information display processing unit 37 receives warning information from the evaluation unit 36, it includes a notification regarding the driving of the subject P evaluated by the evaluation unit 36 ​​in the evaluation result image G2. The driving information display processing unit 37 transmits the generated evaluation result image G2 to the display control unit 34.

[0043] The vibration generating unit 50 generates vibrations based on the detection result of the G sensor 3 when the surroundings of the vehicle are photographed by the camera unit 2 in accordance with the display of the display image G3 on the image display unit 60, and applies vibrations to the subject P similar to those experienced when driving an actual vehicle. The vibration generating unit 50 mainly includes a seat body 51, a seat driving unit 52, and a seat angle reproducing unit 53.

[0044] The seat body 51 is a member on which the subject P sits. The seat drive unit 52 is controlled by the seat angle reproduction unit 53. The seat drive unit 52 can drive the seat body 51 with six degrees of freedom: rotation in each of the X-axis direction (front-back direction), Y-axis direction (left-right direction), and Z-axis direction (up-down direction). The seat drive unit 52 is composed of an electric actuator such as a stepping motor or a cylinder. The seat angle reproduction unit 53 receives seat angle information from the seat angle information transmission unit 33, and drives and controls the seat drive unit 52 based on the seat angle information.

[0045] The image display unit 60 is an image display device that presents the photographed image G1 photographed by the camera unit 2 and the display image G3 including the photographed image G1 to the subject P under the control of the display control unit 34. The image display unit 60 is composed of a head mounted display and is worn on the head of the subject P, but is not limited to this. The image display unit 60 mainly includes a VR display unit 61, a line of sight detection unit 62, and an image reproduction unit 63. The image display unit 60 also includes components that are included in a general image display device, such as an optical system such as a backlight, a projection optical system, etc.

[0046] The VR display unit 61 is disposed in front of the eyes of the subject P who wears the image display unit 60 on his / her head, and the display image G3 is displayed (reproduced) on the display surface 61a. The VR display unit 61 is composed of a liquid crystal display, an organic EL display, or the like. The VR display unit 61 may be any type that allows the subject P to view the display image G3. For example, the VR display unit 61 may be a transparent type that allows the subject P to view the external real environment together with the display image, or may be a non-transparent type. The line-of-sight detection unit 62 is composed of an acceleration sensor or the like, and detects the direction of the line of sight based on the movement of the head of the subject P. In this embodiment, the gaze position, which is the direction of the line of sight, is expressed as the position where the extension of the line of sight of the subject P intersects with the display image G3 displayed on the display surface 61a of the VR display unit 61, that is, the three-dimensional coordinates in the display image G3. The line-of-sight detection unit 62 transmits position information of the gaze position, which is the direction of the detected line of sight, to the image reproduction unit 63.

[0047] The line-of-sight detection unit 62 is not limited to an acceleration sensor, and may be, as another example, an imaging device that captures an image of the subject P. Such line-of-sight detection unit 62 detects the eyes from the captured image of the subject P by image recognition, and detects the direction of the line of sight.

[0048] The image reproducing unit 63 generates a display image G3 including the photographed image G1 and the evaluation result image G2 received from the display control unit 34, and displays the generated display image G3 on the display surface 61a of the VR display unit 61 (see FIG. 4). At this time, as shown in FIG. 1, the image reproducing unit 63 sets a spherical projection surface V in a 360-degree direction around the subject P, and displays the photographed image G1 of a predetermined range of area W centered on the line of sight direction detected by the line of sight detection unit 62 on the VR display unit 61 while moving the area in response to changes in the line of sight direction. This allows the subject P to view the photographed image G1 corresponding to his or her own line of sight direction and the display image G3 including the evaluation result image G2.

[0049] That is, when the subject P faces forward or backward, an image of the front or rear of the vehicle is displayed on the VR display unit 61, respectively. Also, when the subject P faces left, right, up, or down, an image of the left, right, up, or down of the vehicle is displayed on the VR display unit 61, respectively. Also, the image reproduction unit 63 transmits the direction of the line of sight to the action detection unit 35. This direction of the line of sight is transmitted from the action detection unit 35 to the evaluation unit 36 ​​and is used for evaluation of the driving.

[0050] 4, the image reproducing unit 63 displays an evaluation result image G2 including the evaluation result of the driving evaluation and a notification regarding the driving of the subject P (e.g., a warning message, a warning image, etc.) in the driving evaluation display area 61b of the display surface 61a of the VR display unit 61. The image display unit 60 may also include a speaker, and may output the notification regarding the driving of the subject P by voice (e.g., a warning sound, a warning message, etc.).

[0051] The operation unit 70 is a member that accepts various operations (motions) of the subject P. The operation unit 70 mainly includes a steering unit 71, an accelerator unit 72, and a brake unit 73. The operation unit 70 may also include a clutch unit, a turn signal unit, etc.

[0052] The steering unit 71 is a member simulating a steering wheel of a vehicle, and receives input of a steering operation from the subject P. The steering unit 71 includes a steering angle detection unit 71a, and detects the time when the steering operation was performed, the steering angle (steering angle) during the steering operation, etc. based on the operation of the steering unit 71 by the subject P, and transmits the detection result to the motion detection unit 35.

[0053] The accelerator unit 72 is a member simulating a vehicle accelerator pedal and receives accelerator operation by the subject P. The accelerator unit 72 includes an accelerator detection unit 72a. The accelerator detection unit 72a detects whether or not the accelerator is operated, the time when the accelerator is operated, the amount of operation, etc. based on the operation of the accelerator unit 72 by the subject P, and transmits the detection results to the motion detection unit 35. The brake unit 73 is a member simulating a vehicle brake pedal and receives brake operation by the subject P. The brake unit 73 includes a brake detection unit 73a. The brake detection unit 73a detects whether or not the brake is operated, the time when the brake is operated, the amount of operation, etc. based on the operation of the brake unit 73 by the subject P, and transmits the detection results to the motion detection unit 35.

[0054] An example of the operation of the simulation system 100 according to the first embodiment configured as described above will be described below with reference to the flowchart in Fig. 7. Fig. 7 shows an example of the operation of the simulation device 20, but the operation of the simulation device 20 is not limited to the operation in Fig. 7. Furthermore, the steps (processes) in the flowchart in Fig. 7 are not necessarily executed in the order shown in Fig. 7, and they can be executed in a different order as appropriate, or multiple steps can be executed simultaneously.

[0055] The operation shown in Fig. 7 is started when the power supply of the simulation device 20 is turned on by an operator or the like who operates the simulation system 100, and the simulation system 100 is started. Also, it is assumed that the server device 10 stores various event information in the event information DB 13. Also, steps S1 to S3 are executed before the subject P experiences the simulation. In other words, the processing in steps S1 to S3 is pre-processing. Step S4 and onwards are executed when the subject P sits in the seat body 51 of the simulation device 20, and the subject P or the operator presses a start button or the like (not shown).

[0056] First, in step S1 of the flowchart shown in the left diagram of FIG. 7, the communication unit 31 of the simulation device 20 accesses the server device 10, receives event information from the server device 10, and stores it in the event information storage unit 41. In the next step S2, the recognition unit 32 executes a classification process of the event information. Specifically, the recognition unit 32 acquires desired event information from the event information storage unit 41, and acquires an event occurrence time T3 of an accident, an event that does not lead to an accident but has a high risk of causing an accident, etc., based on the impact information included in the event information. In addition, the recognition unit 32 detects the visual member and the moving object by image recognition based on the captured image G1, and identifies a time other than the event occurrence time T3, such as T1 and T2 shown in FIG. 6, based on the detection result and the impact information, and classifies the period into the motion period of the subject P based on the identified time.

[0057] In the next step S3, the recognition unit 32 creates a score table shown in Fig. 5 based on the identified times T1 to T3 and period classification results, and stores it in the score table storage unit 42. When the process of step S3 is completed, the program goes into a waiting state. Then, when the simulation device 20 receives an operation input such as a start button by the subject P or the operator, the program proceeds to step S4.

[0058] In step S4, the control unit 30 acquires an elapsed time from the event information. The elapsed time indicates the time at which the event was detected from the shooting start time to the shooting end time of the captured image G1, in minutes and seconds from the shooting start time. In the next step S5, the display control unit 34 acquires one frame of the captured image G1 from the event information according to the elapsed time acquired in step S4.

[0059] In the next step S6, the seat angle information transmission unit 33 acquires the accelerations in the X, Y, and Z axis directions for one frame as seat angle information corresponding to the elapsed time acquired from the event information. In the next step S7, the seat angle information transmission unit 33 transmits the seat angle information to the seat angle reproduction unit 53. Based on this seat angle information, the seat angle reproduction unit 53 drives and controls the seat drive unit 52 to move the seat main body 51 with six degrees of freedom, which is the combination of the X, Y, and Z axis directions and the rotation angles in each direction, to apply vibration to the subject P.

[0060] In the next step S8, the motion detection unit 35 acquires motion information. That is, the motion detection unit 35 detects steering operation information, accelerator operation information, and brake operation information based on the detection results from the steering angle detection unit 71a, accelerator detection unit 72a, and brake detection unit 73a, respectively. The motion detection unit 35 also detects gaze information based on the detection result from the gaze detection unit 62. The motion detection unit 35 transmits the detected motion information to the evaluation unit 36.

[0061] In the next step S9, the evaluation unit 36 ​​evaluates the driving of the subject P based on the motion information detected in step S8 and the score table created in step S3 and stored in the score table storage unit 42. Details of the process of step S9 will be described as follows based on the flowchart shown on the right side of the page in FIG.

[0062] When the driving evaluation process is started, first, in step S901, the evaluation unit 36 ​​determines whether the gaze position of the subject P based on the gaze information is within gaze area A or gaze area B at the elapsed time point acquired in step S4 above. In the next step S902, when the evaluation unit 36 ​​determines that the gaze position is within gaze area A or gaze area B (YES), the program proceeds to step S903. On the other hand, when the evaluation unit 36 ​​determines that the gaze position is outside gaze area A or gaze area B (NO), the program skips step S903 and proceeds to step S904.

[0063] In step S903, the evaluation unit 36 obtains a score from the score table based on the time T when the subject P gazed at the gaze area A or the gaze area B, and adds the score. Specifically, when the time T at which the operation indicating that the gaze position is within the gaze area A or the gaze area B is detected satisfies T < T1, T1 ≤ T < T2, or T2 ≤ T < T3, the score obtained from the score table is added. Then, the program proceeds to step S904.

[0064] In step S904, when there is a braking operation by the subject P, the evaluation unit 36 determines whether the time T at which the braking operation was performed is within the period preset in the score table. In the next step S905, when the evaluation unit 36 determines that the time T at which the subject P performed the braking operation is within the preset period (T < T1, T1 ≤ T < T2, or T2 ≤ T < T3) (YES), the program proceeds to step S906. On the other hand, when the evaluation unit 36 determines that the time T at which the subject P performed the braking operation is outside the preset period (T3 ≤ T) (NO), the program skips step S906 and proceeds to step S907.

[0065] In step S906, the evaluation unit 36 obtains a score from the score table based on the time T at which the braking operation was detected, and adds the score. Specifically, when the time T at which the braking operation was detected satisfies T < T1, T1 ≤ T < T2, or T2 ≤ T < T3, the score obtained from the score table is added. Here, the evaluation unit 36 can also be configured to quantify the operation amount of the brake and take it into account in the score. Then, the program proceeds to step S907.

[0066] In step S907, when there is a steering operation by the subject P, the evaluation unit 36 determines whether the steering direction matches the correct steering direction at the time point of the elapsed time (T < T1, T1 ≤ T < T2, or T2 ≤ T < T3) obtained in step S4 above.

[0067] In the next step S908, when the evaluation unit 36 determines that the steering direction of the subject P matches the preset correct steering direction (YES), the program proceeds to step S909. On the other hand, when the evaluation unit 36 determines that they do not match (NO), the program skips step S909 and proceeds to step S910.

[0068] In step S909, the evaluation unit 36 obtains a score from the score table based on the timing of the steering operation and adds the score. Specifically, according to whether the time T when the steering operation is detected is T < T1, T1 ≤ T < T2, or T2 ≤ T < T3, the score obtained from the score table is added. Then, the program proceeds to step S910.

[0069] In step S910, the evaluation unit 36 transmits the evaluation result including the calculated score to the driving information display processing unit 37. Also, when the time T when the subject is gazing, braking, or steering is out of the scoring range (T3 ≤ T), depending on the operation, the evaluation unit 36 transmits to the driving information display processing unit 37 an evaluation result including warning information indicating that the time T is after the event occurrence time T3. Then, the program proceeds to end and the process of driving evaluation ends.

[0070] When the driving evaluation process ends, the program returns to the flowchart in the left figure of FIG. 7 and proceeds to step S10. In this step S10, the driving information display processing unit 37 generates an evaluation result image G2 based on the evaluation result received from the evaluation unit 36 in step S910. The driving information display processing unit 37 transmits the generated evaluation result image G2 to the display control unit 34.

[0071] In the next step S11, the display control unit 34 transmits the evaluation result image G2 and the photographed image G1 received from the driving information display processing unit 37 to the image reproduction unit 63, and controls the image reproduction unit 63 to generate a display image G3 and display it on the VR display unit 61. The image reproduction unit 63 generates a display image G3 including the photographed image G1 of a predetermined range according to the direction of the line of sight of the subject P and the evaluation result image G2, as shown in FIG. 4, for example, and displays it on the display surface 61a of the VR display unit 61. The evaluation result image G2 is displayed in the driving evaluation display area 61b. Therefore, the subject P can check the score, steering angle, warning, etc. for his / her driving from the evaluation result image G2 while viewing the photographed image G1 of the in-vehicle camera 1 reproduced in the display image G3.

[0072] In the next step S12, the evaluation unit 36 ​​adds the display time for one frame to the elapsed time. In the next step S13, the evaluation unit 36 ​​determines whether the captured image G1 reproduced by the image reproduction unit 63 has ended. When the evaluation unit 36 ​​determines that the captured image G1 has not ended (there is a next frame) (NO), the program returns to step S4, and the process of steps S4 to S12 is repeated for the next frame.

[0073] On the other hand, when the evaluation unit 36 ​​judges that the photographed image G1 has ended, that is, that there is no next frame (YES), the program proceeds to the end, and the simulation of the subject P ends. Thereafter, when the subject P executes a simulation of another event, or when another subject executes a simulation, the simulation device 20 repeats the process of steps S1 to S13. In addition, the simulation device 20 executes a predetermined end process automatically or in response to a power-off operation by the operator, and then turns off the power. This ends the simulation process by the simulation system 100.

[0074] Second embodiment In a simulation system 100 according to a second embodiment of the present disclosure, a server device 10 includes a recognition unit (a server-side recognition unit 14 indicated by a dashed line in FIG. 2) instead of the recognition unit 32 of the simulation device 20. The simulation system 100 according to the second embodiment includes a basic configuration similar to that of the simulation system 100 according to the first embodiment shown in FIGS. 1, 2, etc., except that the server device 10 includes the server-side recognition unit 14 and the simulation device 20 does not include the recognition unit 32.

[0075] The server-side recognition unit 14 has almost the same functions as the recognition unit 32 of the first embodiment. More specifically, the server-side recognition unit 14 acquires event information from the event information DB 13, and executes a classification process similar to the classification process of step S2 in FIG. 7. At this time, the server-side recognition unit 14 can be configured to execute the classification process for all event information stored in the event information DB 13, and store the classification results in the event information DB 13 or the like. Then, the transmission unit 12 can be configured to transmit the event information specified by the simulation device 20 and the classification results from the event information DB 13 to the simulation device 20. Alternatively, the server-side recognition unit 14 can be configured to acquire one event information specified by the simulation device 20 from the event information DB 13, execute the classification process for the event information, and transmit the event information and the classification results to the simulation device 20.

[0076] As described above, the simulation device 20 of the first embodiment includes a display control unit 34 that controls the display unit (image display unit 60) to display a captured image G1 captured by a camera (vehicle-mounted camera 1) mounted on a vehicle, a recognition unit 32 that recognizes, from the captured image G1, a gaze area A or a gaze area B to be gazed upon by a subject P performing a simulation by image recognition, and an evaluation unit 36 ​​that evaluates the driving of the subject P based on whether the subject P's line of sight is directed toward the gaze area A or the gaze area B of the captured image G1. Also, in the simulation system 100 of the second embodiment, the server device 10 includes a recognition unit (server-side recognition unit 14).

[0077] In this way, the recognition unit 32 automatically recognizes the gaze area A or the gaze area B that the subject P should gaze at, so there is no need for the business operator or the like to recognize the gaze area A or the gaze area B while viewing the captured image G1. Therefore, in the simulation device 20 and the simulation system 100 of the first embodiment described above, the recognition unit 32 automatically reduces the burden on the business operator and makes it possible to more appropriately evaluate driving. As a result, the subject P performing the simulation can appropriately grasp the evaluation of his / her own driving in real time while viewing the captured image G1 captured by the vehicle-mounted camera 1 more realistically in the virtual space.

[0078] Recently, a driver monitoring system (DMS) mounted on a vehicle monitors the driver's behavior based on real-time captured images during actual driving. Therefore, the DMS cannot grasp the time when the driver should gaze at the gaze area A or the gaze area B, or the time when the driver should perform a brake operation, a steering operation, or the like, before an event occurs that is likely to cause an accident or an accident does not occur. In contrast, the simulation device 20 of each of the above-mentioned embodiments classifies the past period before the occurrence of an event such as an accident into a plurality of periods according to the situation based on the already captured captured image G1, and sets a score for each period to evaluate whether the time when the subject P gazed at the gaze area A or the gaze area B, the time when the brake operation was detected, and the time when the steering operation was detected are appropriate. The simulation device 20 reproduces the captured image G1 and the vibration of the vehicle, and evaluates the driving of the subject P in stages based on a score table created in advance. Therefore, the simulation device 20 and the simulation system 100 of each of the above-mentioned embodiments can evaluate the behavior of the subject P not only when an event occurs, but also before the event occurs in more detail and more appropriately.

[0079] Furthermore, the display control unit 34 in each of the above embodiments can control the image display unit 60 (VR display unit 61) to display the captured image G1 including a visual component (e.g., a mirror, a monitor, etc.) for visually recognizing the periphery of the vehicle. With this configuration, the simulation device 20 and the simulation system 100 can provide the subject P with a higher sense of realism because an image of the visual component to be visually recognized by the subject P is presented. Furthermore, the subject P can properly visually recognize the visual component to be visually recognized.

[0080] Moreover, the recognition unit (recognition unit 32, server-side recognition unit 14) in each of the above embodiments recognizes, by image recognition, a visual recognition member (e.g., a mirror, a monitor, etc.) for visually recognizing the periphery of the vehicle in the captured image G1 as the gaze area A. With this configuration, the simulation device 20 and the simulation system 100 can more appropriately evaluate the driving of the subject P depending on whether the subject P gazes at the visual recognition member.

[0081] Furthermore, the recognition unit (recognition unit 32, server-side recognition unit 14) in each of the above embodiments recognizes a moving object (e.g., another vehicle, a pedestrian, etc.) in the captured image G1 as the gaze area B. With this configuration, the simulation device 20 and the simulation system 100 can more appropriately evaluate the driving based on whether the subject P properly visually recognized the moving object and whether the subject P properly performed braking or steering to avoid the moving object.

[0082] Moreover, the simulation device 20 of each of the above embodiments includes a storage unit 40 that stores an event occurrence time T3 at which a predetermined event occurs in the captured image G1, and a motion detection unit 35 that detects motion information related to a predetermined motion by the subject P. The evaluation unit 36 ​​evaluates the driving of the subject P based on the motion detection time T at which the motion detection unit 35 detected the motion information and the event occurrence time T3. With this configuration, the simulation device 20 and the simulation system 100 can more accurately determine whether the motion input by the subject P is appropriate.

[0083] Furthermore, when the motion detection time T is after the event occurrence time T3, the evaluation unit 36 ​​in each of the above embodiments outputs a notification regarding the driving of the subject P. With this configuration, the simulation device 20 and the simulation system 100 in each of the above embodiments can notify that the motion input by the subject P is inappropriate.

[0084] Furthermore, the evaluation unit 36 ​​in each of the above embodiments acquires the event occurrence time T3 based on the impact information acquired by the acceleration sensor equipped in the vehicle-mounted camera 1, and evaluates the driving of the subject P based on the acquired event occurrence time T3, motion detection time T, and predetermined motion. With this configuration, the simulation device 20 and the simulation system 100 in each of the above embodiments can recognize information related to the event in more detail, such as the event occurrence time, the type of event, and the motion that the subject P should make in each period when the period in which the event information is recorded is classified into a plurality of periods according to the situation, and can more appropriately evaluate the driving.

[0085] Furthermore, the evaluation unit 36 ​​in each of the above embodiments positively evaluates the driving of the subject P when the motion detection time T is before the event occurrence time T3, and evaluates the driving of the subject P in stages according to the period from the motion detection time T to the event occurrence time T3. With this configuration, the simulation device 20 and the simulation system 100 can evaluate the driving of the subject P in more detail.

[0086] Moreover, the simulation device 20 of each of the above embodiments includes an on-board camera 1 that captures images in all 360-degree directions around the vehicle, and displays a captured image G1 of a predetermined range on the VR display unit 61 according to the direction of the subject P's line of sight. With this configuration, the subject P can view a desired direction in all 360-degree directions around the vehicle, particularly the rear, simply by moving his or her line of sight or head. This allows the subject P to experience a more realistic simulation.

[0087] The simulation device 20 of each of the above embodiments includes a seat body 51, a seat drive unit 52, and a seat angle reproduction unit 53, and includes a vibration generation unit 50 that generates vibrations based on vibration information acquired by the G sensor 3 of the vehicle-mounted camera 1. With this configuration, the subject P can experience a more realistic simulation by experiencing the same vibrations as when driving an actual vehicle while viewing the captured image G1 reproduced by the VR display unit 61. Since the vibration generation unit 50 of the simulation device 20 is controlled based on the vibration information acquired by the G sensor 3 of the vehicle-mounted camera 1, it is not necessary to connect the vehicle-mounted camera 1 to a gyro sensor of the vehicle. The simulation device 20 can collect information required for the simulation with a simple configuration in which the vehicle-mounted camera 1 is mounted on the vehicle.

[0088] Furthermore, the simulation device 20 of each of the above embodiments reproduces the 360-degree captured image G1 in all directions, while vibrating the seat body 51 in response to the captured image G1. This allows the subject P to experience a more realistic simulation.

[0089] The server device 10 of the second embodiment transmits the captured image G1 captured by the vehicle-mounted camera 1 mounted on the vehicle to the simulation device 20. The server device 10 of the second embodiment includes a receiving unit 11 that receives the captured image G1 from the vehicle-mounted camera 1, a recognition unit (server-side recognition unit 14) that recognizes, by image recognition, the gaze area A or the gaze area B to be gazed upon by the subject P performing the simulation based on the captured image G1, and a transmission unit 12 that transmits the gaze area A or the gaze area B in the captured image G1 to the simulation device 20. With this configuration, in the simulation system 100 of the second embodiment, the server device 10 recognizes the gaze area A or the gaze area B in advance, so there is no need to perform image processing in the simulation device 20 to recognize the gaze area A or the gaze area B. Therefore, the simulation system 100 of the second embodiment can further simplify and speed up the processing performed by the simulation device 20.

[0090] Although the embodiments of the present disclosure have been described in detail above with reference to the drawings, the specific configurations are not limited to these embodiments, and design changes that do not deviate from the gist of the present disclosure are included in the present disclosure.

[0091] As a modified example, the simulation device 20 may include an information terminal (controller) operated by an operator. Suitable examples of the information terminal include a tablet terminal, a smartphone, a notebook computer, and the like, which are equipped with an operation unit such as a touch panel and a display unit such as a liquid crystal display. The operator can select a desired event from a menu screen displayed on the information terminal. The simulation device 20, which has received this selection, can obtain corresponding event information from the server device 10 and execute a simulation process. Furthermore, the simulation device 20 displays the captured image G1 and the display image G3 including the evaluation result image G2 displayed on the VR display unit 61 on the display unit of the information terminal, so that the operator can grasp the driving state and evaluation result of the subject P. [Explanation of symbols]

[0092] 1: Vehicle-mounted camera (camera) 10: Server device 11: Receiving section 12: Transmitting section 14: Server side recognition unit (recognition unit) 20: Simulation device 32: Recognition unit 34: Display control unit 35: Motion detection unit 36: Evaluation unit 40: Storage section 60: Image display section (display section) A: Gaze area B: Gaze area G1: Photographed image P: Subject T: Motion detection time T3: Event occurrence time

Claims

1. a display control unit that controls the display unit to display an image captured by a camera mounted on the vehicle; a recognition unit that recognizes a gaze area to be gazed upon by a subject performing a simulation based on the captured image by image recognition; an evaluation unit that evaluates the driving of the subject based on whether the line of sight of the subject is directed toward the gaze area of ​​the captured image; A simulation device comprising:

2. The display control unit controls the display unit to display a captured image including a visual recognition member for visually recognizing the periphery of the vehicle. The simulation device according to claim 1 .

3. The recognition unit recognizes, as the gaze area, a visual recognition member for visually recognizing the periphery of the vehicle in the captured image by image recognition. The simulation device according to claim 1 .

4. The recognition unit recognizes a moving object in the captured image as the gaze area. The simulation device according to claim 1 .

5. a storage unit that stores an event occurrence time when a predetermined event occurs in the captured image; a motion detection unit that detects motion information related to a predetermined motion by the subject, The evaluation unit evaluates the driving of the subject based on a motion detection time at which the motion detection unit detects the motion information and a time at which the event occurs. The simulation device according to claim 1 .

6. The evaluation unit outputs a notification regarding the driving of the subject when the motion detection time is after the event occurrence time. The simulation device according to claim 5.

7. The evaluation unit acquires the event occurrence time based on impact information acquired by an acceleration sensor equipped in the camera, and evaluates the driving of the subject based on the acquired event occurrence time, the motion detection time, and the predetermined motion.

6. The simulation device according to claim 5.

8. A server device that transmits images captured by a camera mounted on a vehicle to a simulation device, a receiving unit that receives the captured image from the camera; a recognition unit that recognizes a gaze area to be gazed upon by a subject performing a simulation based on the captured image by image recognition; a transmission unit that transmits the gaze area in the captured image to the simulation device; A server device comprising:

Citation Information

Patent Citations

  • Driving simulator

    JP2005316004A

  • Driving aptitude diagnosing method, evaluation standard determining method for driving aptitude diagnosis, and driving aptitude diagnostic program

    JP2008139553A

  • Driving simulator

    JP2020008664A

  • Multimode Driver Simulator for Multipurpose Vehicle using VR HMD and Monitor simultaneously

    KR102120553B1

  • Systems and methods for virtual reality based driver situational awareness recovery training

    US20220051589A1

Cited By

  • Simulation system and simulation device

    WO2026126625A1

  • On-vehicle device

    WO2026126626A1