Simulation scene generator, experiencing device, and action improvement assistance system

Abstract

To effectively enhance a skill of a less skilled person.SOLUTION: A simulation scene generator according to the present invention comprises: an improvement factor selection unit for selecting an improvement factor for an action of a subject by at least comparing subject action data 330a that pertains to the action of the subject for whom technical improvement is to be made with exemplar person's action data 330b that pertains to an action of an exemplar person, and outputting selected improvement factor data that is the data of the improvement factor; a base scene selection unit for selecting, as a base scene, a simulation space based on the selected improvement factor data; and a scene correction unit for generating and outputting simulation scene data that is data of a simulation scene in which an addition / correction model corresponding to the improvement factor is added to the base scene. The simulation space refers to an action space of the subject.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a simulated scene generation device, an experience device, and a motion improvement support system. [Background technology]

[0002] The rapid expansion of demand for electronic commerce (EC) has led to a chronic shortage of professional drivers, weakening the transportation infrastructure and creating a global social problem. Under these circumstances, there is a need for transportation efficiency improvements based on zero accidents. Along with this increase in transportation efficiency, it is becoming increasingly important to provide support for the efficient improvement of professional drivers' driving skills and safety awareness.

[0003] In response to this, Patent Document 1 states that "the driving simulation device is equipped with a control device, an operating device, and a display device, and simulates vehicle driving based on a simulated training scenario, and the control device has an image generation unit that generates a simulated image based on a simulated training scenario of a predetermined difficulty level and outputs it to the display device, an operation acquisition unit that acquires from the operating device an operation signal representing operation of the operating device by the trainee during the mock training using the simulated image displayed on the display device, a determination unit that determines the driving proficiency of the trainee based on the operation signal, and a difficulty change unit that changes the difficulty of the mock training scenario by changing at least the number of caution objects that are displayed simultaneously on the simulated image depending on the trainee's proficiency" (see abstract). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-163488 Summary of the Invention [Problem to be solved by the invention]

[0005] The invention described in Patent Document 1 is based on the presentation of course information prepared in advance. Therefore, it is different from the environment in which subjects with inexperienced driving skills actually drive. Furthermore, in Patent Document 1, the difficulty level is changed only by changing the number of warning objects displayed simultaneously depending on the subject's level of proficiency, which leaves room for improvement.

[0006] The present invention has been made in view of the above background, and an object of the present invention is to efficiently improve the skills of people who are not skilled in the art. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention comprises an improvement factor selection unit that selects improvement factors for the behavior of a subject by at least comparing subject behavior data, which is data on the behavior of a subject who is the target of technical improvement, with exemplary behavior data, which is data on the behavior of an exemplary person, and outputs selected improvement factor data, which is data on the improvement factors; a base scene selection unit that selects a simulated space based on the selected improvement factor data as a base scene; and a scene modification unit that generates and outputs simulated scene data, which is data on a simulated scene in which an additional / modified model, which is a model corresponding to the improvement factors, is added to the base scene, wherein the simulated space is the behavioral space of the subject. Other solutions will be described as appropriate in the embodiments. [Effects of the Invention]

[0008] According to the present invention, the skills of people with less skill can be improved efficiently. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a functional block diagram showing the configuration of an action improvement support system according to an embodiment of the present invention. [Figure 2] 1 is a functional block diagram showing the configuration of a simulated space generating device according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram showing data constituting simulated space data. [Figure 4] 1 is a functional block diagram showing the configuration of a behavioral data collection device according to an embodiment of the present invention, and a diagram showing data constituting behavioral data. [Figure 5] 1 is a functional block diagram showing the configuration of a simulated scene generation device according to an embodiment of the present invention. [Figure 6] FIG. 2 is a functional block diagram of an improvement factor extraction unit in the present embodiment. [Figure 7] 10 is a flowchart showing the procedure of an operation performed by an improvement factor extraction unit. [Figure 8] FIG. 2 is a functional block diagram of a simulated scene generation unit in the present embodiment. [Figure 9] 10 is a flowchart showing the procedure of an operation performed by a simulated scene generating unit. [Figure 10] 10A and 10B are diagrams illustrating a specific example of the operation of a scene modification unit in the simulated scene generation unit. [Figure 11] FIG. 2 is a functional block diagram of the experience device according to the present embodiment. [Figure 12] 10 is a flowchart showing the procedure of the operation performed by the experience device. [Figure 13] 1 is a diagram illustrating a hardware configuration of a simulated scene generation device according to an embodiment of the present invention. [Figure 14] FIG. 2 is a diagram illustrating the hardware configuration of the experience device according to the present embodiment. [Figure 15] FIG. 2 is a diagram showing an example of a display screen displayed on the experience device according to the present embodiment. [Figure 16] FIG. 10 is a diagram showing an example of an attention-calling display displayed on a display screen. [Figure 17] 10 is a flowchart showing the procedure of an operation performed by a simulated scene generating section in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Next, a mode for carrying out the present invention (referred to as an "embodiment") will be described in detail with reference to the drawings as appropriate. Hereinafter, embodiments of a simulated scene generating device 400 and a motion improvement support device according to the present invention will be described with reference to FIGS.

[0011] [Movement Improvement Support System 1] FIG. 1 is a functional block diagram showing the configuration of a motion improvement support system 1 according to this embodiment.

[0012] The movement improvement support system 1 includes a simulated space generation device 100, a behavioral data collection device 300, a simulated scene generation device 400, and an experience device 500. The movement improvement support system 1 further includes subject behavior data 330a, model behavior data 330b, and simulated space data 200.

[0013] The simulated space generating device 100 generates simulated space data 200 based on sensor data 111 (see FIG. 2 ) collected by the behavioral data collecting device 300, and outputs the generated simulated space data 200. The simulated space data 200 is data of a simulated space. The simulated space is a virtual space that is a reproduction (simulation) of the scenery seen by a subject P1 who is the target of technical improvement and receives behavior improvement support while driving. It is desirable that the simulated space data 200 be a simulation of a space that the subject P1 normally uses. The output simulated space data 200 is input to a simulated scene generating device 400. The simulated space generating device 100 will be described in detail later with reference to FIG. 2A.

[0014] The behavioral data collection device 300 generates and outputs subject behavior data 330a and exemplary driver behavior data 330b. The subject behavior data 330a is data collected from a vehicle driven by a subject P1 with low driving proficiency who receives behavior improvement support. The exemplary driver behavior data 330b is data collected from a vehicle driven by a skilled driver (referred to as exemplary driver P2) who is capable of ideal behavior and has very little room for behavior improvement. In this way, "exemplary" means a person with high proficiency (higher proficiency than subject P1) with respect to the technology that is the subject of the simulated scene 712.

[0015] The output subject behavior data 330a and model subject behavior data 330b are input to the simulated scene generation device 400. The behavior data collection device 300 will be described later with reference to Fig. 4. Note that the subject P1 and the model subject P2 will be collectively referred to as "drivers" where appropriate.

[0016] The simulated scene generating device 400 generates and outputs simulated scene data 431, which is data for a simulated scene 712 (see FIG. 10), which is a scene emphasizing the subject P1's behavior that needs to be improved. The simulated scene 712, which will be described later, is a scene in which the subject P1's driving needs to be improved. The simulated scene data 431 is generated based on the simulated space data 200 generated by the simulated space generating device 100, and the subject behavior data 330a and the model behavior data 330b generated by the behavior data collecting device 300. The output simulated scene data 431 is input to the experience device 500. The simulated scene generating device 400 will be described later with reference to FIGS. 5 to 10.

[0017] The experience device 500 displays a simulated scene 712 (see FIG. 10 ) in which improvement factors appropriate for the subject P1 are emphasized, based on the simulated scene data 431 output from the simulated scene generation device 400. The simulated scene 712 may be CG (Computer Graphics) or may be generated based on video captured by the camera 311 (see FIG. 4 ). This allows the subject P1 to virtually experience scenes in which the subject P1 should be careful when driving. The virtual experience is achieved by visualizing the simulated scene 712 in which the subject P1 normally drives, for example, using VR goggles or a monitor. The subject P1 performs the virtual experience by operating control devices such as the steering wheel, accelerator, and brake to drive the vehicle within the simulated scene 712.

[0018] Various data of the subject P1 in the experience device 500 (such as the operation log of the control device and line of sight) is output as feedback data 541. The feedback data 541 is reflected in the subject behavior data 330a and is used to update the simulated scene 712 emphasized by the simulated scene generation device 400.

[0019] Although the simulated space generation device 100 and the simulated scene generation device 400 are assumed to be independent devices, they may be one device. Also, the subject behavior data 330a, the model behavior data 330b, and the simulated space data 200 are assumed to be stored in databases, but they may be stored in a storage device 442 included in the simulated scene generation device 400 or the like.

[0020] (Simulated space generation device 100) Fig. 2 is a functional block diagram showing the configuration of the simulated space generating device 100 according to this embodiment, and Fig. 3 is a diagram showing data constituting simulated space data 200.

[0021] The simulated space generating device 100 includes a sensor data acquiring unit 101 , a map data acquiring unit 102 , and a simulated space generating unit 103 .

[0022] The sensor data acquisition unit 101 acquires sensor data 111 and outputs the acquired sensor data 111 to the simulated space generation unit 103. The sensor data 111 is, for example, data acquired by a sensor mounted on a data collection vehicle for acquiring the sensor data 111. The data collection vehicle is a vehicle that is driven by the subject P1 or the model P2 for work or the like and has a sensor attached to it. The sensor data 111 may be acquired from a sensor attached to the data collection vehicle, or may be acquired from a sensor originally provided in the vehicle. As shown in FIG. 1, the sensor data 111 is data collected by a behavioral data collection device 300 mounted on the vehicle driven by the subject P1.

[0023] The sensor data 111 includes image data captured by a plurality of cameras 311 (see FIG. 4) mounted so as to be able to observe 360 ​​degrees around the vehicle. The sensor data 111 also includes data acquired by a millimeter-wave radar for acquiring the distance and relative speed to a preceding vehicle, etc. The sensor data 111 also includes point cloud data acquired by a LiDAR (Light Detection And Ranging) 316 (see FIG. 4) for acquiring three-dimensional data of structures around the vehicle as a point cloud. The sensor data 111 also includes vehicle speed data 331 (see FIG. 4) representing the speed of the host vehicle acquired from a vehicle speed sensor 317 (see FIG. 4). The sensor data 111 also includes steering angle data 332 (see FIG. 4) representing the steering angle of the steering wheel of the host vehicle acquired from a steering angle sensor 318 (see FIG. 4). The sensor data 111 also includes latitude and longitude data representing the absolute position of the vehicle acquired by a GPS 319 (see FIG. 4) including an RTK-GPS (Real Time Kinematic-Global Positioning System) or the like. In this way, the sensor data 111 partially overlaps with the behavior data 330 (subject behavior data 330a) described later in FIG.

[0024] The generated simulated space desirably reproduces the space normally used by the subject P1 receiving the motion improvement support. Therefore, the sensor data 111 is desirably obtained by sensing information about the space normally used by the subject P1 receiving the motion improvement support. The sensor data 111 may include information necessary for the simulated space generation unit 103 (described later) to generate the simulated space data 200, and may include sensor data 111 other than the above. The sensor data acquisition unit 101 receives the sensor data 111 at a constant frequency and outputs it to the simulated space generation unit 103. For example, if the sensor is the camera 311 described above, the sensor data acquisition unit 101 continuously acquires the sensor data 111 at a high frequency, for example, 30 times per second. The same applies to other sensor data 111. The sensor data acquisition unit 101 receives the sensor data 111 at a frequency determined for each sensor and outputs the received sensor data 111 to the simulated space generation unit 103.

[0025] The map data acquisition unit 102 acquires map data 112 and outputs the acquired map data 112 to the simulated space generation unit 103. As described above, it is desirable that the generated simulated space reproduces a space normally used by the subject P1 who receives the motion improvement support. Therefore, it is desirable that the map data 112 include information corresponding to the places normally used by the subject P1 who receives the motion improvement support. The map data 112 must include at least information on the places where the sensor data 111 was acquired. The map data 112 is information included in a high-precision map for autonomous driving or a map for a car navigation system. Specifically, the map data 112 includes, for example, vector data of lane centerlines that indicate the center positions of driving lanes. The map data 112 also includes vector data of road markings such as white lines that indicate driving lanes, stop lines, and pedestrian crossings. The map data 112 also includes vector data of road boundaries that indicate the boundaries of drivable roads. The map data 112 also includes position data of traffic signals that include corresponding driving lane information. The map data 112 also includes vector data of road network information that indicates the connection state of driving lanes based on the connection state of lane center lines, etc. The map data 112 also includes road attribute information that indicates the allowed driving directions of driving lanes, etc. The map data 112 also includes traffic rule information that indicates the speed limit of driving lanes, etc. The map data 112 only needs to include information necessary for the simulated space generation unit 103 (described later) to generate the simulated space data 200, and may include information other than information included in high-precision maps for autonomous driving and maps for car navigation systems. In this embodiment, vector data refers to data based on position vectors.

[0026] The simulated space generation unit 103 receives sensor data 111 output from the sensor data acquisition unit 101 and map data 112 output from the map data acquisition unit 102. Then, the simulated space generation unit 103 uses the sensor data 111 output from the sensor data acquisition unit 101 to generate simulated space data 200 that enables visual reproduction of the simulated space of the range acquired by the map data 112 output from the map data acquisition unit 102.

[0027] The simulated space data 200 includes information included in maps for automated driving and car navigation systems (i.e., map data 112). As shown in Fig. 3, the simulated space data 200 includes lane centerline data 201, road boundary data 202, road marking data 203, traffic signal data 204, and mirror data 205. Furthermore, the simulated space data 200 includes road network data 206, road attribute data 207, traffic rule data 208, simulated space rendering source data 209, and traffic participant data 210.

[0028] Lane centerline data 201 is vector data that indicates the center position of a driving lane. Road boundary data 202 is vector data that indicates the boundaries of drivable roads. Road boundaries are determined by curbs, guardrails, etc. Road marking data 203 is vector data that indicates white lines that indicate driving lanes, stop lines, crosswalks, etc.

[0029] The traffic light data 204 is vector data that represents the position of a traffic light. The traffic light data 204 includes information about the driving lanes corresponding to the traffic lights. For example, the driving lanes corresponding to the traffic lights are driving lanes in which vehicles must stop when the traffic light is "red."

[0030] The mirror data 205 is vector data that represents the positions of mirrors installed on the road. The road network data 206 is vector data that represents the connection state of driving lanes based on the connection state of lane center lines, etc. The road attribute data 207 is data that represents the allowed direction of travel on the driving lane, etc. The traffic rule data 208 is data that represents the speed limit of the driving lane, etc.

[0031] Each piece of data from the lane center line data 201 to the road network data 206 is directly expressed as absolute coordinate information such as latitude and longitude information. Alternatively, each piece of data from the lane center line data 201 to the road network data 206 is expressed as reference absolute coordinates (for example, GPS data) and relative data based on the reference absolute coordinates. When relative data is used, a form that can be converted into absolute coordinate information (for example, a conversion formula) is also stored in the lane center line data 201 to the road network data 206.

[0032] The simulated space rendering source data 209 is generated by known techniques such as SLAM (Simultaneous Localization and Mapping) and NeRF (Neural Radiance Fields) included in the information output from the sensor data 111. For example, the simulated space rendering source data 209 is a combination of a 3D point cloud acquired by a LiDAR 316 (see FIG. 4) and a brightness value acquired from a camera 311 (see FIG. 4) corresponding to each coordinate of the point cloud. In other words, one specific example of the simulated space rendering source data 209 is 3D point cloud data measured by a LiDAR 316.

[0033] The coordinate system of the simulated space rendering source data 209 is integrated based on the coordinate system of the map data 112 output from the map data acquisition unit 102. Such integration of the coordinate systems is performed by, for example, comparing highly accurate absolute coordinates acquired by RTK-GPS or the like included in the sensor data 111 with the map absolute coordinates included in the map data 112.

[0034] The simulated space generation unit 103 can visually reproduce the simulated space by applying known rendering techniques based on the simulated space rendering source data 209. As described above, since the coordinate systems are integrated, the image reproduced in the simulated space corresponds to the map attribute information in the lane center line data 201 to the traffic rule data 208.

[0035] The traffic participant data 210 is data on other vehicles, motorcycles, and pedestrians that are placed by the simulated space generation unit 103 using data such as camera images, millimeter-wave radar, and LiDAR 316 point clouds included in the sensor data 111. The simulated space generation unit 103 recognizes the positions and movements of other vehicles, motorcycles, pedestrians, etc. based on the camera images, millimeter-wave radar, and LiDAR point clouds. The simulated space generation unit 103 then records the types, positions, speeds, etc. of objects such as other vehicles, motorcycles, pedestrians, etc. in the simulated space data 200. Other vehicles are vehicles other than the vehicle being driven by the driver. By including the traffic participant data 210 in the simulated space data 200, the movements of vehicles, pedestrians, and motorcycles can be reproduced exactly as observed in the simulated space displayed on the experience device 500.

[0036] In this way, the simulated space data 200 is data relating to the simulated space which is the behavior space of the subject P1.

[0037] As shown in FIG. 1, simulated space data 200 output from a simulated space generating device 100 is input to a simulated scene generating device 400.

[0038] (Behavioral data collection device 300) FIG. 4 is a functional block diagram showing the configuration of the behavioral data collection device 300 according to this embodiment, and a diagram showing data constituting the behavioral data 330. As shown in FIG.

[0039] The behavioral data collection device 300 is attached to a vehicle that a driver drives for work or the like. Hereinafter, the vehicle that a driver drives for work or the like will be referred to simply as a vehicle as appropriate. The behavioral data collection device 300 includes a camera 311, an in-vehicle camera 314, a radar 315, a LiDAR 316, and the like. Furthermore, the behavioral data collection device 300 includes a vehicle speed sensor 317, a steering angle sensor 318, and a GPS 319. Furthermore, the behavioral data collection device 300 includes a behavioral data collection unit 320.

[0040] The cameras 311 are attached to the vehicle and arranged so that, for example, external information from 360 degrees around the vehicle can be observed. In the example shown in Fig. 4, three cameras 311 are shown, but more than three cameras 311 may be installed, or two or less cameras 311 may be installed. The images acquired by the cameras 311 are output to the behavioral data collection unit 320 as image log data while the vehicle is traveling.

[0041] The interior camera 314 is attached to the interior of the vehicle and is positioned so as to mainly observe the direction of the driver's face and line of sight. In the example shown in Fig. 4, one interior camera 314 is shown, but two or more interior cameras 314 may be provided in the vehicle. The video captured by the interior camera 314 is output to the behavioral data collection unit 320 as video log data that indicates the driver's state while the vehicle is traveling.

[0042] The radar 315 is a millimeter wave radar or the like. The radar 315 is attached to the vehicle and positioned so as to be able to observe the distance and relative speed to other vehicles traveling in the vicinity. In the example shown in FIG. 4, one radar 315 is shown, but two or more radars 315 may be provided in the vehicle. Data acquired by the radar 315 is output to the behavioral data collection unit 320 as log data indicating the state of the vehicle relative to other surrounding vehicles while the vehicle is traveling.

[0043] The LiDAR 316 is attached to the vehicle and positioned so as to be able to observe 360 ​​degrees around the vehicle. Although one LiDAR 316 is shown in the example shown in Fig. 4, two or more LiDARs 316 may be provided on the vehicle. The data acquired by the LiDAR 316, together with the data acquired by the camera 311 and radar 315, is output to the behavioral data collection unit 320 as log data representing external environment information of 360 degrees around the vehicle while the vehicle is traveling.

[0044] The vehicle speed sensor 317 and the steering angle sensor 318 are mounted on the vehicle and measure the vehicle speed and steering angle of the vehicle they are mounted on. The measured data is output to the behavior data collection unit 320.

[0045] The GPS 319 is, for example, an RTK-GPS or the like, and is installed in the vehicle. The GPS 319 measures the absolute position of the vehicle in which it is installed, and outputs the measurement result to the behavior data collection unit 320.

[0046] The behavioral data collection unit 320 receives various data output from the camera 311 to the GPS 319. The behavioral data collection device 300 then processes the collected various data and outputs it as behavioral data 330.

[0047] The behavior data 330 is a compilation of driving characteristics of the driver who drove the vehicle. The behavior data 330 includes vehicle speed data 331, steering angle data 332, acceleration data 333, yaw rate data 334, and driving trajectory data 335. The behavior data 330 further includes inter-vehicle distance data 336, merging timing data 337, gaze position data 338, and the like.

[0048] The vehicle speed data 331 and steering angle data 332 are, for example, the outputs of the vehicle speed sensor 317 and steering angle sensor 318. The acceleration data 333 and yaw rate data 334 are calculated by the behavior data collection unit 320 by calculating the time-series difference between the outputs of the vehicle speed sensor 317 and steering angle sensor 318. The behavior data collection unit 320 records the calculated vehicle speed data 331 and steering angle data 332 in the behavior data 330. However, the vehicle speed data 331 to yaw rate data 334 may be acquired using other calculation methods or by sensors additionally installed in the behavior data collection device 300.

[0049] The behavior data collection unit 320 calculates the traveling trajectory data 335 using the outputs of the vehicle speed sensor 317 and the steering angle sensor 318 by a known dead reckoning technique and records the calculated data in the behavior data 330. The traveling trajectory data 335 may also be calculated using the output of the GPS 319.

[0050] The behavior data collection unit 320 recognizes surrounding vehicles using outputs from the camera 311, radar 315, LiDAR 316, etc., and then calculates the inter-vehicle distance data 336 between the subject vehicle and each of the surrounding vehicles. The behavior data collection unit 320 then records the calculated inter-vehicle distance data 336 in the behavior data 330.

[0051] The gaze position data 338 is generated by the behavior data collection unit 320 using a known technique to recognize the driver's head direction and line of sight based on the output of the in-vehicle camera 314, and then calculating the gaze position. The behavior data collection unit 320 then records the gaze position data 338 in the behavior data 330.

[0052] A subject P1 (see FIG. 1) operates a vehicle equipped with the behavior data collection device 300, and the resulting behavior data 330 becomes subject behavior data 330a. Also, a model subject P2 (see FIG. 1) operates a vehicle equipped with the behavior data collection device 300, and the resulting behavior data 330 becomes model subject behavior data 330b. Each of the subject behavior data 330a and model subject behavior data 330b (behavior data 330) is input to the simulated scene generation device 400.

[0053] That is, the subject behavior data 330a is data regarding the behavior of the subject P1. Also, the model subject behavior data 330b is data regarding the behavior of the model subject P2. In this embodiment, the subject behavior data 330a is data regarding the behavior of the subject P1 when driving. Also, the model subject behavior data 330b is data regarding the behavior of the model subject P2 when driving.

[0054] (Simulated scene generating device 400) Fig. 5 is a functional block diagram showing the configuration of a simulated scene generating device 400 according to this embodiment. In Fig. 5, the same components as those shown in Fig. 1 are given the same reference numerals and their description will be omitted.

[0055] The simulated scene generating device 400 includes an improvement factor extracting section 410 and a simulated scene generating section 420 .

[0056] The improvement factor extraction unit 410 receives the simulated space data 200, the subject person behavior data 330a, and the model person behavior data 330b. Then, the improvement factor extraction unit 410 uses the subject person behavior data 330a to the simulated space data 200 to extract and select factors (improvement factors) that the subject P1 should improve, and outputs them. Details of the improvement factor extraction unit 410 will be described later with reference to FIGS. 6 and 7. The output improvement factors are input to the simulated scene generation unit 420.

[0057] The simulated scene generating unit 420 generates simulated scene data 431, which is data of a simulated scene 712 (see FIG. 10) that emphasizes a scene in which the subject P1 should improve his / her behavior. The simulated scene data 431 is generated by emphasizing a scene in which the subject P1 should improve his / her behavior in the simulated space using the improvement factors output from the improvement factor extracting unit 410 and the simulated space data 200, etc. Details of the simulated scene generating unit 420 will be described later with reference to FIGS. 8 to 10. The output simulated scene data 431 is input to the experience-based device 500.

[0058] (Improvement factor extraction unit 410) Fig. 6 is a functional block diagram of the improvement factor extraction unit 410 in this embodiment. Fig. 7 is a flowchart showing the procedure of the operation performed by the improvement factor extraction unit 410. The operation performed by the improvement factor extraction unit 410 will be described with reference to Figs. 1, 3, and 4 as appropriate.

[0059] The improvement factor extraction unit 410 includes an evaluation unit 411 and an improvement factor selection unit 415. The evaluation unit 411 includes a traffic rule compliance level evaluation unit 411a, a safety confirmation level evaluation unit 411b, a safety margin evaluation unit 411c, and an operation skill evaluation unit 411d.

[0060] The traffic rule compliance evaluation unit 411a calculates a traffic rule compliance score to evaluate whether the subject P1 is complying with traffic rules (S101 in FIG. 7). At this time, the traffic rule compliance evaluation unit 411a uses the subject behavior data 330a and information included in the simulated space data 200. The higher the traffic rule compliance score, the better the compliance with traffic rules. The traffic rule compliance score is calculated in chronological order for the entire simulated space data 200 corresponding to the acquisition section of the subject behavior data 330a.

[0061] For example, the traffic rule compliance level evaluation unit 411a refers to the vehicle speed data 331 included in the subject person behavior data 330a and the traffic rule data 208 included in the simulated space data 200. Then, the traffic rule compliance level evaluation unit 411a calculates the difference between the vehicle speed data 331 and the speed limit included in the traffic rule data 208. As a result, the traffic rule compliance level evaluation unit 411a calculates a score indicating whether the vehicle speed data 331 exceeds the speed limit set in the traffic rule data 208.

[0062] Alternatively, the traffic rule compliance evaluation unit 411a refers to the stop line positions included in the road marking data 203 included in the simulated space data 200 and the vehicle speed data 331 included in the subject behavior data 330a. Then, the traffic rule compliance evaluation unit 411a calculates a score indicating whether the vehicle speed at the stop line position is "0", i.e., whether the vehicle is temporarily stopped.

[0063] Alternatively, the traffic rule compliance evaluation unit 411a may refer to the traffic light color state included in the simulated space data 200 and the vehicle speed data 331 included in the subject behavior data 330a to calculate a score indicating whether or not a traffic light has been ignored. For example, if the vehicle speed is not "0" when the traffic light is red, the score will be the lowest, and if the vehicle speed is not "0" when the traffic light is yellow, the score will be the next lowest. Then, if the traffic light is green, the score will be higher. This score may take into account not only whether or not the vehicle speed is "0" but also the magnitude of the speed and the traffic light state.

[0064] Alternatively, the traffic rule compliance evaluation unit 411a refers to information such as lane change prohibition and overtaking prohibition in the traffic rule data 208 included in the simulated space data 200, and the driving trajectory data 335 included in the subject behavior data 330a. Then, the traffic rule compliance evaluation unit 411a calculates a score indicating whether or not a lane change or overtaking has been performed in a lane change prohibition or overtaking prohibition section.

[0065] The traffic rule compliance score may include other information related to the degree of compliance with traffic rules than those listed above. The traffic rule compliance evaluation unit 411a outputs a traffic rule compliance score including one or more of the scores presented above. The traffic rule compliance score is calculated in a time series manner for the entire acquisition section of the subject behavior data 330a. A score calculated in a time series manner in this way is appropriately referred to as a time series score.

[0066] The safety confirmation level evaluation unit 411b uses the subject behavior data 330a and the exemplary person behavior data 330b to calculate a safety confirmation level evaluation score representing the difference in gaze positions between the subject P1 and the exemplary person P2 (S102 in FIG. 7). The higher the safety confirmation level evaluation score, the more similar the gaze position of the subject P1 is to that of the exemplary person P2. The safety confirmation level evaluation score is calculated chronologically for the entire acquisition section of the subject behavior data 330a. However, the safety confirmation level evaluation unit 411b may weight the safety confirmation level evaluation score for gaze positions in scenes requiring specific attention, such as when turning right or left, or in cases involving parking lots, residential roads, or parked vehicles. For example, if there is a large difference in gaze positions in scenes requiring specific attention, the safety confirmation level evaluation unit 411b further reduces the safety confirmation level evaluation score by weighting. In this case, the safety confirmation level evaluation unit 411b also references the simulated space data 200 as necessary. The safe delivery confirmation degree evaluation score may include scores other than those related to the safety confirmation degree presented above. The safety confirmation degree evaluation unit 411b outputs the calculated safety confirmation degree evaluation score.

[0067] The safety margin evaluation unit 411c calculates a safety margin evaluation score that evaluates the safety margin of the subject P1 using the simulated space data 200, the subject behavior data 330a, and the model subject behavior data 330b (S103 in FIG. 7). The safety margin is the distance between vehicles, the distance to an oncoming vehicle when turning right, etc.

[0068] A higher safety margin evaluation score indicates that a sufficient safety margin is secured and safety is ensured. The safety margin evaluation score is calculated in time series over the entire acquisition section of the subject behavior data 330a.

[0069] The safety margin evaluation score is, for example, a comparison result between the inter-vehicle distance for each vehicle speed of the subject P1 (see FIG. 1) and the inter-vehicle distance for each vehicle speed of the model subject P2 (see FIG. 2). The inter-vehicle distance for each vehicle speed is calculated from the vehicle speed data 331 and inter-vehicle distance data 336 of the subject subject behavior data 330a and the model subject behavior data 330b. The safety margin evaluation unit 411c calculates a safety margin evaluation score that is set so that the larger the difference, the smaller the value.

[0070] Alternatively, the safety margin evaluation unit 411c calculates a safety margin evaluation score that indicates whether a right turn or merging is being performed at an appropriate timing. Such a safety margin evaluation score is calculated using the following data (A1) to (A3): (A1) Road network data 206 included in the simulated space data 200. (A2) Merging timing data 337 included in the subject person behavior data 330a. (A3) Merging timing data 337 included in the model person behavior data 330b.

[0071] That is, the safety margin evaluation unit 411c calculates a safety margin evaluation score that indicates whether the right turn / merging timing is appropriate for the surrounding vehicle speed and the distance to other vehicles. The appropriate right turn / merging timing is calculated from the merging timing data 337 included in the model driver behavior data 330b.

[0072] Alternatively, the safety margin evaluation unit 411c calculates the safety margin evaluation score by referring to the following data (B1) to (B3): (B1) Vehicle speed data 331 included in the subject behavior data 330a. (B2) Vehicle speed data 331 included in the model person behavior data 330b. (B3) Road attribute data 207 and traffic rule data 208 included in the simulated space data 200.

[0073] Then, the safety margin evaluation unit 411c calculates a safety margin evaluation score that indicates whether or not the vehicle is traveling at a speed appropriate for the road, for example, whether or not the vehicle is traveling at a speed that is sufficiently slower than the speed limit on a community road.

[0074] The safety margin evaluation score may include other safety margin-related information than those presented above. The safety margin evaluation unit 411c outputs one or more of the safety evaluation scores presented above.

[0075] The operation skill evaluation unit 411d uses the subject behavior data 330a and the model behavior data 330b to calculate an operation skill evaluation score that evaluates the operation skill of subject P1 (S104 in FIG. 7). A higher operation skill evaluation score indicates higher operation skill. The operation skill evaluation score is calculated chronologically for the entire acquisition section of the subject behavior data 330a. As the operation skill evaluation score, the operation skill evaluation unit 411d evaluates, for example, the difference at each time between the driving trajectory data 335 included in the subject behavior data 330a and the driving trajectory data 335 included in the model behavior data 330b. In this case, the operation skill evaluation unit 411d calculates an operation skill evaluation score that is set to a smaller value as the difference increases.

[0076] Alternatively, the operation skill evaluation unit 411d compares the acceleration data 333 included in the subject behavior data 330a with the acceleration data 333 included in the model behavior data 330b. Then, the operation skill evaluation unit 411d calculates an operation skill evaluation score that is set to a smaller value as the difference between the acceleration data 333 of the subject behavior data 330a and the acceleration data 333 of the model behavior data 330b increases.

[0077] Alternatively, the operation skill evaluation unit 411d compares the yaw rate data 334 included in the subject behavior data 330a with the yaw rate data 334 included in the model subject behavior data 330b. Then, the operation skill evaluation unit 411d calculates a score that is set to a smaller value as the difference between the yaw rate data 334 of the subject behavior data 330a and the yaw rate data 334 of the model subject behavior data 330b increases.

[0078] The operation skill evaluation score may include content other than the content presented above. The operation skill evaluation unit 411d outputs one or more of the operation skill evaluation scores presented above.

[0079] In this embodiment, the traffic rule compliance score, the safety confirmation level evaluation score, the safety margin evaluation score, and the operation skill evaluation score are calculated, but scores other than these may be calculated. Also, for example, the calculation of the traffic rule compliance score may be omitted.

[0080] When calculating the traffic rule compliance score, safety confirmation evaluation score, safety margin evaluation score, and operation skill evaluation score, behavioral data 330 relating to areas where the driving areas of the subject P1 and the exemplary person P2 overlap is used.

[0081] The traffic rule compliance score includes multiple time series scores, such as a time series score for compliance with speed limits, a time series score for compliance with stop line positions, etc. The safety margin evaluation score includes multiple time series scores, such as a time series score for safety margins when turning right and while driving.

[0082] In this way, the evaluation unit 411 calculates a score for the behavior of the subject P1 by comparing at least the subject behavior data 330a and the model subject behavior data 330b. In this embodiment, the scores are a traffic rule compliance score, a safety confirmation level evaluation score, a safety margin evaluation score, and an operation skill evaluation score. The evaluation unit 411 then calculates a score for each of the multiple behaviors of the subject P1. In this embodiment, the multiple behaviors of the subject P1 are safety confirmation, ensuring a safety margin, operation skill, etc.

[0083] The improvement factor selection unit 415 selects factors (improvement factors) that should be improved by the subject P1 (S105 in FIG. 7). At this time, the improvement factor selection unit 415 uses the time-series scores output from the traffic rule compliance level evaluation unit 411a, the safety confirmation level evaluation unit 411b, the safety margin evaluation unit 411c, and the operation skill evaluation unit 411d. For example, the improvement factor selection unit 415 calculates the average of the time-series scores included in the traffic rule compliance level evaluation unit 411a. Similarly, the improvement factor selection unit 415 averages the time-series scores included in the safety confirmation level evaluation unit 411b, the safety margin evaluation unit 411c, and the operation skill evaluation unit 411d. Then, the improvement factor selection unit 415 selects the factor with the lowest average score as the improvement factor.

[0084] For example, if the safety confirmation level evaluation score is the lowest, the improvement factor selection unit 415 selects the safety confirmation level as the improvement factor. Then, the improvement factor selection unit 415 outputs a predetermined numerical value representing the selected improvement factor and a time-series score as selected improvement factor data 416. The predetermined numerical value representing the selected improvement factor is an ID or the like indicating the improvement factor. In this case, the output time-series score is the safety confirmation level evaluation score itself.

[0085] Alternatively, if the traffic rule compliance score output from the traffic rule compliance evaluation unit 411a is the lowest, the improvement factor selection unit 415 selects the traffic rule compliance as the improvement factor. Then, the improvement factor selection unit 415 outputs a predetermined numerical value representing the selected improvement factor and a time-series score as selected improvement factor data 416. The predetermined numerical value representing the selected improvement factor is an ID or the like indicating the improvement factor. In this case, the output time-series score is the traffic rule compliance score itself.

[0086] In this embodiment, all of the time-series scores included in the selected traffic rule compliance score, safety confirmation level evaluation score, safety margin evaluation score, and operation skill evaluation score are output as the selected improvement factor data 416. However, this is not limiting, and in the case where multiple time-series scores are included, such as the traffic rule compliance score and the operation skill evaluation score, one of the time-series scores may be selected by the improvement factor selection unit 415.

[0087] For example, if the average value of the scores representing the stopping conditions at stop lines output from the traffic rule compliance evaluation unit 411a is the lowest, the improvement factor selection unit 415 selects the time-series score at the stop line as the improvement factor. Then, the improvement factor selection unit 415 outputs a predetermined numerical value representing the selected factor and the time-series score as selected improvement factor data 416. In this case, the predetermined numerical value representing the selected factor is an ID or the like indicating the stopping conditions at stop lines, and the time-series score is a time-series score related to the stopping conditions at stop lines.

[0088] However, the selection of improvement factors is not limited to the above, and may be performed by other methods utilizing scores. In this case, weighting values ​​for the areas to be improved for the subject P1 may be input from outside, and the improvement factor selection unit 415 may select improvement factors taking into consideration the weighting values. The selected improvement factors are output as selected improvement factor data 416 and input to the simulated scene generation unit 420.

[0089] In this manner, the improvement factor selection unit 415 at least compares the subject person behavior data 330a with the model person behavior data 330b. Then, the improvement factor selection unit 415 selects an improvement factor for the subject person P1's behavior and outputs selected improvement factor data 416, which is data on the improvement factor. Furthermore, the improvement factor selection unit 415 selects the subject person P1's behavior as an improvement factor based on the score from among the plurality of behaviors. In this embodiment, the plurality of behaviors include safety confirmation, ensuring safety margins, operation skills, etc., as described above. Furthermore, in this embodiment, the score includes a safety confirmation level evaluation score, a safety margin evaluation score, an operation skill evaluation score, etc.

[0090] 6, in the improvement factor extraction unit 410, the improvement factor selection unit 415 outputs score data 417 together with selected improvement factor data 416. The score data 417 includes all of the traffic rule compliance score, safety confirmation level evaluation score, safety margin evaluation score, and operation skill evaluation score, as well as the model driver behavior data 330b.

[0091] (Simulated scene generation unit 420) Fig. 8 is a functional block diagram of the simulated scene generation unit 420 in this embodiment. Fig. 9 is a flowchart showing the procedure of the operation performed by the simulated scene generation unit 420. The operation performed by the simulated scene generation unit 420 will be described with reference to Figs. 8 and 9.

[0092] The simulated scene generation unit 420 includes a base scene selection unit 421 and a scene modification unit 423. The scene modification unit 423 includes an improvement factor emphasis model addition / modification unit 424 and model data 425.

[0093] The base scene selection unit 421 selects a base scene 711 (see FIG. 10) using the simulated space data 200, the selected improvement factor data 416, the score data 417, etc. (S201 in FIG. 9).

[0094] The base scene 711 is selected from the simulated space data 200, for example, at the time when the time-series scores included in the selected improvement factor data 416 are minimum. The selected base scene 711 is output to the scene modification unit 423 as base scene data 422. The base scene data 422 is composed of the same information types as the simulated space data 200. In this way, the base scene selection unit 421 selects a simulated space based on the selected improvement factor data 416 as the base scene 711. The selected base scene 711 is sent to the scene modification unit 423 as the base scene data 422.

[0095] As described above, the scene modification unit 423 includes the improvement factor emphasis model adding / modifying unit 424 and model data 425. The model data 425 is configured as a table in which the correspondence between improvement factors and the corresponding added / modified models M (see FIG. 10) is described. The added / modified models M are models of traffic participants such as pedestrians and motorcycles. For example, factors behind ignoring a stop sign and models of pedestrian movements that become dangerous when a stop sign is ignored are associated and stored in the model data 425. The model data 425 may be created in advance or may be created each time the scene modification unit 423 performs processing.

[0096] The improvement factor emphasis model adding / modifying unit 424 generates a simulated scene 712 (see FIG. 10 ), which is a scene in which the improvement factors of the subject P1 receiving behavior improvement support are emphasized. In this case, the improvement factor emphasis model adding / modifying unit 424 generates the simulated scene 712 using the selected improvement factor data 416, the base scene data 422, and the model data 425. The improvement factor emphasis model adding / modifying unit 424 adds traffic participants such as pedestrians, vehicles, and motorcycles to the base scene data 422, or modifies the behavior of the traffic participants included in the base scene data 422, taking into consideration the behavior so as to emphasize the factors selected in the selected improvement factor data 416. An example of modifying the base scene data 422 will be described later with reference to FIG. 10 .

[0097] Specifically, the improvement factor emphasizing model adding / modifying unit 424 refers to the model data 425 and adds an adding / modifying model M corresponding to the improvement factor included in the selected improvement factor data 416 to the base scene 711 (see FIG. 10) (S202 in FIG. 9). Note that in this embodiment, emphasizing an improvement factor means adding a model of a virtual traffic participant to the base scene 711 included in the base scene data 422. This makes it possible to make the target person P1 aware of the possibility that he or she has overlooked a traffic participant.

[0098] 3 is data of actual traffic participants, whereas the added / modified model M added by the improvement factor emphasis model adding / modifying unit 424 is data of virtual traffic participants.

[0099] The simulated scene 712 is obtained by adding an improvement factor emphasis model to the base scene 711. The simulated scene data 431, which is the data of the simulated scene 712, is output from the scene modification unit 423 (i.e., the simulated scene generation unit 420) and input to the experience-based device 500 (see FIG. 1). The simulated scene data 431 is composed of the same information types as the simulated space data 200.

[0100] (Specific example of operation of the scene modification unit 423) 10 is a diagram showing a specific example of the operation of the scene modification section 423 in the simulated scene generation section 420. FIGS. 1 and 8 will be referred to as appropriate.

[0101] In the example shown in FIG. 10, the base scene 711 depicts a scene in which a subject P1 driving a truck T makes a left turn at an intersection. It is assumed that the gaze position data 338 included in the subject behavior data 330a indicates that the subject P1 is not checking the left rear using a side mirror or the like. Furthermore, it is assumed that the selected improvement factor (selected improvement factor data 416) is not checking the left rear. In this case, the scene modification unit 423 emphasizes the improvement factor of the subject P1. Specifically, the scene modification unit 423 generates a simulated scene 712 in which an added / modified model M of a two-wheeled vehicle traveling straight ahead to the left rear of the truck T is placed.

[0102] The subject P1 experiences a re-created scene of the simulated scene 712 in the experience device 500. This experience enables the subject P1 to recognize that if this experience were to be carried out in a scene where the subject P1 is driving on a daily basis, there is a possibility that the subject P1 may suddenly face the risk of being involved in a motorcycle accident. By presenting this simulated scene 712 to the subject P1, it is possible to provide the subject P1 with more effective support for improving his or her behavior than would be possible with a general experience device 500 such as that shown in Patent Document 1.

[0103] Other examples of behavior improvement highlighting scenes include the following examples (C1) to (C4). (C1) If the subject P1 does not stop at the stop line at an intersection, an added / modified model M of a motorcycle coming from the blind spot of the intersection is added to the base scene 711. (C2) If a vehicle is traveling with a large bulge when turning left, an added / modified model M of another vehicle in the oncoming lane is added to the base scene 711. (C3) If visual safety confirmation is insufficient when changing lanes, an added / modified model M of a vehicle traveling at a high speed behind the vehicle in the changing lane is added to the base scene 711. (C4) If surrounding safety confirmation is insufficient when parking, an added / modified model M of a pedestrian is added to the base scene 711 at the location where confirmation was insufficient. Behavior improvement highlighting scenes for other improvement factors may also be generated.

[0104] Furthermore, the scene modification unit 423a generates and outputs simulated scene data 431a by superimposing data relating to the score data 417 on the simulated scene data 431 (see FIG. 8).

[0105] In this way, the scene modification unit 423 generates and outputs simulated scene data 431, which is data of a simulated scene 712 in which an added / modified model M, which is a model corresponding to the improvement factor, is added to a base scene 711. Furthermore, by including the added / modified model M, the simulated scene data 431 includes data related to the improvement factor.

[0106] (Experience Device 500) Fig. 11 is a functional block diagram of the experience device 500 according to this embodiment. Fig. 12 is a flowchart showing the procedure of the operation performed by the experience device 500. The operation performed by the experience device 500 will be described with reference to Figs. 11 and 12.

[0107] The experience device 500 includes a display processing unit 501 , a feedback data generating unit 502 , and a setting unit 503 .

[0108] The display processing unit 501 displays the simulated scene data 431 input from the simulated scene generating device 400 on the display device 514 (see FIG. 14). At this time, the display processing unit 501 performs display processing to display the simulated scene 712 (see FIG. 10) on the display device 514 (see FIG. 14) as the display screen 900 shown in FIG. 15 or 16 (S301 in FIG. 12).

[0109] Next, the feedback data generation unit 502 generates the output of the sensor 511 shown in FIG. 14 as feedback data 541 and outputs the generated feedback data 541 (S302 in FIG. 12). The feedback data 541 is composed of information collected by the experience device 500 when the subject P1 experiences / operates the experience device 500. The feedback data 541 includes, for example, acceleration / deceleration amount data 521, steering amount data 522, gaze position data 523, etc. shown in FIG. 14. The feedback data 541 may include each data item such as vehicle speed data 331 to gaze position data 338, similar to the behavior data 330 shown in FIG. 4. As shown in FIG. 1, the output feedback data 541 is input to the subject behavior data 330a. This updates the subject behavior data 330a. This update can affect the scene that will be output as the simulated scene 712 (see FIG. 10) the next time the simulated scene generation device 400 is executed. In other words, if the subject P1 has completed the tasks in the experience device 500, the simulated scene generation device 400 will extract factors for improvement excluding the tasks that have been completed.

[0110] In this way, the feedback data generation unit 502 generates the output of the sensor 511 that acquires motion information of the subject P1, who is the person operating the experience device 500, as feedback data 541. Then, the feedback data generation unit 502 outputs the generated feedback data 541 and reflects it in the subject behavior data 330a.

[0111] The setting unit 503 sets setting data 524 (see FIG. 14) sent from the simulated scene generating device 400. The setting data 524 is information about the type of vehicle driven by the subject P1, the size of the vehicle, the positions and sizes of the rearview mirror and side mirrors, the size of the windshield and rear window, etc.

[0112] (Hardware configuration diagram of simulated scene generation device 400) Fig. 13 is a diagram showing the hardware configuration of a simulated scene generation device 400 according to this embodiment. In Fig. 13, the same components as those shown in Figs. 8 and 10 are denoted by the same reference numerals, and their description will be omitted.

[0113] The simulated scene generating device 400 includes a calculation device 441 , a storage device 442 , a RAM (Random Access Memory) 443 , and a communication device 444 .

[0114] The arithmetic device 441 is configured with a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), an FPGA, etc. The arithmetic device 441 operates as an execution unit of the simulated scene generating device 400 by reading and executing various programs, etc. from the storage device 442.

[0115] The RAM 443 is a readable / writable storage area and operates as the main storage device of the simulated scene generating device 400. The RAM 443 stores the selected improvement factor data 416, the base scene data 422, the simulated scene data 431, the steering angle data 332, the acceleration data 333, the gaze position data 338, the setting data 524, and the like. The steering angle data 332, the acceleration data 333, and the gaze position data 338 are data included in the subject person behavior data 330a and the model person behavior data 330b. While the example shown in FIG. 4 shows the steering angle data 332, the acceleration data 333, and the gaze position data 338 included in the subject person behavior data 330a, other data included in the subject person behavior data 330a are also stored in the RAM 443. That is, the vehicle speed data 331, the yaw rate data 334, the merging timing data 337 (all of which refer to FIG. 4) included in the subject person behavior data 330a are also stored in the RAM 443. 13, the vehicle speed data 331 to the gaze position data 338 included in the model person behavior data 330b are also stored in the RAM 443. Although not shown in FIG.

[0116] As described above, the setting data 524 stores information about the type of vehicle driven by the subject P1, the size of the vehicle, the positions and sizes of the rearview mirror and side mirrors, the sizes of the windshield and rear window, etc. The setting data 524 is data that is input in advance by a designer.

[0117] The storage device 442 is a storage area configured from a hard disk (HD), a solid state drive (SSD), a read-only memory (ROM), etc. The storage device 442 stores programs for the improvement factor extraction unit 410 and the simulated scene generation unit 420. The storage device 442 also stores programs for the traffic rule compliance evaluation unit 411a to the operation skill evaluation unit 411d (evaluation unit 411) that configure the improvement factor extraction unit 410, and a program for the improvement factor selection unit 415. The storage device 442 also stores programs for the base scene selection unit 421 and the scene modification unit 423 that configure the simulated scene generation unit 420. These programs are loaded into the RAM 443 and executed by the calculation device 441.

[0118] In this way, the arithmetic unit 441 reads and executes the program deployed in the RAM 443. This embodies the traffic rule compliance evaluation unit 411a to the improvement factor selection unit 415 that constitute the improvement factor extraction unit 410, and the base scene selection unit 421 and scene modification unit 423 that constitute the simulated scene generation unit 420.

[0119] The communication device 444 reads the simulated space data 200, the subject behavior data 330a, and the model behavior data 330b, and communicates with the experience device 500.

[0120] (Hardware configuration of the experience device 500) Fig. 14 is a diagram showing the hardware configuration of the experience device 500 according to this embodiment. In Fig. 14, the same components as those shown in Fig. 11 are given the same reference numerals, and the description thereof will be omitted.

[0121] The experience device 500 includes a sensor 511, a display device 514, a computing device 515, an input interface 516, and a communication device 517. The sensor 511 acquires motion information of the person operating the experience device 500. The sensor 511 also includes an acceleration / deceleration sensor 511a, a steering angle sensor 511b, a gaze sensor 511c, etc. The experience device 500 is connected to the simulated scene generation device 400 and the subject behavior data 330a by signal lines, including wireless signals.

[0122] Acceleration / deceleration sensor 511a observes the amount of acceleration / deceleration in experience device 500 when input interface 516, which will be described later, is operated. The amount of acceleration / deceleration is the amount of acceleration / deceleration of a vehicle being simulated by experience device 500.

[0123] Steering angle sensor 511b observes the amount of steering angle in experience device 500 when input interface 516, which will be described later, is operated. The amount of steering angle is the steering angle of the vehicle being simulated by experience device 500.

[0124] The gaze sensor 511c uses the experience device 500 to observe the gaze direction of the subject P1 who is driving a vehicle in a simulated manner.

[0125] The display device 514 is configured with, for example, VR goggles or a monitor, and reproduces and displays the simulated scene 712 (see FIG. 10). Alternatively, the exemplary driving of the model driver P2 may be displayed on the display device 514 by devising a display method.

[0126] The arithmetic device 515 is configured with a CPU, GPU, FPGA, etc. The arithmetic device 515 operates as an execution unit of the experience device 500 by reading and executing various programs, etc. from the ROM 531.

[0127] The input interface 516 has a shape that resembles, for example, a steering wheel, an accelerator, a brake, etc. When the subject P1 operates the input interface 516, the operation of the subject P1 is transmitted to the experience device 500.

[0128] The communication device 517 reads the simulated scene data 431 from the simulated scene generating device 400 and transmits feedback data 541 to the subject behavior data 330a.

[0129] Furthermore, the experience device 500 is provided with a ROM 531 and a RAM 520. The ROM 531 stores programs for a display processing unit 501, a feedback data generating unit 502, and a setting unit 503.

[0130] The RAM 520 stores acceleration / deceleration amount data 521, steering amount data 522, gaze position data 523, etc. The acceleration / deceleration amount data 521 is data acquired from the acceleration / deceleration sensor 511a. The steering amount data 522 is data acquired from the steering angle sensor 511b. The gaze position data 523 is data acquired from the line-of-sight sensor 511c.

[0131] The acceleration / deceleration amount data 521, steering amount data 522, gaze position data 523, etc. are reflected in the subject person behavior data 330a as feedback data 541 (see FIG. 11). For example, the acceleration / deceleration amount data 521 is reflected in acceleration data 333 (see FIG. 4) of the subject person behavior data 330a. Furthermore, the steering amount data 522 is reflected in steering angle data 332 (see FIG. 4) of the subject person behavior data 330a. Furthermore, the gaze position data 523 is reflected in gaze position data 338 of the subject person behavior data 330a.

[0132] Incidentally, the process of reflecting the feedback data 541 in the subject behavior data 330a is performed, for example, by a database server (not shown) or the like that holds the subject behavior data 330a to reflect the feedback data 541 in the subject behavior data 330a. Alternatively, if the subject behavior data 330a is held in the simulated scene generation device 400, the simulated scene generation device 400 performs the process of reflecting the feedback data 541 in the subject behavior data 330a.

[0133] 14, acceleration / deceleration sensor 511a, steering angle sensor 511b, and gaze sensor 511c are shown as sensors 511, but other sensors 511 may be mounted on experience device 500. Also, in the example shown in FIG. 14, acceleration / deceleration amount data 521, steering amount data 522, and gaze position data 523 are shown as data acquired from sensor 511, but other data may be acquired from sensor 511.

[0134] The RAM 520 also stores setting data 524. The setting data 524 is similar to that shown in Fig. 13, and therefore a description thereof will be omitted here. The subject P1 can adjust the setting data 524 via the input interface 516. This allows the subject P1 to select a preferred vehicle model from a plurality of vehicle models, adjust the position of the rearview mirror, side mirrors, etc.

[0135] Furthermore, the program stored in the ROM 531 is executed by the arithmetic unit 515 .

[0136] In this way, the arithmetic unit 525 reads and executes the program stored in the ROM 531. As a result, the display processing unit 501, the feedback data generating unit 502, and the setting unit 503 are realized.

[0137] (Display screen 900) 15 is a diagram showing an example of a display screen 900 in this embodiment, with reference to FIGS. 1, 4, and 14 as appropriate. The display screen 900 shown in Fig. 15 is displayed on the display device 514 (VR goggles or monitor) provided in the experience device 500. Such a display screen 900 is displayed based on the simulated scene data 431. In this way, the simulated scene 712 (see Fig. 10) is used to virtually display a scene that the subject P1 may view. Therefore, the simulated scene data 431 is data related to the simulated scene 712 for virtually displaying a scene that the subject P1 may view.

[0138] In the example shown in Fig. 15, an added / modified model M of a motorcycle traveling to the rear left is displayed on the rearview mirror 911. This added / modified model M has been added by the scene modification unit 423 (Fig. 8). The added / modified model M is data relating to improvement factors included in the simulated scene data 431. In other words, the display processing unit 501 displays the added / modified model M, thereby displaying the data relating to improvement factors included in the simulated scene data 431 on the display device 514.

[0139] In addition to the added / corrected model M, the display screen 900 displays an exemplary gaze position 921 by the model model P2 and a gaze position 922 during the experience by the subject P1 who is receiving behavior improvement. The gaze positions 921 and 922 enable the subject P1 to recognize any deviation from the ideal gaze position 921 to which he or she should direct his or her gaze.

[0140] Also displayed on the display screen 900 are a model steering angle 923 by the model person P2 and a steering angle 924 being experienced by the subject P1. By displaying the model steering angle 923 and the steering angle 924 being experienced, the subject P1 can recognize the deviation from the ideal model steering angle 923 that should be used for steering.

[0141] Then, the display screen 900 displays a model braking amount 925 by the model person P2 and a braking amount 926 currently being experienced by the subject P1. By displaying the model braking amount 925 and the braking amount currently being experienced 926, the subject P1 can recognize the deviation from the ideal model braking amount 925 that should be applied.

[0142] Furthermore, a model acceleration amount 927 by the model person P2 and an acceleration amount 928 currently being experienced by the subject P1 are displayed on the display screen 900. By displaying the model acceleration amount 927 and the acceleration amount 928 currently being experienced, the subject P1 can recognize the deviation from the ideal amount at which acceleration control should be performed.

[0143] 15. Furthermore, the display screen 900 displays an exemplary model driving path 931 by the model driver P2. Furthermore, the display screen 900 displays a space 941 for displaying characters, graphs, etc. For example, the space 941 displays the safety confirmation level evaluation score of the subject P1 as a numerical value. Alternatively, the space 941 may display an action improvement plan generated based on the feedback data 541. Alternatively, the space 941 may display a time series graph of the safety confirmation level evaluation score or a score other than the feedback data 541. Furthermore, the display content and display method of the display screen 900 are not limited to those shown in FIG. 15.

[0144] The exemplary gaze position 921, the exemplary steering angle 923, the exemplary braking amount 925, the exemplary acceleration amount 927, and the exemplary driving path 931 are based on the exemplary driver behavior data 330b included in the score data 417. Specifically, the exemplary gaze position 921 is based on the gaze position data 338 of the exemplary driver behavior data 330b, and the exemplary steering angle 923 is based on the steering angle data 332 of the exemplary driver behavior data 330b. The exemplary braking amount 925 and the exemplary acceleration amount 927 are based on the acceleration data 333 of the exemplary driver behavior data 330b. The exemplary driving path 931 is based on the driving path data 335 of the exemplary driver behavior data 330b.

[0145] Furthermore, gaze position 922, steering angle amount 924, braking amount 926, and acceleration amount 928 during the experience are displayed based on data acquired from sensor 511 provided in experience device 500. Specifically, gaze position 922 during the experience is displayed based on gaze position data 523 acquired by line-of-sight sensor 511c. Steering angle amount 924 during the experience is displayed based on steering amount data 522 acquired by steering angle sensor 511b. Furthermore, braking amount 926 and acceleration amount 928 during the experience are displayed based on acceleration / deceleration amount data 521 acquired by acceleration / deceleration sensor 511a.

[0146] The display of the exemplary gaze position 921 to the exemplary driving trajectory 931 is determined by the scene modification unit 423 (see FIG. 8) based on the score data 417. Specifically, the scene modification unit 423 determines the content to be displayed on the display screen 900 based on the safety confirmation level evaluation score, safety margin evaluation score, and operation skill evaluation score included in the score data 417. In the example shown in FIG. 15, the exemplary gaze position 921 to the exemplary driving trajectory 931 are displayed because the following time series scores are equal to or less than predetermined values: (D1) Time series score of gaze position included in the safety confirmation level evaluation score. (D2) Acceleration, driving trajectory, and steering angle included in the operation skill evaluation score.

[0147] It is possible to omit displaying the exemplary gaze position 921 to the exemplary driving trajectory 931. In this case, the improvement factor selection unit 415 (see FIG. 6) does not generate or output the score data 417. Alternatively, it may be possible to switch between displaying and hiding the exemplary gaze position 921 to the exemplary driving trajectory 931.

[0148] In this way, by displaying the exemplary gaze position 921 to the exemplary driving trajectory 931, the difference in behavior between the subject P1 who is the target of skill improvement and the exemplary driver P2 is displayed on the display device 514. As described above, the gaze position 922, steering angle amount 924, braking amount 926, and acceleration amount 928 during the experience are based on the operation data of the experience device 500. In addition, the exemplary gaze position 921, model steering angle amount 923, model braking amount 925, model acceleration amount 927, and model driving trajectory 931 are based on the exemplary driver behavior data 330b.

[0149] (Warning display 912) FIG. 16 is a diagram showing an example of an attention-calling display 912 in this embodiment. In FIG. 16, the same components as those in FIG. 15 are denoted by the same reference numerals, and the description thereof will be omitted. 16, an attention-calling display 912 is displayed for the added / modified model M. The attention-calling display 912 is displayed based on data relating to improvement factors included in the simulated scene data 431.

[0150] According to this embodiment, a simulated scene 712 that emphasizes factors for improvement according to the characteristics of the subject P1 can be generated in a scene in which the subject P1 normally drives, and the subject P1 can experience the simulated scene 712. This allows the subject P1, who is an inexperienced person, to efficiently improve his or her skills.

[0151] Then, the improvement factor extraction unit 410 selects improvement factors based on the scores. This makes it possible to quantitatively select improvement factors. In addition, the scores are time-series data (time-series scores), and the simulated space with the lowest time-series score is selected as the base scene 711 (see FIG. 10). This makes it possible to select the scene that the subject P1 should pay the most attention to as the base scene 711.

[0152] Furthermore, the attention-calling display 912 as shown in FIG. 16 is displayed, so that the subject P1 can check the points that he or she should be careful about.

[0153] 15, the exemplary gaze position 921 to the exemplary driving path 931 are displayed, so that the subject P1 can visually recognize the difference between the current operation of the experience device 500 and the driving by the exemplary driver P2. This can be expected to improve the technique of the subject P1.

[0154] <Modification> Next, a modified example of the present invention will be described with reference to FIG.

[0155] In this modified example, a case where the model data 425 is created as an AI model will be described. Note that the modified example differs from the present embodiment in the operation of the simulated scene generation unit 420, and therefore, in this modified example, the operation of the simulated scene generation unit 420 will be described.

[0156] Fig. 17 is a flowchart showing the procedure of the operation performed by the simulated scene generating section 420 in the modified example. Fig. 8 will be referred to as appropriate. In Fig. 20, the same processes as those in Fig. 9 are denoted by the same reference numerals.

[0157] In the modified example, the model data 425 is learned as an AI model using a plurality of pieces of simulated space data 200 and a plurality of pieces of behavior data 330 as input. That is, the model data 425 is an AI model that is the result of learning by AI. In this case, the model data 425 is constructed by deep learning or the like, in which a correspondence relationship between improvement factors and the corresponding addition / modification model M (see FIG. 10) is established.

[0158] First, the base scene selection unit 421 selects a base scene 711 (see FIG. 10) using the simulated space data 200 and the selected improvement factor data 416 (S201).

[0159] Then, the selected improvement factor data 416, the base scene data 422, and the simulated space data 200 are input to the improvement factor emphasis model adding / modifying unit 424 (S211). As a result, the improvement factor emphasis model adding / modifying unit 424 outputs an adding / modifying model M to be added to the base scene 711 (see FIG. 10) based on the AI ​​model (S212).

[0160] Then, an appropriate added / modified model M is added to the base scene data 422 (S202).

[0161] According to this modification, the model data 425 is an AI model, so that the effort required to create the model data 425 can be reduced.

[0162] It should be noted that the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the described configurations. Other embodiments conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention.

[0163] The simulated scene generation device 400 and the experience device 500 shown in FIG. 1 may also include an input / output interface (not shown). In this case, the programs for implementing the simulated scene generation device 400 and the experience device 500 may be loaded from another device via an available medium via the input / output interface as needed. Here, the medium refers to, for example, a storage medium detachable from the input / output interface, or a communication medium, such as a wired, wireless, or optical network, or a carrier wave or digital signal propagating through the network. Some or all of the functions implemented by the programs may be implemented by hardware circuits or FPGAs. Furthermore, each of the above configurations, functions, etc. may be implemented by software, with a processor interpreting and executing a program that implements each function. Information such as programs, tables, and files that implement each function may be stored in a memory, a storage device such as a hard disk or SSD, or a storage medium such as an IC (Integrated Circuit) card, a Secure Digital (SD) card, or a Digital Versatile Disc (DVD).

[0164] In this embodiment, the behavior improvement system is configured to contribute to the improvement of driving skills, but is not limited to this. For example, the behavior improvement system may be configured to contribute to the improvement of food delivery in restaurants and the like. In such cases, the actions of the model and target are recorded as corresponding to driving data using footage from action cameras attached to the model and target, or cameras installed inside the restaurant. The subsequent processing is similar to that shown in this embodiment. When the behavior improvement system of this embodiment is applied to food delivery, the score calculated by the improvement factor extraction unit 410 may include the time required for food delivery, the distance of the delivery route, the distance from the customer, etc.

[0165] In addition, in each embodiment, the control lines and information lines shown are those that are considered necessary for explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are interconnected. [Explanation of symbols]

[0166] 1. Motion improvement support system 100 Simulated space generation device 101 Sensor data acquisition unit 102 Map data acquisition unit 103 Simulated space generation section 111 Sensor Data 112 Map Data 200 simulated space data (including simulated space) 330 Behavioral Data 330a Subject behavior data 330b Exemplary Behavioral Data 400 Simulated Scene Generator 410 Improvement factor extraction unit 411 Evaluation Department 411a Traffic Rules Compliance Assessment Department 411b Safety Verification Assessment Department 411c Safety Margin Evaluation Section 411d Operation Skills Evaluation Department 415 Improvement Factor Selection Department 416 Selection improvement factor data (including improvement factors) 417 Score Data 420 Simulated Scene Generation Unit 421 Base Scene Selection Section 422 Base Scene Data 423 Scene Modification Department 424 Improvement Factor Emphasis Model Addition / Modification Section 425 model data (including additional / modified models) 431 Simulated Scene Data 500 Experience Device 501 Display processing unit 502 Feedback data generation unit 503 Settings 511 Sensors 541 Feedback Data 711 Bass Scene 712 Mock Scene 900 display screen 912 Warning display 921 Gaze Position (Based on Model Behavior Data) 922 Gaze position (based on data from the operation of the experience device) 923 Model steering angle (based on model driver behavior data) 924 Steering angle (based on data from the operation of the experience device) 925 Model braking amount (based on model driver behavior data) 926 Braking amount (based on data from the operation of the experience device) 927 Model Acceleration Amount (Based on Model Behavior Data) 928 Acceleration (based on data from the operation of the experience device) 931 Model driving trajectory (based on model behavior data) 941 Space M Addition / Modification Model P1 Target P2 Role model

Claims

1. an improvement factor selection unit that selects improvement factors for the behavior of the subject by at least comparing subject behavior data, which is data on the behavior of the subject that is the target of technical improvement, with exemplary person behavior data, which is data on the behavior of an exemplary person, and outputs selected improvement factor data, which is data on the improvement factors; a base scene selection unit that selects a simulated space based on the selected improvement factor data as a base scene; a scene modification unit that generates and outputs simulated scene data, which is data of a simulated scene obtained by adding an addition / modification model, which is a model corresponding to the improvement factor, to the base scene; Equipped with The simulated space is the subject's behavior space. Simulated scene generator.

2. an evaluation unit that calculates a score for the behavior of the subject by comparing the subject behavior data with the model subject behavior data; the evaluation unit calculates the score for each of a plurality of behaviors of the subject, The improvement factor selection unit Selecting a behavior of the subject as the improvement factor from among the plurality of behaviors based on the score.

2. The simulated scene generating device according to claim 1.

3. The simulated scene data includes data on improvement factors.

2. The simulated scene generating device according to claim 1.

4. The simulated scene is a virtual display of a scene that the subject may see.

2. The simulated scene generating device according to claim 1.

5. The subject behavior data is data on behavior of the subject when driving, and the model behavior data is data on behavior of the model when driving.

2. The simulated scene generating device according to claim 1.

6. a display processing unit that displays, on a display device, simulated scene data, which is data relating to a simulated scene for virtually displaying a scene that may be visually recognized by a subject person who is a target of technical improvement; The display processing unit Displaying data relating to improvement factors included in the simulated scene data on a display device Experience device.

7. a feedback data generating section that generates feedback data from an output of a sensor that acquires motion information of a person operating the experience device and outputs the generated feedback data; 7. The experience device according to claim 6, further comprising:

8. The display processing unit Based on the data on the operation of the experience device and the behavioral data of the model, differences in behavior between the subject who is the target of technical improvement and the model are displayed on the display device; The model behavior data is data related to the model's behavior.

7. The experience device according to claim 6.

9. an improvement factor selection unit that selects improvement factors for the behavior of the subject by at least comparing subject behavior data, which is data on the behavior of the subject that is the target of technical improvement, with exemplary person behavior data, which is data on the behavior of an exemplary person, and outputs selected improvement factor data, which is data on the improvement factors; a base scene selection unit that selects a simulated space based on the selected improvement factor data as a base scene; a scene modification unit that generates and outputs simulated scene data, which is data of a simulated scene obtained by adding an addition / modification model, which is a model corresponding to the improvement factor, to the base scene; and a simulated scene generating device comprising: A motion improvement support system having an experience device equipped with a display processing unit that displays the simulated scene data on a display device, The simulated space is a behavioral space of the subject, The display processing unit Displaying data relating to improvement factors included in the simulated scene data on a display device Behavior improvement support system.

10. The experience device includes: a feedback data generation section that generates feedback data from an output of a sensor that acquires motion information of a person operating the experience device, and reflects the generated feedback data in the subject behavior data; 10. The behavior improvement support system according to claim 9, further comprising:

Images