Simulator

Abstract

This simulator is suitable for objectively evaluating consideration for others. [Solution] The driving simulator 10 acquires sensor information regarding the actions of the person being evaluated, controls the behavior of the simulated vehicle in the virtual space based on the acquired sensor information, and controls the behavior of NPC objects according to the behavior or state of the simulated vehicle. It then determines the behavior of the NPC objects according to the behavior or state of the simulated vehicle and evaluates the actions of the person being evaluated based on the determination result. This makes it possible to evaluate "consideration for others," which was difficult to evaluate in conventional simulators that only used the actions of the person being evaluated and the state of the simulated vehicle as evaluation criteria, based on the objective fact of "what actions others were forced to take." Therefore, consideration for others can be evaluated more objectively than in conventional systems.

Description

Technical Field

[0001] The present invention relates to a simulator for assisting education, training, research, development, testing, etc., and particularly to a simulator suitable for objectively evaluating consideration for others.

Background Art

[0002] Conventionally, in order to evaluate the driving of an evaluation subject with respect to a moving body such as an automobile, a driving simulator using a virtual space has been widely used. The driving simulator simulates the behavior of the moving body based on the operation of the evaluation subject, and evaluates the driving based on the operation information of the evaluation subject (such as the operation of the brake and accelerator) or the state information of the moving body (such as position and speed). For example, the driving is evaluated using evaluation items such as the presence or absence of violation of laws and regulations (such as ignoring signals and speeding), the presence or absence of accidents (such as collisions and wheel detachment), the driving state (such as inter-vehicle distance and swaying), or the operation state (such as brake reaction time).

[0003] On the other hand, as an evaluation technique using a simulation environment, for example, the technique described in Patent Document 1 is known. The technique described in Patent Document 1 relates to a method of evaluating AI in an electronic game environment by driving a vehicle object that has a vehicle model and a driving support function (AI (Artificial Intelligence)) together with at least one other vehicle object controlled by a person, and based on the behavior of the vehicle object that automatically drives in response to the behavior of the other vehicle object.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In recent years, there has been an increase in mobile vehicles that travel on sidewalks and operate in close proximity to pedestrians and other people, such as mobility scooters (electric wheelchairs) and automated delivery robots. For these types of mobile vehicles, it is important not only to comply with laws and regulations and avoid accidents, but also to show "consideration" for others around them. Consideration refers to actions and attitudes taken to anticipate dangers and prevent accidents and troubles, as well as actions and attitudes taken to avoid causing psychological burdens such as fear or anxiety to others, or to avoid hindering the smooth movement of others.

[0006] However, conventional driving simulators used evaluation methods that relied on the subject's operation information or the state information of the moving object as evaluation criteria, making it difficult to objectively evaluate consideration. For example, driving behaviors such as "not slowing down despite oncoming traffic on a narrow road and forcing the other party to yield" or "overtaking a pedestrian at high speed at close range" were not detected as problems in conventional driving simulators as long as they did not result in an accident. However, these are "inconsiderate driving" behaviors that cause fear and anxiety to others and involve potential dangers and nuisances. Furthermore, there was a lack of a mechanism to appropriately evaluate proactive considerate behaviors such as "actively stopping and yielding on a narrow road."

[0007] On the other hand, the technology described in Patent Document 1 is intended to verify the performance of AI and does not evaluate the "consideration" of the human subject being evaluated. The technology described in Patent Document 1 evaluates how the AI ​​reacts to influences from its surroundings (passive evaluation). In contrast, Patent Document 1 does not disclose the perspective of evaluating the consideration of the subject being evaluated based on the behavior of others who are not the subject being evaluated, that is, how the subject's driving affects its surroundings (active evaluation).

[0008] Therefore, the present invention has been made in view of the unresolved problems of the conventional technology, and aims to provide a simulator suitable for objectively evaluating consideration for others. [Means for solving the problem]

[0009] [Invention 1] To achieve the above objective, the simulator of Invention 1 comprises: operation information acquisition means for acquiring operation information relating to the operations of the person being evaluated; control means for controlling the behavior of a first object in a virtual space based on the operation information acquired by the operation information acquisition means and controlling the behavior of a second object according to the behavior or state of the first object; determination means for determining the behavior of the second object according to the behavior or state of the first object; and evaluation means for evaluating the operations of the person being evaluated based on the determination result of the determination means.

[0010] In this configuration, operation information is acquired by the operation information acquisition means, and the behavior of the first object is controlled by the control means based on the acquired operation information. The behavior of the second object is also controlled by the control means according to the behavior or state of the first object. The behavior of the second object is determined by the determination means according to the behavior or state of the first object, and the operation of the person being evaluated is evaluated by the evaluation means based on the determination result.

[0011] Here, the first object is an object whose behavior is controlled based on the actions of the person being evaluated, and can be configured as, for example, a vehicle, a pedestrian, a wheelchair user, an animal, a mobile robot, or other mobile body.

[0012] Furthermore, the second object is an object whose behavior is controlled according to the behavior or state of the first object, and can be configured as, for example, a vehicle, pedestrian, wheelchair user, animal, mobile robot, or other mobile entity. The second object can be configured as, for example, an NPC (Non-Player Character), and one or more of them can exist in the virtual space.

[0013] Furthermore, the means for acquiring operation information may, for example, input operation information from an input device, acquire or receive operation information from an external terminal, read operation information from a storage device or storage medium, or generate or calculate operation information through information processing. Therefore, acquisition includes at least input, acquisition, reception, reading (including retrieval), generation, and calculation. The concept of acquisition remains the same hereafter.

[0014] Furthermore, the means for acquiring operation information can acquire operation information from input devices operated by the person being evaluated (e.g., steering wheel, accelerator lever, accelerator pedal, brake lever, brake pedal, joystick, controller, keyboard, mouse, touch panel, or biometric information detection devices such as eye-tracking devices or electroencephalogram (EEG) measurement devices). The operation information may include, for example, information regarding whether or not an operation was performed on the input device, the amount of operation (e.g., accelerator opening, brake force, steering angle), operation speed, operation timing, or combination of operations (e.g., whether or not there was an intention to pass).

[0015] Furthermore, determination by determination means includes, for example, rule-based determination based on predetermined determination rules or criteria, determination by comparison with a predetermined threshold, or determination using a machine learning model.

[0016] Furthermore, evaluation methods may include, for example, scoring (scoring, adding or subtracting points) based on the judgment result, graded evaluation (e.g., excellent, good, fair), or generating feedback information for the person being evaluated (e.g., displaying evaluation results, voice guidance, report output). The evaluation may be performed using this simulator alone (consideration evaluation), or it may be performed in combination with evaluation using conventional evaluation items (e.g., presence or absence of legal violations, presence or absence of accidents, driving conditions or operating conditions).

[0017] Furthermore, this simulator may be implemented as a single device, apparatus, terminal, or other device, or as a network system in which multiple devices, apparatus, terminals, or other devices are connected in a communicative manner. In the latter case, each component may belong to any of the multiple devices, as long as they are connected in a communicative manner.

[0018] [Invention 2] Furthermore, the simulator of Invention 2 is the simulator of Invention 1, wherein the control means controls the behavior of the second object to be a predetermined reaction behavior according to the behavior or state of the first object, and the behavior information acquisition means acquires behavior information relating to the behavior of the second object, and the determination means determines whether the behavior of the second object is the reaction behavior based on the behavior information acquired by the behavior information acquisition means.

[0019] In this configuration, the control means controls the behavior of the second object so that it exhibits a predetermined reaction behavior according to the behavior or state of the first object. Then, the behavior information acquisition means acquires behavior information, and the determination means determines, based on the acquired behavior information, whether the behavior of the second object is a reaction behavior.

[0020] Here, the reaction behavior includes, for example, specific behaviors exhibited by the second object as a result of the behavior or state of the first object.

[0021] [Invention 3] Furthermore, the simulator of Invention 3, in the simulator of Invention 2, the reaction behavior includes the second object avoiding the first object, the second object stopping, or the second object moving backward or forward relative to the first object.

[0022] In this configuration, the determination means determines whether the second object has avoided, stopped, retreated, or advanced.

[0023] [Invention 4] Further, in the simulator of Invention 4, in the simulator of Invention 2, the reaction behavior includes that the second object operates in response to the operation of the first object, the second object operates in response to the non - operation of the first object, the second object does not operate in response to the operation of the first object, and the second object does not operate in response to the non - operation of the first object.

[0024] With such a configuration, the determination means determines that the second object operates or does not operate according to the operation or non - operation of the first object.

[0025] [Invention 5] Further, in the simulator of Invention 5, in the simulator of any one of Inventions 2 to 4, it includes a first state information acquisition means for acquiring state information regarding the state of the first object and a second state information acquisition means for acquiring state information regarding the state of the second object. The determination means determines whether a predetermined condition including that the behavior of the second object is the reaction behavior is satisfied based on the behavior information acquired by the behavior information acquisition means, the state information acquired by the first state information acquisition means, and the state information acquired by the second state information acquisition means.

[0026] With such a configuration, the first state information acquisition means and the second state information acquisition means respectively acquire state information, and the determination means determines whether a predetermined condition is satisfied based on the acquired behavior information and state information.

[0027] Here, the state information includes, for example, information regarding the position, velocity, acceleration, direction, posture, distance or relative velocity between objects, or the attributes of the objects (for example, age group, attention state).

[0028] [Invention 6] Furthermore, in the simulator of Invention 6, the determination means determines whether the predetermined conditions are met based on the behavior information acquired by the behavior information acquisition means, the state information acquired by the first state information acquisition means, the state information acquired by the second state information acquisition means, and the operation information acquired by the operation information acquisition means.

[0029] In this configuration, state information is acquired by the first state information acquisition means and the second state information acquisition means, respectively, and the determination means determines whether predetermined conditions are met based on the acquired behavior information, state information, and operation information.

[0030] [Invention 7] Furthermore, the simulator of Invention 7 is the simulator of Invention 6, wherein the first state information acquisition means acquires state information relating to the position, velocity, or orientation of the first object, and the second state information acquisition means acquires state information relating to the position, velocity, or orientation of the second object, and the predetermined conditions include (1) the positional relationship between the first object and the second object is a predetermined positional relationship, (2) the velocity of the first object or the second object is a predetermined velocity, (3) the orientation of one of the first object and the second object relative to the other is a predetermined orientation, (4) the operation of the person being evaluated is a predetermined operation, or (5) the area in the virtual space where the first object or the second object is located is a predetermined area.

[0031] With this configuration, the determination means determines whether at least one of the predetermined conditions (1) to (5) is met.

[0032] Here, the predetermined positional relationship includes, for example, the distance between objects satisfying conditions such as being less than or equal to a predetermined value, greater than or equal to a predetermined value, within a predetermined range, outside a predetermined range, or other conditions.

[0033] Furthermore, the predetermined speed includes, for example, the object's speed being below a predetermined level, above a predetermined level, within a predetermined range, outside a predetermined range, or other conditions.

[0034] Furthermore, the predetermined orientation includes, for example, the angle formed by the front-to-back direction of one object and the front-to-back direction of the other object, the angle formed by the left-to-right direction of one object and the left-to-right direction of the other object, the angle formed by the up-to-down direction of one object and the up-to-down direction of the other object, or the angle formed by other combinations of these directions satisfying conditions such as being less than or equal to a predetermined value, greater than or equal to a predetermined value, within a predetermined range, outside a predetermined range, or other conditions.

[0035] Furthermore, the prescribed operations include, for example, the operation of the steering wheel, the operation of the accelerator, the operation of the brakes, the operation of auxiliary devices such as turn signals, and other operations, where the amount of operation is less than or equal to a predetermined value, greater than or equal to a predetermined value, within a predetermined range, outside a predetermined range, or other conditions.

[0036] Furthermore, designated areas include, for example, narrow sidewalks, pedestrian crossings, intersections, or shopping streets. [Effects of the Invention]

[0037] As explained above, the simulator of Invention 1 performs evaluations based on the behavior of the second object in response to the behavior or state of the first object, so it is possible to objectively evaluate consideration for others compared to conventional methods.

[0038] Furthermore, according to the simulator of Invention 2, the evaluation is based on whether the behavior of the second object is a predetermined reaction behavior, so consideration for others can be evaluated more objectively.

[0039] Furthermore, according to the simulator of Invention 3, it is possible to evaluate consideration for others when the second object avoids, stops, retreats, or moves forward in response to the behavior or state of the first object.

[0040] Furthermore, according to the simulator of Invention 4, it is possible to evaluate consideration for others when the second object operates or does not operate in response to the operation or non-operation of the first object.

[0041] Furthermore, according to the simulator of Invention 5, since the evaluation is performed based on the state of the first object and the state of the second object, it is possible to evaluate consideration for others in a variety of situations.

[0042] Furthermore, according to the simulator of Invention 6, since the evaluation is performed based on the state of the first object, the state of the second object, and the actions of the person being evaluated, it is possible to evaluate consideration for others in a wider variety of situations.

[0043] Furthermore, according to the simulator of Invention 7, the evaluation is performed based on whether at least one of the predetermined conditions (1) to (5) is met, so it is possible to evaluate consideration for others in a wider variety of situations. [Brief explanation of the drawing]

[0044] [Figure 1] This diagram shows the hardware configuration of the driving simulator 10. [Figure 2] This diagram shows the hardware configuration of the control device 100. [Figure 3] This figure shows the data structure of the object management table 400, behavior log table 402, status log table 404, and simulated vehicle log table 406. [Figure 4] This is a flowchart showing the control process. [Modes for carrying out the invention]

[0045] Embodiments of the present invention will be described below. Figures 1 to 4 show these embodiments. 〔overview〕 In this embodiment, the present invention will be explained using as an example the case in which it is applied to a driving simulator for evaluating the driving performance of a mobility scooter.

[0046] The driving simulator according to this embodiment is installed, for example, in medical institutions such as hospitals, research institutions such as universities, and public institutions such as police stations and driver's license testing centers, and is used by patients, visitors, etc. (hereinafter referred to as "evaluation subjects"). The driving simulator simulates the driving of a mobility scooter in a virtual space based on the driving operations of the evaluation subject, and focuses on how the evaluation subject's driving operations affect surrounding pedestrians. Based on the behavior of pedestrians, it objectively evaluates to what extent the evaluation subject was able to drive with consideration for pedestrians.

[0047] In the following explanation, actual mobility scooters will be referred to as "real vehicles," and mobility scooters in a virtual space will be referred to as "simulated vehicles."

[0048] 〔composition〕 First, the configuration of this embodiment will be described. Figure 1 shows the hardware configuration of the driving simulator 10.

[0049] As shown in Figure 1, the driving simulator 10 is configured to include a seat (not shown) in which the person being evaluated sits, a display 12 installed in front of the seat, an operating device 14 installed between the seat and the display 12, and a control device 100 that controls the entire system.

[0050] The display 12 shows a video of the scenery (simulated image) that would be seen from the simulated vehicle when it is driven in a virtual environment, based on the operation of the control device 14 by the person being evaluated. This allows the person being evaluated to experience simulated driving with the same feel as operating a real vehicle while looking at the screen of the display 12.

[0051] The control device 14 is configured similarly to the control system of an actual vehicle and consists of a steering wheel 16, an accelerator lever 18, and a brake lever 20. The accelerator lever 18 and the brake lever 20 are biased to return to their respective reference positions when the person being evaluated grips and releases them.

[0052] The operating device 14 is further configured to include a steering angle sensor 40 for detecting the steering angle of the steering wheel 16, an accelerator sensor 42 for detecting the amount of gripping (accelerator opening) of the accelerator lever 18, and a brake sensor 44 for detecting the amount of gripping (brake opening) of the brake lever 20.

[0053] The steering angle sensor 40 is composed of, for example, a potentiometer or a rotary encoder, and outputs a steering angle value corresponding to the steering angle of the steering wheel 16.

[0054] The accelerator sensor 42 outputs an accelerator lever value corresponding to the accelerator opening of the accelerator lever 18. The accelerator sensor 42 increases the accelerator lever value when the accelerator opening is large and decreases the accelerator lever value when the accelerator opening is small.

[0055] The brake sensor 44 outputs a brake lever value corresponding to the brake opening angle of the brake lever 20. The brake sensor 44 increases the brake lever value when the brake opening angle increases and decreases the brake lever value when the brake opening angle decreases.

[0056] [Control device 100] Next, the hardware configuration of the control device 100 will be described. Figure 2 shows the hardware configuration of the control device 100.

[0057] The control device 100 controls a driving simulation that plays back as a video the scenery visible from a simulated vehicle when the simulated vehicle is driven in a virtual environment, based on the operation of the steering wheel 16, etc., by the person being evaluated. The control device 100 can be configured as, for example, a desktop PC (Personal Computer), a notebook PC, or other computer.

[0058] As shown in Figure 2, the control device 100 consists of a CPU (Central Processing Unit) 30 that controls calculations and the entire system based on a control program, a ROM (Read Only Memory) 32 that stores the control program for the CPU 30 in a predetermined area, a RAM (Random Access Memory) 34 for storing data read from the ROM 32 and other memory, as well as calculation results necessary for the calculation process of the CPU 30, and an I / F (Interface) 38 that mediates data input and output to external devices. These components are connected to each other and enable data exchange via a bus 39, which is a signal line for data transfer.

[0059] The I / F38 is connected to external devices including a steering angle sensor 40, an accelerator sensor 42, a brake sensor 44, a display 12, and a storage device 46 for storing data, tables, etc., as files.

[0060] [Data structure] Next, we will explain the data structure of the storage device 46. The storage device 46 stores course data relating to a virtual environment including the course on which the simulated vehicle travels and various traffic conditions, object data such as 3D models that constitute objects to appear while the simulated vehicle is running, and event data relating to events that occur while the simulated vehicle is running (for example, "pedestrian suddenly appearing"). For example, the course data and training scene data described in Patent Document 2 (Japanese Patent Application Publication No. 2012-128375) can be used as the course data and event data.

[0061] Figure 3 shows the data structures of the object management table 400, behavior log table 402, status log table 404, and simulated vehicle log table 406.

[0062] As shown in Figure 3, the storage device 46 stores the object management table 400, the behavior log table 402, the status log table 404, and the simulated vehicle log table 406.

[0063] The object management table 400 is a table that registers object information about objects. As shown in Figure 3(a), the object management table 400 is composed of columns for registering an object identifier that uniquely identifies an object, a column for registering the name of the object, a column for registering the type of object, a column for registering an area tag that identifies the area where the object is located, a column for registering the file name of the object data, and columns for registering other information. The object identifier is the primary key.

[0064] Object types include NPC (Non-Player Character) objects, which act autonomously in the virtual space, and static objects, which are placed in a fixed position in the virtual space. Examples of NPC objects include pedestrians and vehicles. Examples of static objects include structures, buildings, sidewalks, roadways, traffic lights, and tunnels.

[0065] Area tags are registered for static objects, such as narrow sidewalks, pedestrian crossings, intersections, and shopping streets.

[0066] The behavior log table 402 is a table that registers behavioral information about the behavior of an object as behavior log information. As shown in Figure 3(b), the behavior log table 402 is composed of a column for registering the object identifier, a column for registering the object's behavior, and a column for registering other information. The object identifier is the primary key.

[0067] In terms of behavior, the NPC object registers behaviors in which the object reacts to the behavior or state of the simulated vehicle (hereinafter referred to as "reaction behavior"). Examples of reaction behaviors that may be registered include "avoidance" where the NPC object avoids the simulated vehicle, "stop" where the NPC object comes to a complete stop, "reverse" where the NPC object moves backward relative to the simulated vehicle, and "advance" where the NPC object moves forward relative to the simulated vehicle.

[0068] The status log table 404 is a table that registers status information about the state of an object as status log information. As shown in Figure 3(c), the status log table 404 is composed of columns for registering the object identifier, the object's position (coordinates in virtual space), the object's velocity, the object's acceleration, the object's orientation (direction vector), and other information. The object identifier is the primary key.

[0069] The state of NPC objects is calculated and registered using a predetermined behavior model (for example, a pedestrian behavior model of the Social Force Model) or an AI (Artificial Intelligence) engine, which determines their position, velocity, acceleration, and orientation.

[0070] The simulated vehicle log table 406 is a table that registers sensor information from sensors 40 to 44 and state information related to the state of the simulated vehicle. As shown in Figure 3(d), the simulated vehicle log table 406 is configured to have a column for registering a log ID that uniquely identifies the log information, a column for registering the acquisition time indicating the date and time the log was acquired, a column for registering the steering angle value of the steering angle sensor 40, a column for registering the accelerator lever value of the accelerator sensor 42, and a column for registering the brake lever value of the brake sensor 44. Furthermore, it is configured to have a column for registering the position (coordinates in virtual space) of the simulated vehicle, a column for registering the speed of the simulated vehicle, a column for registering the acceleration of the simulated vehicle, a column for registering the orientation (direction vector) of the simulated vehicle, and a column for registering other information. The log ID is the primary key.

[0071] The state of the simulated vehicle is calculated and registered based on steering angle, accelerator lever value, and brake lever value, including position, speed, acceleration, and direction. The simulated vehicle's state is registered not only as current values ​​but also in association with sensor information to maintain a history. The accelerator lever value is used to determine whether the person being evaluated intends to pass.

[0072] [Operation] Next, the operation of this embodiment will be described. The CPU 30 consists of an MPU (Micro-Processing Unit) and the like, and it starts a predetermined program stored in a predetermined area of ​​the ROM 32, and executes the control processing shown in the flowchart of Figure 4 according to that program.

[0073] Figure 4 is a flowchart showing the control process. The control process is the process that controls the entire driving simulation, and when it is executed in the CPU 30, it first proceeds to step S100, as shown in Figure 4.

[0074] In step S100, a driving simulation control process is executed to control the driving simulation. In the driving simulation control process, first, a virtual environment is recreated based on the course data, object data, and event data in the memory device 46, objects are made to appear, and predetermined events are generated. Then, sensor information is acquired from the steering angle sensor 40, accelerator sensor 42, and brake sensor 44, and the behavior of the simulated vehicle is controlled based on the acquired sensor information. For example, a physics calculation engine is used to apply driving force and braking force according to the sensor information to the simulated vehicle and control the movement of the simulated vehicle.

[0075] Furthermore, the behavior of NPC objects is controlled according to the behavior or state of the simulated vehicle. Specifically, the movement of NPC objects is controlled using a predetermined behavior model or AI engine. In particular, when controlling the behavior of pedestrians, the system controls them to exhibit human-like reactions based on their positional relationship (distance), relative speed, and direction relative to the simulated vehicle. For example, if a simulated vehicle approaches, pedestrians are controlled to avoid, stop, or reverse.

[0076] Then, the view from the controlled simulated vehicle is generated in real time using a CG (Computer Graphics) model and played back as a video on the display 12. For the driving simulation control processing, for example, the technology described in Patent Document 2 can be employed.

[0077] Next, the process moves to step S102. In step S102, log information recording processing is performed. In the log information recording processing, behavior log information and state log information for each object controlled in step S100 are acquired and recorded chronologically in the behavior log table 402 and the state log table 404, respectively. In addition, the sensor information acquired in step S100 and the state information of the simulated vehicle controlled in step S100 are associated and recorded chronologically in the simulated vehicle log table 406.

[0078] Next, the process moves to step S104. In step S104, a behavior determination process is executed to determine the behavior of pedestrians in accordance with the behavior or state of the simulated vehicle. In the behavior determination process, based on the information from the behavior log table 402, the state log table 404, and the simulated vehicle log table 406, it is determined whether or not the behavior of the NPC object satisfies behavior determination conditions, including whether or not the behavior is reactive behavior. For example, the following five behavior determinations can be performed.

[0079] (1) First behavioral judgment: Intentional disregard of slowing down on a narrow sidewalk (subject to penalty) The first behavioral judgment determines a situation where a simulated vehicle did not slow down despite the presence of oncoming pedestrians on a narrow sidewalk, forcing the pedestrians to take evasive action.

[0080] The behavior determination condition is considered to have been met if all four of the following conditions are met. The scenario requires that a simulated vehicle and a pedestrian are located within the "narrow sidewalk" area, and that they are facing each other. Whether or not they are located within the area can be determined by the position of the simulated vehicle in the simulated vehicle log table 406, the position of the pedestrian in the status log table 404, and the sidewalk area tag at those positions in the object management table 400. Whether or not the simulated vehicle and pedestrian are facing each other can be determined by whether or not the direction of the pedestrian in the status log table 404 relative to the direction of the simulated vehicle in the simulated vehicle log table 406 is within a predetermined range (150° to 180°).

[0081] Regarding pedestrian behavior, it refers to whether the pedestrian performed "avoidance" or "stopped". Whether or not "avoidance" or "stopping" occurred can be determined from the pedestrian's behavior in the behavior log table 402.

[0082] The condition of the simulated vehicle is that its speed is above a predetermined value. Whether or not the speed is above a predetermined value can be determined by checking whether the speed of the simulated vehicle in the simulated vehicle log table 406 is above a predetermined value (for example, 2.1 km / h).

[0083] The intention of the person being evaluated is determined by whether the person being evaluated is operating the accelerator. If the accelerator is being operated, it can be determined that the person being evaluated has the intention to proceed. Whether or not the accelerator is being operated can be determined by whether or not the accelerator lever value in the simulated vehicle log table 406 is equal to or greater than a predetermined value (for example, 0.1).

[0084] (2) Second behavioral judgment: yielding to others on a narrow sidewalk (eligible for bonus points) The second behavioral assessment determines a situation where, on a narrow sidewalk, the simulated vehicle voluntarily stopped and yielded the right of way to an oncoming pedestrian, thus eliminating the need for the pedestrian to change their behavior.

[0085] The behavior determination condition is considered to have been met if all four of the following conditions are met. The scenario involves a simulated vehicle and pedestrians being located within a "narrow sidewalk" area, and the simulated vehicle and pedestrians facing each other. The judgment method is the same as in (1) above.

[0086] Regarding the pedestrian's behavior, the pedestrian must be continuing to "move forward." The method of determination is the same as in (1) above.

[0087] The state of the simulated vehicle is that it is stopped or moving slowly. Whether or not it is stopped or moving slowly can be determined by whether or not the speed of the simulated vehicle in the simulated vehicle log table 406 is below a predetermined value (for example, 0.1 km / h).

[0088] The intention of the person being evaluated is determined by whether or not the person being evaluated is operating the accelerator. If the person is not operating the accelerator, it can be determined that the person being evaluated has no intention of passing. Whether or not the accelerator is being operated can be determined by whether or not the accelerator lever value in the simulated vehicle log table 406 is less than or equal to a predetermined value (for example, 0.1).

[0089] (3) Third behavior judgment: Forcing one's way (penalty) The third behavioral judgment determines whether the simulated vehicle continues to aggressively approach the pedestrian, forcing the pedestrian to back up (leaving no escape route).

[0090] The behavior determination condition is considered to have been met if all four of the following conditions are met. The scenario involves a simulated vehicle and pedestrians being located within a "sidewalk" or "shopping street" area, and the simulated vehicle and pedestrians facing each other. The determination method is the same as in (1) above.

[0091] The pedestrian's behavior is defined as the pedestrian "reversing" and this behavior continuing for a predetermined period of time or longer. The method for determining the behavior is the same as in (1) above. Whether or not the behavior continues for a predetermined period of time or longer can be determined by whether or not a predetermined period of time (for example, 5 seconds) has elapsed since the pedestrian's behavior in the behavior log table 402 became "reversing".

[0092] The condition of the simulated vehicle is that the distance between the simulated vehicle and the pedestrian is less than or equal to a predetermined value. Whether or not the distance is less than or equal to a predetermined value can be determined by checking whether the distance between the position of the simulated vehicle in the simulated vehicle log table 406 and the position of the pedestrian in the status log table 404 is less than or equal to a predetermined value (for example, 2m).

[0093] The evaluation criteria for the person being evaluated are that the person being evaluated is operating the accelerator. The evaluation method is the same as in (1) above.

[0094] (4) Fourth behavioral judgment: Obstructing a following pedestrian on the sidewalk (subject to penalty) The fourth behavioral judgment determines a situation where, as a result of the simulated vehicle stopping on the sidewalk, it blocks the path of a following pedestrian traveling in the same direction, forcing the pedestrian to stop.

[0095] The behavior determination condition is considered to have been met if all four of the following conditions are met. The scenario is defined as a simulated vehicle and pedestrians being located within the "sidewalk" area, and both facing the same direction. The method for determining the area is the same as in (1) above. Whether or not the simulated vehicle and pedestrians are facing the same direction can be determined by checking whether the direction of the pedestrians in the state log table 404 relative to the direction of the simulated vehicle in the simulated vehicle log table 406 is within a predetermined range (0° to 30°).

[0096] Regarding the pedestrian's behavior, the pedestrian "stopped". The method of determination is the same as in (1) above.

[0097] The state of the simulated vehicle is that it is stopped or moving slowly. The determination method is the same as in (2) above.

[0098] The evaluation criteria for the person being evaluated are that they did not operate the accelerator. The evaluation method is the same as in (2) above.

[0099] (5) Fifth behavior judgment: Obstructing pedestrians at a crosswalk (penalty) The fifth behavioral judgment determines a situation in which a simulated vehicle intentionally stops on a crosswalk, forcing other pedestrians to take evasive action.

[0100] The behavior determination condition is considered to have been met if all four of the following conditions are met. The scenario requires that a simulated vehicle and pedestrians are present within the "crosswalk" area. The determination method is the same as in (1) above.

[0101] Regarding the pedestrian's behavior, it was determined that the pedestrian performed an "evasive maneuver." The method of determination is the same as in (1) above.

[0102] The state of the simulated vehicle is that it is stopped or moving slowly. The determination method is the same as in (2) above.

[0103] The evaluation criteria for the person being evaluated are that they did not operate the accelerator. The evaluation method is the same as in (2) above.

[0104] Next, the process moves to step S106. In step S106, an evaluation process is performed to evaluate the driving operations of the person being evaluated. In the evaluation process, evaluation information including evaluation points is generated based on the judgment result in step S104 and recorded in the storage device 46. Specifically, if it is determined that the behavior judgment condition is met in the first behavior judgment, third behavior judgment, fourth behavior judgment, or fifth behavior judgment, the evaluation points are deducted. On the other hand, if it is determined that the behavior judgment condition is met in the second behavior judgment, the evaluation points are added.

[0105] Next, the process moves to step S108. In step S108, it is determined whether the driving simulation has finished (for example, whether the person being evaluated has reached the goal or whether the predetermined simulation time has elapsed). If it is determined that the driving simulation has finished (YES), the process moves to step S110. On the other hand, if it is determined in step S108 that the driving simulation has not finished (NO), the process moves to step S100 and the process is repeated.

[0106] Next, the process moves to step S110. In step S110, an evaluation information generation process is executed to generate final evaluation information regarding the driver's operation based on the evaluation information recorded in the storage device 46, and the series of processes ends. The generated evaluation information is displayed on the display 12 as an evaluation report, for example. In addition to the overall evaluation, the evaluation report includes specific feedback such as "the number of times pedestrians were prompted to take evasive action" and "situations where consideration was insufficient."

[0107] 〔effect〕 Next, the effects of this embodiment will be described. In this embodiment, the behavior of an NPC object is determined according to the behavior or state of the simulated vehicle, and the actions of the person being evaluated are evaluated based on the determination result.

[0108] This allows for the evaluation of "consideration for others," which was difficult to assess with conventional simulators that only evaluated the operator's actions and the state of the simulated vehicle, based on objective facts such as "what actions others were forced to take." Therefore, consideration for others can be evaluated more objectively than before. Furthermore, by providing feedback to the operator about "inconsiderate driving" that, while not resulting in accidents or violations, causes fear or anxiety to others, higher quality safety education can be provided.

[0109] Furthermore, in this embodiment, it is determined whether the behavior determination conditions are met based on the behavior information and state information of the NPC object, as well as the state information of the simulated vehicle.

[0110] This allows for evaluation based on the state of the simulated vehicle and the state of NPC objects, enabling assessment of consideration for others in a variety of situations.

[0111] Furthermore, in this embodiment, it is determined whether the behavior determination conditions are met based on the behavior information and state information of the NPC object, the state information of the simulated vehicle, and the sensor information of sensors 40 to 44.

[0112] This allows for evaluation based on the state of the simulated vehicle, the state of NPC objects, and the actions of the person being evaluated, enabling assessment of consideration for others in a wider variety of situations.

[0113] Furthermore, in this embodiment, the behavior determination conditions include: (1) the positional relationship between the simulated vehicle and the NPC object is a predetermined positional relationship; (2) the speed of the simulated vehicle or the NPC object is a predetermined speed; (3) the orientation of one of the simulated vehicle and the NPC object relative to the other is a predetermined orientation; (4) the operation performed by the person being evaluated is a predetermined operation; or (5) the area in the virtual space where the simulated vehicle or the NPC object is located is a predetermined area.

[0114] This allows for evaluation based on whether at least one of conditions (1) to (5) is met, enabling assessment of consideration for others in a wider range of situations.

[0115] In this embodiment, step S100 corresponds to the operation information acquisition means of Invention 1 or 6, or the control means of Invention 1 or 2; step S102 corresponds to the behavior information acquisition means of Invention 2, 5, or 6, the first state information acquisition means of Inventions 5 to 7, or the second state information acquisition means of Inventions 5 to 7. Furthermore, step S104 corresponds to the determination means of Invention 1, 2, 5, or 6; step S106 corresponds to the evaluation means of Invention 1; sensor information corresponds to the operation information of Invention 1 or 6; and the simulated vehicle corresponds to the first object of Inventions 1 to 5 or 7.

[0116] Furthermore, in this embodiment, the NPC object corresponds to the second object of inventions 1 to 5 or 7.

[0117] [Variation] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from its spirit. The configurations, functions, processes, means, etc., described in the embodiments described above can be combined or partially omitted as long as they do not contradict the technical aspects. The main modifications are shown below.

[0118] (1) Target object In the above embodiment, the first object was set as a simulated vehicle and the second object as a pedestrian (NPC object), but the embodiment is not limited to this.

[0119] The first object can be set as, for example, an automobile (passenger car, truck, bus, etc.), a motorcycle, a moped (e.g., a specific small moped, an electric kick scooter), a bicycle, a small mobile vehicle, a personal mobility device or other vehicle, a pedestrian (e.g., a pedestrian as the subject of evaluation), a wheelchair user, an animal, a mobile robot (e.g., an automated delivery robot, a bipedal robot), construction machinery, agricultural machinery, a drone, a ship, an aircraft or other mobile object.

[0120] The second object can be set as, for example, an automobile, motorcycle, moped, bicycle, small mobile vehicle, personal mobility device or other vehicle, pedestrian, wheelchair user, animal, mobile robot, construction machinery, agricultural machinery, drone, ship, aircraft or other mobile object. Furthermore, specific attributes (for example, age group (child, elderly, etc.), attention state, presence or absence of luggage, etc.) can be set for the second object, and evaluation can be performed based on the reaction behavior corresponding to those attributes.

[0121] Furthermore, the second object is not limited to NPC objects; it can also be set as an object operated by other people (other evaluators, instructors, etc.) in a multiplayer environment. Additionally, if multiple second objects exist, the evaluation can consider not only the behavior of each individual object but also the collective behavior (crowd behavior).

[0122] (2) System configuration and environment In the above embodiment, the driving simulator 10 was configured as a single device (standalone configuration), but it is not limited to this. For example, it can be configured as a network system in which a client device having an operating device 14 and a display 12, etc., and a server having the functions of the control device 100 are connected via a network. The functions of the control device 100 can be built on a cloud server and provided as cloud gaming or SaaS (Software as a Service). In addition, some or all of the functions of this simulator can be provided as an API (Application Programming Interface).

[0123] Furthermore, the driving simulator 10 is not limited to a dedicated stationary device; it can also be implemented using a PC, smartphone, tablet, or other device.

[0124] The display format is not limited to display 12; it can also utilize VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality) devices such as HMDs (Head Mounted Displays) or smart glasses, any number of displays (e.g., one, two, or four or more), or an immersive display system using multiple projectors (such as a CAVE (Cave Automatic Virtual Environment) type display). For example, a configuration can be adopted in which a virtual second object is superimposed onto a video of the real world using AR or MR technology for evaluation.

[0125] Furthermore, it is possible to adopt a configuration that integrates with motion platforms and haptic feedback devices to provide the evaluator with movements and vibrations corresponding to the behavior of the simulated vehicle.

[0126] (3) Operation information In the above embodiment, a handle 16, an accelerator lever 18, and a brake lever 20 were used as input devices, but the system is not limited to these. Joysticks, game controllers, keyboards, mice, touch panels, etc., can be used as input devices. Furthermore, the accelerator lever value and brake lever value can be controlled using the accelerator lever 18 without using the brake lever 20. Specifically, when the accelerator opening of the accelerator lever 18 increases, the accelerator lever value is increased and the brake lever value is set to 0 or decreased, and when the accelerator opening decreases, the accelerator lever value is set to 0 or decreased and the brake lever value is increased.

[0127] Furthermore, biometric information obtained from biometric information detection devices can be used as operational information. For example, eye-tracking devices can be used to acquire the eye movements of the person being evaluated, and their attention status to others can be used as operational information. In addition, electroencephalogram (EEG) devices, heart rate sensors, electromyography (EMG) sensors, sweat sensors, etc., can be used to acquire the psychological state of the person being evaluated (fear, anxiety, discomfort, stress, relief, arousal level, etc.) and use as operational information. This operational information can also be used for behavioral judgment and evaluation.

[0128] Furthermore, voice input via a voice input device (microphone) (for example, saying "excuse me") and gesture information based on camera images can be acquired and used as operation information. This operation information can also be used for behavior determination and evaluation.

[0129] Furthermore, in addition to real-time operation input, it is possible to adopt a configuration (offline evaluation, post-hoc analysis) that acquires operation log data recorded in advance on actual equipment, and then performs simulations and evaluations based on the acquired operation log data.

[0130] (4) Judgment methods and behavioral models In the above embodiment, the determination was made using a rule-based method, but it is not limited to this. For example, a machine learning model (AI) can be used to determine the behavior of the second object or to evaluate the actions of the person being evaluated. Specifically, a configuration can be considered in which time-series data of the first and second objects are taken as input, the AI ​​estimates the psychological burden and degree of danger that others may have felt, and the consideration is evaluated based on the results. Alternatively, a physical calculation model (for example, a social force model) can be used to calculate the psychological burden on the second object due to the approach of the first object, and the determination can be made based on that calculated value.

[0131] The behavior determination conditions are not limited to the five behavior determinations in the above embodiment; the number and specific conditions (thresholds, area definitions, etc.) can be changed as appropriate. The behavior determination conditions can include one or more of the following conditions, provided that the behavior of the second object is a reactive behavior: (1) the positional relationship between the simulated vehicle and the second object is a predetermined positional relationship; (2) the speed of the simulated vehicle or the second object is a predetermined speed; (3) the orientation of one of the simulated vehicle and the second object relative to the other is a predetermined orientation; (4) the operation of the person being evaluated is a predetermined operation; or (5) the area in the virtual space where the simulated vehicle or the second object is located is a predetermined area. The behavior determination conditions do not have to include conditions (1) to (5). In addition, environmental conditions of the virtual space (weather, time of day, road surface conditions, etc.) and the above attributes of the second object can be added to the behavior determination conditions. For example, if the other party is a child or if there is bad weather, the consideration can be determined under stricter conditions.

[0132] Furthermore, it is possible to adopt a configuration that makes a comprehensive determination not only based on an event at a single point in time, but also based on the history (time-series data) of the behavior of the second object in a series of operations.

[0133] Furthermore, although the behavior determination conditions in the above embodiment include "the intention of the person being evaluated (operation information)," this is not mandatory. For example, a configuration can be adopted in which the determination is made based on the behavior information and state information of the first and second objects without using operation information.

[0134] For controlling the behavior of the second object, not only behavioral models such as social force models can be used, but also reinforcement learning-based AI agents and other behavioral models (for example, cellular automaton-based models).

[0135] (5) Reaction behavior In the above embodiment, examples of the reaction behavior of the second object were given as movement-related behaviors such as "avoidance," "stop," "backward," and "forward," but the embodiment is not limited to these.

[0136] For example, you can set reaction behaviors such as "increasing speed (hurrying)" or "decreasing speed (hesitating)," or changes in attitude such as "turning around" or "looking."

[0137] Furthermore, changes in the emotions, facial expressions, or psychological state of the second object can be set as reaction behaviors and used for evaluation. For example, changes in facial expressions (surprise, anger, gratitude), vocal utterances (e.g., warnings such as "dangerous" or thanks such as "thank you"), and gestures (e.g., stopping with a hand, bowing, or urging someone to pass) can be set as reaction behaviors. In addition, the psychological state of the second object can be parameterized and set as a reaction behavior. This allows for a more detailed evaluation of the psychological impact on others.

[0138] (6) Evaluation methods and feedback In the above embodiment, evaluation points were added or subtracted based on the judgment result, but the evaluation method is not limited to this. For example, graded evaluation (rank evaluation) or category-based evaluation can be performed. Furthermore, feedback comments and improvement advice tailored to specific situations can be automatically generated using generation AI, etc. In addition, a comprehensive evaluation can be performed by combining the consideration evaluation according to the present invention with conventional evaluation items (violation of laws and regulations, presence or absence of accidents, etc.).

[0139] Furthermore, it is possible to adopt a configuration in which the level of reaction behavior of the second object (for example, minor avoidance, emergency avoidance, falling) is set in stages, and the weighting of the evaluation (the magnitude of deductions or additions) is changed according to that level.

[0140] Feedback can be provided not only after the simulation is complete, but also in real time during the simulation. For example, if inconsiderate driving is detected, the person being evaluated can be immediately notified through a warning sound, voice guidance, visual effects, or haptic feedback (such as vibration of the control device or seat).

[0141] The feedback can include not only the evaluation results, but also a function to replay scenes where consideration was lacking as video from the perspective of the first or second object, or from an overhead view. In particular, log information of high-priority scenes, such as when the second object showed emotions such as fear, anxiety, or discomfort, or when predetermined behavioral judgment conditions were met, can be provided as evaluation information. This allows for an effective understanding of how the evaluated person's driving affected others.

[0142] The evaluation results can be communicated or shared not only to the person being evaluated, but also to third parties such as instructors or managers, or they can be printed out.

[0143] (7) Data structure and processing The configuration of tables 400 to 406 in the above embodiment is an example and can be modified as appropriate. For example, the behavior log table 402 and the status log table 404 can be configured in a single table. Furthermore, not only relational databases but also NoSQL databases and the like can be used.

[0144] The state of the simulated vehicle is registered in the simulated vehicle log table 406, but the simulated vehicle can also be defined as an object and registered in the state log table 404. Furthermore, although the behavior of the simulated vehicle was not registered in the above embodiment, it is not limited to this, and the behavior of the simulated vehicle can also be registered in the simulated vehicle log table 406, etc. The behavior of the simulated vehicle can be used for behavior control and evaluation of the second object.

[0145] Area tags were initially registered for static objects, but they can also be used to manage coordinate ranges on course data (map data).

[0146] Although the system is configured to record log information (step S102) at all times, it is also possible to record detailed logs only when a specific event occurs or when a behavioral judgment condition is likely to be met (conditional logging).

[0147] The control process of the above embodiment (Figure 4) is just one example, and the processing order can be changed or the processing can be executed in parallel (for example, in parallel execution of step S100 and step S102) as appropriate.

[0148] (8) Fields of application The present invention can be applied to any simulator that supports education, training, research, development, or testing. In the case of education or training, for example, it refers to the education or training of an evaluator, and in the case of research, development, or testing, for example, it refers to research, development, or testing using the results of a mobility simulation performed by an evaluator.

[0149] The present invention is not limited to evaluating the driving of mobility scooters, but can also be applied to driving training systems (driving schools, senior citizen training courses, etc.), safety education systems, rehabilitation support systems, evaluation systems for research and development or testing of mobile devices, or systems for entertainment (games, etc.), insurance premium calculation support, or traffic environment design (evaluation of road structures, etc.).

[0150] (9) Others In the above embodiment, the process shown in the flowchart of Figure 4 was described as executing a program pre-stored in ROM 32. However, the invention is not limited to this, and a program describing these procedures may be read into RAM 34 from a storage medium containing such a program and then executed.

[0151] Furthermore, the above embodiments and their modified forms (including their respective constituent technologies) are mutually applicable.

[0152] Furthermore, the invention is applicable not only to the above embodiments and their modifications, but also to other cases without departing from the spirit of the present invention. [Explanation of Symbols]

[0153] 10…Driving simulator, 12…Display, 14…Control device, 16…Steering wheel, 40…Steering angle sensor, 18…Accelerator lever, 42…Accelerator sensor, 20…Brake lever, 44…Brake sensor, 100…Control device, 30…CPU, 32…ROM, 34…RAM, 38…I / F, 39…Bus, 46…Storage device, 400…Object management table, 402…Behavior log table, 404…Status log table, 406…Simulated vehicle log table

Claims

1. An operation information acquisition means for acquiring operation information related to the actions of the person being evaluated, A control means that controls the behavior of a first object based on operation information acquired by the operation information acquisition means in a virtual space, and controls the behavior of a second object according to the behavior or state of the first object, A determination means for determining the behavior of the second object according to the behavior or state of the first object, A simulator characterized by comprising: an evaluation means for evaluating the operation of the person being evaluated based on the determination result of the determination means.

2. In claim 1, The control means controls the behavior of the second object so that it exhibits a predetermined reaction behavior according to the behavior or state of the first object. The system includes a behavior information acquisition means for acquiring behavior information relating to the behavior of the second object, The simulator is characterized in that the determination means determines whether the behavior of the second object is the reaction behavior based on the behavior information acquired by the behavior information acquisition means.

3. In claim 2, The simulator is characterized in that the reaction behavior includes the second object avoiding the first object, the second object stopping, or the second object moving backward or forward relative to the first object.

4. In claim 2, The simulator is characterized in that the reaction behavior includes the second object acting in response to the first object acting, the second object acting in response to the first object not acting, the second object not acting in response to the first object acting, and the second object not acting in response to the first object not acting.

5. In any one of claims 2 to 4, A first state information acquisition means for acquiring state information relating to the state of the first object, The system includes a second state information acquisition means for acquiring state information relating to the state of the second object, The simulator is characterized in that the determination means determines whether the behavior of the second object satisfies predetermined conditions, including that the behavior is the reaction behavior, based on the behavior information acquired by the behavior information acquisition means, the state information acquired by the first state information acquisition means, and the state information acquired by the second state information acquisition means.

6. In claim 5, The simulator is characterized in that the determination means determines whether the predetermined conditions are met based on the behavior information acquired by the behavior information acquisition means, the state information acquired by the first state information acquisition means, the state information acquired by the second state information acquisition means, and the operation information acquired by the operation information acquisition means.

7. In claim 6, The first state information acquisition means acquires state information relating to the position, velocity, or orientation of the first object, The second state information acquisition means acquires state information relating to the position, velocity, or orientation of the second object, The simulator is characterized in that the predetermined conditions include (1) the positional relationship between the first object and the second object is a predetermined positional relationship, (2) the velocity of the first object or the second object is a predetermined velocity, (3) the orientation of one object relative to the other is a predetermined orientation, (4) the operation of the person being evaluated is a predetermined operation, or (5) the area in the virtual space where the first object or the second object is located is a predetermined area.

Images