Vehicle action simulator system

Abstract

The present application provides a vehicle motion simulator system capable of efficiently performing preparation work and post-test arrangement work. Specifically, a virtual object is arranged in an actual scene, a vehicle (2) having a virtual sensor (22A-22L) for detecting the virtual object performs driving in the actual scene based on detection information of the virtual sensor (22A-22L), and driving results of the vehicle are collected. On the other hand, a sensor switching switch (21) is provided, which switches information for driving assistance of the vehicle between the detection information of the virtual sensor (22A-22L) and detection information of an actual sensor.

Description

Technical Field

[0001] This invention relates to a vehicle motion simulator system that simulates the movements of a vehicle. Background Technology

[0002] In recent years, in addition to manual driving based on user input, a new mode of vehicle operation has emerged: automated driving assistance systems (AGS). These systems assist users in driving by executing some or all of the user's driving operations on the vehicle side. AGS systems equip vehicles with sensors such as ultrasonic sensors, millimeter-wave radar sensors, and LiDAR (Light Detection and Ranging) sensors to detect surrounding objects (e.g., people, bicycles, other vehicles, walls). Based on the detection results, the system automatically controls the vehicle's steering, drive system, brakes, and other functions.

[0003] In the development of the aforementioned automated driving assistance system, it is necessary to test whether the prototype automated driving assistance system can accurately perform its actions in actual driving environments. Previously, the prototype automated driving assistance system was installed in a vehicle and tested by driving in a scene (field) where the aforementioned objects were actually located. However, this testing needs to be conducted in various driving environments, but changing the environment requires changing the type of objects or reconfiguring them, resulting in a very long preparation period before the start of testing. Therefore, a method has been proposed to create a virtual driving environment by configuring virtual objects (hereinafter referred to as virtual objects) that do not actually exist in the actual scene, and to conduct the aforementioned tests by installing virtual sensors for detecting virtual objects in the vehicle (e.g., Japanese Patent Publication No. 2021-507247).

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Publication No. 2021-507247 (paragraphs 0010-0016) Summary of the Invention

[0007] The problem the invention aims to solve

[0008] The virtual sensor used in Patent Document 1 can detect virtual objects that do not actually exist in the driving environment, but it cannot detect actual objects that are unrelated to the test. Therefore, when moving the vehicle to the starting position of the test and when moving it to the storage position after the test, it is necessary to switch from a virtual sensor to an actual sensor in order to make the sensor information effective.

[0009] Previously, switching such sensors required manual wiring replacement, which was extremely inefficient. On the other hand, without switching from virtual to physical sensors, the vehicle needed to be manually moved to the designated position while simultaneously assessing the surrounding environment, incurring time costs in preparation and post-test cleanup. Furthermore, while the above specifically used the testing of advanced driver assistance systems (ADAS) as an example to illustrate the problem, it is not limited to ADAS. For instance, similar issues arise when testing in a virtual driving environment to verify the appropriateness of manual driving assistance functions such as deceleration control and warnings for obstacles around the vehicle.

[0010] This invention was developed to eliminate the aforementioned problems and aims to provide a vehicle motion simulator system that can efficiently perform preparation and post-test cleanup work during vehicle driving tests in which virtual objects are configured in a real-world scenario.

[0011] means for solving problems

[0012] To achieve the above objectives, the vehicle motion simulator system of the present invention comprises: a virtual object configuration unit for configuring virtual objects in an actual scene; and a driving result collection unit for collecting driving results of a vehicle having virtual sensors for detecting the virtual objects in the actual scene, wherein the vehicle has, in addition to the virtual sensors, actual sensors for detecting objects actually existing around the vehicle; and a sensor switching unit for switching driving assistance information for the vehicle between the detection information of the virtual sensors and the detection information of the actual sensors.

[0013] It should be noted that "sensors" also include cameras that perform image recognition-based detection.

[0014] Furthermore, "virtual objects" are not limited to three-dimensional objects; they can be any object that can be detected using image recognition based on range sensors and cameras, including lane lines and road markings. Also, they are not limited to stationary objects but can also be moving objects. In the case of moving objects, a pre-set plan for their movement is implemented.

[0015] Furthermore, "vehicle" can also refer to a vehicle capable of assisted driving, in addition to manual driving based on user input, or to driving automatically without user input. In this case, particularly when the vehicle performs assisted driving in a real-world scenario based on detection information from virtual sensors, the sensor switching unit can switch between information used for assisted driving between the detection information from virtual sensors and the detection information from actual sensors. On the other hand, the vehicle is not necessarily limited to vehicles capable of assisted driving based on the aforementioned assisted driving; it can also be a vehicle capable only of manual driving. In this case, particularly when the vehicle performs manual driving in a real-world scenario, driving assistance can be provided, for example, by performing deceleration control and warnings for obstacles around the vehicle. Furthermore, the sensor switching unit can switch between information used for such deceleration control and warnings between the detection information from virtual sensors and the detection information from actual sensors.

[0016] The effects of the invention

[0017] According to the vehicle motion simulator system of the present invention with the above structure, in the vehicle driving test where a virtual object is configured in a real scene, a sensor switching unit is set up to switch the information used for vehicle driving assistance between the detection information of virtual sensors and the detection information of actual sensors. Compared with the past, preparation work and post-test cleanup work can be performed more efficiently. Attached Figure Description

[0018] Figure 1 This is a schematic structural diagram of the vehicle motion simulator system according to the first embodiment.

[0019] Figure 2 This diagram illustrates an example of displaying a real-world image of a scene overlaid with an image representing a virtual object on an in-vehicle display.

[0020] Figure 3 This diagram illustrates an example of displaying a real-world image of a scene overlaid with an image representing a virtual object on a VR headset.

[0021] Figure 4 This diagram illustrates an example of displaying a real-world image of a scene overlaid with an image representing a virtual object on the rearview mirror.

[0022] Figure 5 This is a diagram illustrating an example of displaying real-world images of an actual scene, overlaid with images representing virtual objects, from multiple viewpoints.

[0023] Figure 6This is a schematic structural diagram of the vehicle involved in the first embodiment.

[0024] Figure 7 This is an image showing the appearance of the virtual sensors on the simulator device.

[0025] Figure 8 This is a diagram showing the internal structure of a virtual sensor.

[0026] Figure 9 This is a diagram illustrating the sensor information switching method based on a sensor switching switch according to the first embodiment.

[0027] Figure 10 This is a block diagram illustrating the structure of the simulator device according to the first embodiment.

[0028] Figure 11 This is a flowchart of the motion evaluation procedure involved in the first embodiment.

[0029] Figure 12 This diagram illustrates the sensor information switching method based on a sensor switching switch according to the second embodiment. Detailed Implementation

[0030] Hereinafter, the first and second embodiments of the vehicle motion simulator system according to the present invention will be described in detail with reference to the accompanying drawings.

[0031] [First Implementation Method]

[0032] First, use Figure 1 The general structure of the vehicle motion simulator system 1 according to the first embodiment will be described. Figure 1 This is a schematic structural diagram of the vehicle motion simulator system 1 according to this embodiment.

[0033] like Figure 1As shown, the vehicle motion simulator system 1 according to this embodiment basically includes a vehicle 2 and a simulator device 3. The vehicle 2 is the object of driving tests conducted through simulation. The simulator device 3 provides the vehicle 2 with a virtual driving environment as the driving environment for the driving test, and acquires, analyzes, evaluates, and outputs the driving test motion results of the vehicle 2. Furthermore, in the first embodiment below, the evaluation object that becomes the quality evaluation object in the above-mentioned driving test is an automated driving assistance system that assists the occupant in driving the vehicle by performing part or all of the occupant's driving operations on the vehicle side. The vehicle 2 is a vehicle equipped with the automated driving assistance system that becomes the evaluation object. However, as will be described later, the vehicle 2 is not necessarily limited to a vehicle equipped with the above-mentioned automated driving assistance system, and may also be a vehicle that can only perform driving based on manual driving. In this case, for example, it becomes a driving test for evaluating whether driving assistance such as deceleration control and warnings for obstacles located around the vehicle is appropriately performed in manual driving.

[0034] Furthermore, driving tests are one of the processes used to identify and correct malfunctions of the automated driving assistance system on the manufacturer's side before delivery. This involves simulating the actions of the automated driving assistance system under various hypothetical driving environments, acquiring and analyzing the action results, thereby confirming whether any malfunctions have occurred. Moreover, driving tests are conducted by actually driving the vehicle 2 in real-world scenarios (e.g., a wide, flat road surface with nothing around), and by using a virtual driving environment provided by a simulator device 3, driving tests can be performed under various driving conditions even in the same real-world scenario.

[0035] Furthermore, the simulator device 3 is mounted on the vehicle 2 and can be bidirectionally connected to the vehicle 2 via an in-vehicle network such as CAN (Controller Area Network). However, the simulator device 3 does not necessarily need to be mounted on the vehicle 2; it can be partially or entirely located outside the vehicle 2. In this case, it can be bidirectionally connected to the vehicle 2, for example, via wireless communication. Additionally, the simulator device 3 includes an operation unit such as a keyboard and a display unit, configured to allow for the input and output of various information through the operation of an evaluator (observer / occupant) 4 who is evaluating the autonomous driving assistance system. Specifically, the evaluator 4 inputs the positions and types of virtual objects configured as the driving environment in the actual driving scenario of the vehicle 2 to the simulator device 3. Furthermore, the simulator device 3 constructs a virtual driving environment for testing the vehicle 2 based on the input driving environment.

[0036] Furthermore, the simulator device 3 has virtual sensors for vehicle 2 to detect virtual objects constructed as the aforementioned driving environment. The detection results of virtual objects detected by the virtual sensors while vehicle 2 is driving in the actual scene are output to the control device controlling the autonomous driving assistance system on the vehicle 2 side. Thus, the vehicle side recognizes the virtual objects as actual objects (e.g., people, bicycles, other vehicles, walls, etc.), enabling the reproduction of a driving environment closer to reality.

[0037] Furthermore, the analysis of the driving results of vehicle 2 in simulator device 3 is configured to perform either "real-time analysis" or "post-analysis". "Real-time analysis" monitors vehicle 2 during the driving test and automatically detects and outputs any adverse conditions that occur. On the other hand, "post-analysis" evaluates and outputs the control content, driving trajectory, etc. of vehicle 2 stored during the driving test after the driving test of vehicle 2 has ended.

[0038] Therefore, when conducting a driving test of vehicle 2, firstly, as the driving environment for the driving test of vehicle 2, the evaluator 4 inputs the types and positions of virtual objects, specifically those configured relative to the actual scene, into the simulator device 3. This generates the driving environment for the driving test. Furthermore, "virtual objects" are not limited to three-dimensional objects; for example, they include lane lines, road markings, etc. Moreover, they are not limited to stationary objects; they can also be moving objects. In the case of moving objects, a plan for their movement (at what speed and how) is pre-set. For example, in the case of a driving test in a parking lot, parking area markings drawn on the road surface within the parking lot, other vehicles parked in the parking space, pedestrians moving within the parking lot, and other vehicles traveling along the passageways within the parking lot are input as virtual objects. Then, when the driving test begins, the virtual sensors mounted on the vehicle detect the virtual objects generated as the driving environment, thereby simulating the vehicle's driving in the virtual driving environment using autonomous driving assistance. Furthermore, the driving results are analyzed, evaluated, and output through "real-time analysis" or "post-analysis," and in the event of an adverse situation, the occurrence of the adverse situation can be determined.

[0039] However, since the virtual objects deployed in the actual scene using the simulator device 3 are not actually real objects, even if they are identified using virtual sensors on the vehicle side, they cannot be directly visually confirmed by the evaluator 4. Therefore, especially in the case of "real-time analysis," the positional relationship between the vehicle 2 and the virtual objects is opaque, making it difficult to evaluate whether the vehicle's behavior is appropriate from the evaluator 4's perspective. Therefore, in the vehicle motion simulator system 1 of the first embodiment, the following unit is provided to enable the evaluator 4 to visually confirm that the actual scene where the image representing the virtual object overlaps at the location where the virtual object is deployed is included. Furthermore, there are cases where the evaluator 4 is riding in the vehicle 2 as a passenger, and cases where the observation is performed outside the vehicle; each case will be described under assumption.

[0040] First, the situation where the evaluator 4, as a passenger in vehicle 2, makes a visual confirmation will be explained. In the first example, as... Figure 2 As shown, an in-vehicle display 5, mounted on vehicle 2 and visually accessible to vehicle occupants, displays a real-time scene image 6 captured by an external camera. Then, by synthesizing the scene image 6 displayed on the in-vehicle display 5 with an image 7 representing a virtual object, the evaluator 4 can visually confirm the actual scene where the image 7 representing the virtual object overlaps at the location where the virtual object is positioned. Furthermore, the simulator device 3 can determine the position of the virtual object in the scene image 6 displayed on the in-vehicle display 5 by converting the configuration coordinates (absolute coordinates) of the virtual object set as the driving environment into the coordinate system (relative coordinates) of the external camera.

[0041] Next, in the second example, VR (Virtual Reality) technology is used, and the evaluator 4, as a vehicle occupant, wears a VR headset 8. In the VR headset 8, as... Figure 3 As shown, in addition to pre-recorded footage of the interior of the vehicle, a real-time scene image 6 captured by an external camera is displayed on the outside of the vehicle window. Then, by synthesizing the displayed scene image 6 with an image 7 representing a virtual object, the evaluator 4 can visually confirm that the image 7 representing the virtual object overlaps at the location where the virtual object is positioned. Furthermore, the simulator device 3 can determine the position of the virtual object in the scene image 6 displayed on the VR headset 8 by converting the configuration coordinates (absolute coordinates) of the virtual object set as the driving environment into the VR (i.e., user's perspective) coordinate system (relative coordinates).

[0042] Furthermore, in cases where the evaluator 4, as a passenger in vehicle 2, visually confirms the image 7 representing a virtual object, as in the first and second examples described above, the image 7 representing the virtual object can be further overlaid on the actual scene reflected or displayed in one or both of the interior and exterior rearview mirrors of the vehicle. That is, the image 7 representing the virtual object is overlaid on the actual scene reflected or displayed in one or both of the interior and exterior rearview mirrors. For example, if the interior and exterior rearview mirrors are monitor types displaying images captured by external cameras, such as... Figure 4 As shown, by synthesizing the real-scene image 6 of the actual scene displayed in the image display area of ​​the interior rearview mirror 9 with an image 7 representing a virtual object, the evaluator 4 can visually confirm that the image 7 representing the virtual object is superimposed at the location where the virtual object is located. On the other hand, when the interior and exterior rearview mirrors are reflective mirror type displays with a transmission display on the reflective mirror surface, the transmission display superimposes the real-scene image 7 representing the virtual object onto the reflection in the reflective mirror, thereby enabling the evaluator 4 to visually confirm that the image 7 representing the virtual object is superimposed at the location where the virtual object is located.

[0043] In addition, as a component for enabling the evaluator 4, who is a passenger in vehicle 2, to visually confirm the virtual objects positioned in the actual scene, besides the examples mentioned above, a HUD (Head-Up Display) or a perspective-type VR headset can also be used. Regarding these, since the visual confirmation is achieved by overlaying the image with the actual scenery outside the vehicle as seen through the front and side windows, the display control is performed by overlaying the image of the virtual object at the position of the virtual object within the actual scene encompassed by the passenger's field of vision.

[0044] Next, the situation where the evaluator 4, observing from outside the vehicle, can visually confirm the scene will be explained. For the evaluator 4, observing from outside the vehicle, a real-time scene image 6 of the actual scene captured by an external camera is displayed on the display of a tablet-type external terminal 11 that is communicatively connected to the simulator device 3. In detail, similar to the case where the scene is displayed on the in-vehicle display 5, by synthesizing an image 7 representing a virtual object onto the scene image 6 of the actual scene displayed on the external terminal 11, the evaluator 4 can visually confirm the actual scene where the image 7 representing the virtual object is superimposed at the location where the virtual object is located.

[0045] However, especially when the external terminal 11 displays the actual scene, it is not necessary to display the actual scene contained in the field of view from the vehicle 2's perspective; it is also possible to display the actual scene contained in the field of view from a position different from the vehicle 2. For example, if the external terminal 11 is equipped with a camera, then... Figure 5As shown, a real-time live-scene image 6 of the actual scene captured by the camera of the external terminal 11 is displayed on the monitor of the external terminal 11. Furthermore, the simulator device 3 acquires the optical axis information of the camera of the external terminal 11 and converts the configuration coordinates (absolute coordinates) of the virtual objects set as the driving environment into the coordinate system (relative coordinates) of the camera of the external terminal 11, thereby determining the position of the virtual objects in the actual scene captured by the camera of the external terminal 11. Then, by synthesizing an image 7 representing the virtual objects onto the live-scene image 6 of the actual scene displayed on the monitor of the external terminal 11, the evaluator 4 can visually confirm that the image 7 representing the virtual objects overlaps at the position where the virtual objects are located.

[0046] Furthermore, the display of the actual scene from the perspective of vehicle 2's position and the display of the actual scene from the perspective of evaluator 4 can be appropriately switched through the operation of the external terminal 11 of evaluator 4. This allows for real-time visual confirmation of the actual scene containing virtual objects from various viewpoints within the actual scene, without a fixed viewpoint. In addition to displaying the actual scene from the perspective of vehicle 2's position and the actual scene from the perspective of evaluator 4, if a camera is installed in the actual scene, the actual scene can also be displayed from the perspective of that camera.

[0047] Next, the vehicle 2, which is equipped with an autonomous driving assistance system that is the subject of evaluation of the vehicle motion simulator system 1, will be described below. Figure 6 This is a schematic structural diagram of the vehicle 2 involved in this embodiment.

[0048] Vehicle 2 can be, for example, an automobile powered by an internal combustion engine (engine, etc.) (internal combustion engine vehicle), an automobile powered by an electric motor (motor, etc.) (electric vehicle, fuel cell vehicle, etc.), or an automobile powered by both (hybrid vehicle). Regarding vehicle type, it can be a regular car, or a large commercial truck, bus, construction machinery, etc. Furthermore, while the following description refers to a four-wheeled vehicle, it can also be a two-wheeled or three-wheeled vehicle.

[0049] However, vehicle 2 is a vehicle that, in addition to manual driving based on the user's driving operation, can also perform assisted driving based on automatic driving without relying on the user's driving operation.

[0050] Furthermore, autonomous driving assistance can be implemented only in specific situations such as parking or exiting a parking space, or it can be implemented across all road sections, or it can be configured to only be implemented while the vehicle is traveling on a specific road section (e.g., a highway with gates at the boundary (regardless of whether there are people or it is toll-free)). In the following description, the autonomous driving range for which autonomous driving assistance is implemented includes not only all road sections such as general roads and highways, but also parking lots, and it is implemented only when the user selects to implement autonomous driving assistance (e.g., activates the autonomous driving start button) and it is determined that autonomous driving assistance-based driving is possible. On the other hand, vehicle 2 can also be a vehicle that can only perform assisted driving based on autonomous driving assistance. Alternatively, assisted driving based on autonomous driving assistance can be implemented only for the vehicle's movement from parking to the parking space (i.e., parking assistance).

[0051] Furthermore, in the vehicle control within the automated driving assistance system of this embodiment, for example, the vehicle's current position, the lane in which the vehicle is traveling, and the positions of surrounding obstacles are detected at all times, and the vehicle is automatically controlled by adjusting the steering mechanism, drive system, brakes, etc., so that it travels along a generated travel trajectory at the same speed as the generated speed plan. In particular, in the case of parking assistance, the vehicle is automatically controlled as follows: the parking space and its surrounding conditions are confirmed using the detection results of sensors and cameras, and the parking trajectory up to the parking space is calculated, so that the vehicle enters the parking space along the calculated parking trajectory to complete the parking.

[0052] In addition, such as Figure 6 As shown, vehicle 2 includes: an operation unit 13 for receiving operations from occupants; and an in-vehicle display 5 (or a VR head-mounted display 8) for displaying real-world images 6 of the aforementioned actual scene to the occupants. Figure 2 Other output information from simulator device 3; speaker 15, outputting sound guidance from simulator device 3; front camera 16, rear camera 17, side cameras 18A and 18B, used to photograph the vehicle's surroundings; ultrasonic sensors 19A to 19L, serving as actual sensors for detecting obstacles around the vehicle; driver assistance ECU (electronic control unit) 20, serving as a control unit that performs various calculations related to autonomous driving assistance based on input information; simulator device 3 as described above; and sensor switching switch 21, which switches the sensor information input to driver assistance ECU 20 as described later.

[0053] The following describes the various components of vehicle 2. First, the operation unit 13 is, for example, located on the front surface of the steering wheel (also called the steering wheel), and includes operation buttons for activating automatic driving assistance and for switching sensor information via sensor switching switch 21. By operating the operation unit 13, the user can switch between manual driving based on the user's driving operations and assisted driving based on automatic driving assistance, where the vehicle drives automatically without the user's driving operations. Furthermore, the sensor information input to the driving assistance ECU 20 (described later) can be switched between actual sensors and virtual sensors. Additionally, the operation unit 13 may also have a touch panel located on the front surface of the vehicle display. Furthermore, it may also include a microphone and a voice recognition device.

[0054] The in-vehicle display 5 is installed on the dashboard of the vehicle 2, and displays a real-world image 6 of the actual scene, as described above, with the image 7 representing the virtual object superimposed at the location where the virtual object is configured. Figure 2 Alternatively, the same VR head-mounted display 8 described above can be used instead of the in-vehicle display 5. Furthermore, a display device ( Figure 4 The display device displays a real-world image 6 of the actual scene, as described above, with the same image 7 representing the virtual object superimposed on the interior and exterior rearview mirrors. Furthermore, the real-world image 6 of the actual scene displayed on the vehicle display 5 and the VR headset display 8 is an image that composites the image 7 representing the virtual object onto real-time images captured by the front camera 16, the rear camera 17, and the side cameras 18A and 18B.

[0055] In addition, speaker 15 is installed on the dashboard of vehicle 2 and outputs guidance sounds, warning sounds, etc. from simulator device 3.

[0056] In addition, the front camera 16 is, for example, a shooting device with a camera using a solid-state imaging element such as a CCD (charge-coupled device), and is positioned above the front bumper of the vehicle 2 or on the back side of the interior rearview mirror, for example, with the optical axis direction facing forward of the vehicle's direction of travel.

[0057] The rear camera 17 is also a camera device that uses a solid-state imaging element such as a CCD. For example, it is installed near the center of the license plate mounted at the rear of the vehicle 2, with the optical axis facing the rear of the vehicle.

[0058] Furthermore, the side cameras 18A and 18B are also shooting devices that use cameras with solid-state imaging elements such as CCDs, for example, mounted on the left and right rearview mirrors of vehicle 2, with the optical axis direction facing the side of the vehicle.

[0059] Furthermore, the simulator device 3 reads information related to the types and locations of virtual objects in the driving environment pre-set by the evaluator 4, and synthesizes an image 7 representing the virtual objects at the locations where the virtual objects are located from images captured by the front camera 16, rear camera 17, and side cameras 18A and 18B, thereby generating a real-scene image 6 of the actual scene displayed on the vehicle display 5 and the VR headset display 8. Moreover, the real-scene image that forms the object of the synthesized image 7 representing the virtual objects can be the real-scene image itself captured by the front camera 16, rear camera 17, and side cameras 18A and 18B, or it can be, for example, a composite of images from multiple cameras or an image with a changed viewpoint (e.g., a bird's-eye view).

[0060] On the other hand, the ultrasonic sensors 19A to 19L are equivalent to actual sensors used to detect objects that actually exist around the vehicle. Furthermore, in contrast to the actual sensors (ultrasonic sensors 19A to 19L), there are also virtual sensors 22A to 22L (described later), which cannot detect actual objects but can detect virtual objects; these are also mounted on the vehicle 2. The ultrasonic sensors 19A to 19L are respectively arranged at predetermined intervals at the front, rear, and sides of the vehicle, emitting ultrasonic waves as detection waves around the vehicle 2, and receiving reflected waves from objects located around the vehicle, thereby detecting objects that reflected the detection waves. Specifically, they are ranging sensors capable of measuring the distance (range value) from the object reflecting the detection waves to the time from transmission to reception. Additionally, the ultrasonic sensors 19A to 19L are configured to generate an output signal (including the distance to the detected object) corresponding to the reception result of the received wave and output it to the control unit. Furthermore, the objects detected by the ultrasonic sensors 19A to 19L can include, for example, people, bicycles, other vehicles, walls, and other obstacles that vehicles 2 need to avoid while driving, or obstacles that create parking spaces. Additionally, millimeter-wave sensors or radar sensors can be used as ranging sensors instead of ultrasonic sensors.

[0061] Furthermore, the placement position and orientation of each ultrasonic sensor 19A-19L can be appropriately set. However, in this embodiment, to cover the entire area in front of, behind, and to the left and right sides of the vehicle 2 as the detection range for the object, ultrasonic sensors 19A-19D are positioned facing the front of the vehicle 2 with the direction of their detection waves aligning with the direction of travel. Ultrasonic sensors 19E and 19F are positioned facing left with the direction of their detection waves aligning with the left side of the vehicle's direction of travel. Ultrasonic sensors 19G and 19H are positioned facing right with the direction of their detection waves aligning with the right side of the vehicle's direction of travel. Ultrasonic sensors 19I-19L are positioned facing the rear of the vehicle 2 with the direction of their detection waves aligning with the rear of the vehicle, in the opposite direction of travel. All ultrasonic sensors 19A-19L are positioned at the same height above the ground surface.

[0062] Furthermore, in this embodiment, among the ultrasonic sensors 19A to 19L, particularly the ultrasonic sensors 19A to 19D on the front surface of vehicle 2 and the ultrasonic sensors 19I to 19L on the rear surface of vehicle 2, are positioned such that they can mutually receive reflected waves as indirect waves between adjacent sensors. Therefore, by receiving both direct and indirect waves, not only can the distance to the object be determined, but also the specific position of the object (relative to the vehicle) can be determined using triangulation. Regarding the side ultrasonic sensors 19E to 19H, since they are separately arranged, they cannot receive indirect waves. However, by using triangulation to measure the distance measured at the previous position, the distance measured at the current position, and the distance moved between them, the specific position of the object (relative to the vehicle) can be determined in the same way as the vehicle moves.

[0063] On the other hand, the driver assistance ECU 20 is an electronic control unit that performs various processes related to autonomous driving assistance. The driver assistance ECU 20 is also connected to various sensors used to detect vehicle behavior, such as vehicle speed sensors, wheel speed sensors, acceleration sensors, gyroscope sensors, steering sensors, and gear position sensors, as well as various drive units of the vehicle, such as the steering wheel, brakes, accelerator, and transmission. Based on the detection results of these sensors, it detects the current behavior of the vehicle and calculates control information (control quantities) for controlling each drive unit, thereby implementing autonomous driving assistance for vehicle 2 by controlling each drive unit. Specific aspects of autonomous driving assistance include, for example, continuously detecting the vehicle's current position, the lane the vehicle is traveling in, and the positions of surrounding obstacles, and performing vehicle control of the steering device, drive source, brakes, etc., so as to travel along the generated driving trajectory at a speed according to the same generated speed plan. Especially in the case of parking assistance, the vehicle control is performed as follows: confirming the parking space and its surrounding conditions, calculating the parking trajectory to the parking space, and causing the vehicle to enter the parking space along the calculated parking trajectory and complete parking.

[0064] The system includes a unit for acquiring information related to surrounding obstacles and parking space required by the driver assistance ECU 20 for performing the aforementioned autonomous driving assistance. This unit acquires detection information from ultrasonic sensors 19A-19L and a unit acquires detection information from virtual sensors 22A-22L. The ultrasonic sensors 19A-19L are sensors used to detect objects actually existing around the vehicle. The virtual sensors 22A-22L are sensors used to detect virtual objects in a driving environment virtually generated by the simulator device 3. The virtual sensors 22A-22L are part of the simulator device 3 and are mounted on the vehicle 2.

[0065] in, Figure 7 This is an external view of the virtual sensors 22A to 22L of the simulator device 3. Figure 8 This diagram shows the internal structure of virtual sensors 22A to 22L. Furthermore, virtual sensors 22A to 22L have essentially the same structure; the following explanation uses virtual sensor 22A as an example.

[0066] like Figure 7 and Figure 8As shown, the virtual sensor 22A has a cuboid box shape, with a first ultrasonic sensor 25 at one end and a second ultrasonic sensor 26 at the other end inside the box. Furthermore, the first ultrasonic sensor 25 and the second ultrasonic sensor 26 have essentially the same structure as the aforementioned ultrasonic sensors 19A-19L, and are arranged opposite each other with their detection axis X coaxial. The detection axis X is a virtual straight line extending from the first ultrasonic sensor 25 and the second ultrasonic sensor 26 along the transmission and reception direction of the probe wave. The first ultrasonic sensor 25 and the second ultrasonic sensor 26 each have a piezoelectric vibrator (e.g., a ceramic plate) for transmitting and receiving probe waves. The piezoelectric vibrator is formed as a plane with the detection axis X as its normal. The first ultrasonic sensor 25 and the second ultrasonic sensor 26 are configured such that the piezoelectric vibrator vibrates ultrasonically based on a drive signal applied to an electromechanical conversion element, thereby transmitting probe waves along the detection axis X. Furthermore, the first ultrasonic sensor 25 and the second ultrasonic sensor 26 are configured such that when a received wave is received from the outside, an electromechanical conversion element generates an electrical signal, i.e., a received signal, corresponding to the excitation state of the piezoelectric oscillator based on the received ultrasonic wave.

[0067] Furthermore, the virtual sensor 22A detects the received reflected wave by causing the electromotive force generated by the vibration of the piezoelectric vibrator, which accompanies the reception of the probe wave by the first ultrasonic sensor 25, to exceed a threshold. It then calculates the distance to the object based on the time elapsed between the transmission and reception of the probe wave by the first ultrasonic sensor 25. However, since the object is virtual, the first ultrasonic sensor 25 does not actually transmit a probe wave; instead, it receives the probe wave transmitted by the second ultrasonic sensor 26 as a reflected wave, assuming that a probe wave has been transmitted. In this case, the distance value detected by the virtual sensor 22A can be freely controlled based on the timing of the transmission of the probe wave from the second ultrasonic sensor 26.

[0068] That is, the simulator device 3 determines the current position and orientation of the vehicle based on information from various sensors installed on the vehicle, such as the vehicle speed sensor, wheel speed sensor, acceleration sensor, gyroscope sensor, steering sensor, and gear sensor. Based on the determined current position and orientation of the vehicle and the position of a virtual object configured in a driving environment pre-set by the evaluator 4, it determines the relative position of the virtual object relative to the vehicle 2. Then, based on the determined relative position of the virtual object relative to the vehicle 2 (more specifically, the ultrasonic sensors 19A to 19L on the vehicle), it controls the timing of sending a detection wave from the second ultrasonic sensor 26, thereby enabling the virtual sensors 22A to 22L to detect virtual objects that do not actually exist.

[0069] Furthermore, in the vehicle motion simulator system 1 of the first embodiment, during the driving test (during the collection of driving results), the driver assistance ECU 20 detects surrounding obstacles and parking spaces based on the detection information from virtual sensors 22A-22L, and performs vehicle control of the steering device, drive source, brakes, etc. On the other hand, during the preparation stage before the driving test (the preparation stage before collecting driving results) and after the driving test (after the collection of driving results is completed), it is necessary to avoid actual walls and obstacles to drive the vehicle. Therefore, the driver assistance ECU 20 detects surrounding obstacles and parking spaces based on the detection information from ultrasonic sensors 19A-19L, which are actual sensors, and performs vehicle control of the steering device, drive source, brakes, etc. In addition, the switching of sensor information input to the driver assistance ECU 20 is performed by sensor switching switch 21.

[0070] Furthermore, the number of virtual sensors 22A to 22L is the same as that of the actual ultrasonic sensors 19A to 19L, with virtual sensor 22A corresponding to ultrasonic sensor 19A. That is, during a driving test, the driver assistance ECU 20 identifies virtual sensor 22A as ultrasonic sensor 19A and detects an object (a virtual object that does not actually exist) located to the left front of the vehicle. For example, if the distance from ultrasonic sensor 19A to the virtual object is 1m, the timing of sending a detection wave from the second ultrasonic sensor 26 is controlled so that the distance detected in virtual sensor 22A is 1m. Thus, an object (actually a virtual object) is detected at a position 1m from ultrasonic sensor 19A. Similarly, virtual sensor 22B corresponds to ultrasonic sensor 19B, virtual sensor 22C corresponds to ultrasonic sensor 19C, and virtual sensor 22D corresponds to ultrasonic sensor 19D. The same applies to the remaining virtual sensors 22E to 22L.

[0071] In addition, vehicle 2 Figure 6 In addition to the constituent elements shown, there are also basic constituent elements of vehicle 2, but only the structure related to the control of autonomous driving assistance and the control related to the structure will be described.

[0072] Next, the method for switching sensor information input to the driver assistance ECU 20 by the sensor switching switch 21 will be explained. Figure 9 As shown, in the vehicle motion simulator system 1 of the first embodiment, there are a first input path for inputting the detection information of virtual sensors 22A to 22L to the driving assistance ECU 20, and a second input path for inputting the detection information of ultrasonic sensors 19A to 19L, which are actual sensors, to the driving assistance ECU 20.

[0073] The sensor switching switch 21 can switch between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L for autonomous driving assistance in the driver assistance ECU 20 by selectively cutting off one of the first and second input paths. Furthermore, the switching control of the sensor switching switch 21 is performed by the simulator device 3. However, during the collection of driving results in the autonomous driving assistance system (i.e., during driving tests), the information used by the driver assistance ECU 20 for autonomous driving assistance is switched to the detection information of virtual sensors 22A-22L. After the collection of driving results is completed or during the preparation phase before the collection of driving results, the information used by the driver assistance ECU 20 for autonomous driving assistance is switched to the detection information of ultrasonic sensors 19A-19L. However, these switches can also be performed automatically, for example, based on the operation of the evaluator 4 on the operation unit 13 and the external terminal 11. Specifically, in cases where operation is performed based on the external terminal 11, for example, when the test start is selected on the external terminal 11, a signal is sent from the external terminal 11 to the simulator device 3. Based on the received signal, the simulator device 3 controls the sensor switching switch 21 to switch the information used by the autonomous driving assistance in the driver assistance ECU 20 from the detection information of the ultrasonic sensors 19A-19L to the detection information of the virtual sensors 22A-22L. On the other hand, when the test end is selected on the external terminal 11, a signal is sent from the external terminal 11 to the simulator device 3. Based on the received signal, the simulator device 3 controls the sensor switching switch 21 to switch the information used by the autonomous driving assistance in the driver assistance ECU 20 from the detection information of the virtual sensors 22A-22L to the detection information of the ultrasonic sensors 19A-19L.

[0074] Next, the details of the simulator device 3 in the vehicle motion simulator system 1 will be explained. Figure 10 This is a block diagram illustrating the structure of the simulator device 3 according to this embodiment.

[0075] like Figure 10 As shown, the simulator device 3 has a control unit, namely a virtual control ECU (electronic control unit) 40, which generates the driving environment during driving tests, controls the virtual sensors 22A to 22L, analyzes, evaluates, and outputs the action results of the driving test of the vehicle 2, and performs various calculations and processing, as well as the aforementioned virtual sensors 22A to 22L.

[0076] The virtual control ECU 40 is an electronic control unit that performs overall control of the simulator device 3. It has a CPU 41 that serves as both a computing and control unit, and a RAM 42 that acts as working memory when the CPU 41 performs various calculations. In addition to the control program, it also records the motion evaluation program (described later). Figure 11 The system includes internal storage devices such as a ROM 43 and a flash memory 44 storing programs read from the ROM 43. Furthermore, the virtual control ECU 40, together with the aforementioned driving assistance ECU 20, has various units serving as processing algorithms. For example, a virtual object configuration unit configures virtual objects in the actual scene. A driving result collection unit collects the driving results of the vehicle 2, which has virtual sensors 22A-22L for detecting virtual objects. A sensor switching unit switches driving assistance information for the vehicle between the detection information from the virtual sensors 22A-22L and the detection information from the ultrasonic sensors 19A-19L.

[0077] In addition, the virtual control ECU 40, via a vehicle network such as CAN, connects not only to the aforementioned operating unit 13, vehicle display 5 (or VR head-mounted display 8), speaker 15, front camera 16, rear camera 17, side cameras 18A, 18B, ultrasonic sensors 19A-19L, driver assistance ECU 20, and sensor switch 21, but also to various sensors 47 used to detect vehicle behavior, such as vehicle speed sensor, wheel speed sensor, acceleration sensor, gyroscope sensor, steering sensor, and gear position sensor. It can detect the vehicle's current position, orientation, and current behavior based on the detection results of these sensors 47. Furthermore, the virtual control ECU 40 provides the vehicle 2 with a virtual driving environment for driving tests. On the other hand, it can obtain, analyze, evaluate, and output the action results of the autonomous driving assistance system based on the vehicle's behavior during driving tests. Similarly, it generates real-scene images 6 of the actual scene displayed on the aforementioned vehicle display 5, VR head-mounted display 8, and external terminal 11 based on the vehicle's behavior during driving tests. Figures 2-4 ), output images to these display devices.

[0078] Additionally, the flash memory 44 includes driving environment information 45 and driving result database 46. The driving environment information 45 stores the driving environment for the vehicle 2 driving test, pre-set by the evaluator 4. Specifically, it contains the types and locations of virtual objects configured in the actual scene, which are input in advance by the evaluator 4. Furthermore, multiple driving environment patterns can be pre-stored.

[0079] On the other hand, the driving result DB46 is a storage unit used to cumulatively store the vehicle's control content, driving trajectory, and behavior history during the driving test. Furthermore, after the driving test of vehicle 2 is completed, the simulator device 3 evaluates and outputs the vehicle 2's control content, driving trajectory, etc., stored in the driving result DB46.

[0080] Next, based on Figure 11 The motion evaluation program executed in the simulator device 3 that constitutes the vehicle motion simulator system 1 having the above-described structure will be described. Figure 11 This is a flowchart of the motion evaluation procedure involved in this embodiment. The motion evaluation procedure is executed when the simulator device 3 receives a corresponding program initiation operation, constructs a virtual driving environment, and evaluates the quality of the vehicle's automated driving assistance system based on the results of driving tests. Furthermore, the program initiation operation can be triggered, for example, by an evaluator 4 operating the operation unit 13 while riding in the vehicle, or by an evaluator 4 outside the vehicle performing a prescribed operation on an external terminal 11. The following... Figure 11 The program shown in the flowchart is stored in the RAM42, ROM43, etc. of the simulator device 3 and is executed by the CPU41.

[0081] First, in step (hereinafter referred to as S) 1, the CPU 41 of the simulator device 3 performs various initial setting processes before executing the driving test. For example, it confirms the operation of the virtual sensors 22A to 22L.

[0082] Next, in S2, CPU 41 reads the setting information of the driving environment for conducting driving tests on vehicle 2 from flash memory 44. The driving environment includes, for example, the types and locations of virtual objects configured for the actual scene.

[0083] The driving environment is basically preset and stored in the simulator device 3 through input operations by the evaluator 4. Multiple driving environments can be set, allowing for the selection of any environment to be used in the current driving test. Furthermore, the driving environment can be set to, for example, driving on public roads or driving in a parking lot. Additionally, "virtual objects" are not limited to three-dimensional objects; they also include lane lines, road markings, etc. For example, in a parking lot driving test, parking area markings drawn on the parking lot road surface, other vehicles parked in the parking space, pedestrians moving in the parking lot, and other vehicles traveling along the passageways in the parking lot are set as virtual objects. Moreover, a movement plan (at what speed and how) is set for moving virtual objects.

[0084] Next, in S3, CPU41 constructs a virtual driving environment based on the setting information read in S2 above, representing the actual scene in which the vehicle is currently located. That is, it configures virtual objects of a set type at set locations in the actual scene. Furthermore, regarding moving virtual objects, it moves them according to a set plan.

[0085] Subsequently, in S4, CPU41 operates sensor switching switch 21 to switch the information used for autonomous driving assistance in the driving assistance ECU 20 to the detection information of virtual sensors 22A-22L. Furthermore, as described above, sensor switching switch 21 switches between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L by selectively cutting off one of the first and second input paths, thereby switching the information used for autonomous driving assistance in the driving assistance ECU 20 between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L. Figure 9 The switching of sensor information based on sensor switch 21 can be performed automatically upon the start of the driving test, or it can be triggered by receiving a specified operation from the evaluator 4. For example, when the test start is selected on the external terminal 11, a signal can be sent from the external terminal 11 to the simulator device 3, and the simulator device 3 can operate sensor switch 21 based on the received signal.

[0086] Furthermore, prior to the switch in S4 described above, the information used by the driver assistance ECU 20 for autonomous driving assistance is the detection information from actual sensors, namely ultrasonic sensors 19A to 19L. Therefore, during the preparation phase before starting the driving test, vehicle 2 can perform autonomous driving assistance based on the detection information from actual sensors, namely ultrasonic sensors 19A to 19L. For example, during the movement of vehicle 2 to the starting position of the driving test, it can avoid actual obstacles such as walls and other vehicles and move using autonomous driving.

[0087] Subsequently, in S5, CPU41 begins driving tests of vehicle 2. When the driving test begins, vehicle control based on automated driving assistance is initiated in vehicle 2. For example, the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles are constantly detected. Automatic vehicle control of the steering system, drive system, brakes, etc., is then implemented to drive along the generated driving trajectory at a speed according to the same generated speed plan in the real-world scenario. Specifically, when evaluating parking assistance using a parking lot as the driving environment, the following vehicle control is automatically performed: using sensor detection information to confirm the parking space and its surrounding conditions, and calculating the parking trajectory up to the parking space, the vehicle enters the parking space along the calculated parking trajectory and completes parking.

[0088] During driving based on the aforementioned autonomous driving assistance, the CPU 41 of the simulator device 3 outputs the detection results of virtual objects detected by the virtual sensors 22A-22L on the vehicle 2 in the actual driving scenario to the driving assistance ECU 20 controlling the autonomous driving assistance on the vehicle 2 side. Specifically, the CPU 41 determines the current position and orientation of the vehicle based on information from various sensors 47 installed on the vehicle, such as the vehicle speed sensor, wheel speed sensor, acceleration sensor, gyroscope sensor, steering sensor, and gear sensor. Based on the determined current position and orientation of the vehicle and the position of the virtual objects configured in the driving environment constructed in S3 above, it determines the relative position of the virtual objects relative to the vehicle 2. Then, based on the determined relative position of the virtual objects relative to the vehicle 2 (more specifically, the ultrasonic sensors 19A-19L of the vehicle), it controls the timing of sending detection waves from the second ultrasonic sensors 26 of the virtual sensors 22A-22L, thereby allowing the virtual sensors 22A-22L to detect virtual objects that do not actually exist.

[0089] Next, in S6, the CPU 41 acquires real-time real-scene images of the actual scene of the object to be displayed on the in-vehicle display 5, the VR head-mounted display 8, or the external terminal 11, based on the images captured by the front camera 16, the rear camera 17, and the side cameras 18A and 18B. Furthermore, the real-scene images can be the real-scene images captured by the front camera 16, the rear camera 17, and the side cameras 18A and 18B themselves, or they can be, for example, images synthesized from images of multiple cameras or images with viewpoint transformations (e.g., overhead images, bird's-eye view images).

[0090] Furthermore, the real-time scene image of the actual object displayed on the external terminal 11 may not be set to an image captured by the camera of the vehicle 2, but rather an image captured by the camera of the external terminal 11. Alternatively, it may be set to an image captured by a camera located in the actual scene. Furthermore, the following processing is performed on a per-frame basis for the image acquired in S6 described above.

[0091] Next, in S7, the CPU 41 determines the position of the virtual object in the real-world image of the actual scene acquired in S6 by converting the configuration coordinates (absolute coordinates) of the virtual object in the driving environment constructed in S3 into the coordinate system (relative coordinates) of the real-world image of the actual scene acquired in S6. For example, when displaying the real-world image of the actual scene captured by the front camera 16 on the vehicle display 5, the position of the virtual object in the real-world image of the actual scene displayed on the vehicle display 5 can be determined by converting the configuration coordinates (absolute coordinates) of the virtual object into the coordinate system (relative coordinates) of the front camera 16. On the other hand, when performing VR display using the VR headset 8, the position of the virtual object in the real-world image of the actual scene displayed on the VR headset 8 can be determined by converting the configuration coordinates (absolute coordinates) of the virtual object into the coordinate system (relative coordinates) of VR (i.e., the user's viewpoint).

[0092] Next, in S8, CPU 41 synthesizes an image 7 representing a virtual object from the real-world image of the actual scene acquired in S6, at the position determined in S7. As the image 7 representing the virtual object, for example, 3D model images are pre-stored in flash memory 44 for each type of virtual object, such as vehicles or people. Then, based on reading the model image of the corresponding type from flash memory 44, CPU 41 rotates and scales it to match the virtual object positioned in the actual scene, and synthesizes it into the real-world image. For virtual objects with orientation, such as vehicles, the orientation is also matched to the constructed driving environment. Furthermore, texture mapping is performed on the surface of the model image. Regarding the effect of the texture image, it is preferable to consider the orientation of the configuration and the surrounding brightness, the position of light sources (e.g., sunlight, streetlights), etc., when configuring the model image in the actual scene, to depict it in a way that does not appear unnatural. The synthesized model image can be an opaque image with 0% transmittance or a semi-transparent image.

[0093] Next, in S9, CPU 41 outputs a real-time live-action image of the actual scene, which is composited with the image 7 representing the virtual object, to the in-vehicle display 5, the VR headset display 8, or the external terminal 11, and displays it on each display device. As a result, the evaluator 4 can visually confirm that the image 7 representing the virtual object is superimposed at the location where the virtual object is located. Figure 2 , Figure 3 Furthermore, as described above, a real-world image of the actual scene, superimposed with the image 7 representing the virtual object, can also be displayed on both the interior and exterior rearview mirrors. Figure 4 ).

[0094] Next, in S10, CPU41 determines whether the driving test has been completed. For example, when evaluating parking assistance using a parking lot as the driving environment, the driving test is considered complete by parking the vehicle into the parking space in P gear. Alternatively, the driving test can be completed based on whether a specified distance or a specified time has been traveled since the start of the test.

[0095] Furthermore, if the driving test is determined to be completed (S10: Yes), proceed to S11. Conversely, if the driving test is determined to be incomplete (S10: No), return to S5 and continue the driving test of vehicle 2.

[0096] In S11, the CPU41 operates the sensor switching switch 21 to switch the information used for autonomous driving assistance in the driving assistance ECU 20 from the information of the virtual sensors 22A-22L to the detection information of the ultrasonic sensors 19A-19L, which are the actual sensors. Furthermore, as described above, the sensor switching switch 21 switches between the detection information of the virtual sensors 22A-22L and the detection information of the ultrasonic sensors 19A-19L by selectively cutting off one of the first and second input paths, thereby switching the information used for autonomous driving assistance in the driving assistance ECU 20 between the detection information of the virtual sensors 22A-22L and the detection information of the ultrasonic sensors 19A-19L. Figure 9 The switching of sensor information based on sensor switch 21 can be performed automatically upon completion of the driving test, or it can be triggered by receiving a specified operation from the evaluator 4. For example, when the test is selected to end on the external terminal 11, a signal can be sent from the external terminal 11 to the simulator device 3, and the simulator device 3 can operate sensor switch 21 based on the received signal.

[0097] As a result, after completing the driving test, vehicle 2 can perform autonomous driving assistance based on the detection information from actual sensors, namely ultrasonic sensors 19A to 19L. For example, during the movement of vehicle 2 from the end of the driving test to the vehicle's storage location, it can avoid actual obstacles such as walls and other vehicles and move using autonomous driving. After that, the action evaluation procedure ends.

[0098] When a vehicle driving test is conducted using a motion evaluation program, the analysis of the driving results of vehicle 2 is configured to perform either "real-time analysis" or "post-analysis." "Real-time analysis" monitors vehicle 2 during the driving test and automatically detects and outputs any adverse conditions that occur. On the other hand, "post-analysis," after the driving test of vehicle 2 has ended, evaluates and outputs the control data, driving trajectory, etc., of vehicle 2 stored during the driving test based on the detection data from virtual sensors 22A to 22L.

[0099] Furthermore, CPU 41 can also output an evaluation window containing information about adverse conditions that occurred after the driving test to an external display. In addition, the evaluation window displays the time when the adverse condition occurred, along with the information about the adverse condition. As a result, evaluator 4 can understand at what time and what kind of adverse condition occurred in the autonomous driving assistance system. Moreover, CPU 41 can also be configured to retain the real-world image of the actual scene output in S9 (including the image 7 of the virtual object) as an image log, extract the image from the image log that was output when it was determined that an adverse condition occurred during the operation of the autonomous driving assistance system, and make it available to evaluator 4. Based on the output driving test results, evaluator 4 can perform program corrections for the autonomous driving assistance system, etc.

[0100] As detailed above, in the vehicle motion simulator system 1, the simulation method of the vehicle motion simulator system 1, and the computer program executed by the vehicle motion simulator system 1 according to this embodiment, virtual objects are configured in the actual scene (S3). The vehicle 2, equipped with virtual sensors 22A-22L for detecting virtual objects, performs autonomous driving assistance-based driving in the actual scene based on the detection information from the virtual sensors 22A-22L (S5). The driving results of the vehicle are collected and evaluated. On the other hand, the actual scene where an image 7 representing the virtual object is superimposed at the location where the virtual object is configured is visually confirmed (S9). Therefore, in autonomous driving assistance driving tests conducted with virtual objects configured in the actual scene, occupants and observers can visually confirm the actual scene with the virtual objects configured in real time. As a result, the positional relationship between the vehicle and the virtual object becomes clear, and it is also possible to evaluate whether the vehicle's behavior is appropriate from a human perspective.

[0101] Furthermore, at least one of the following is provided: an in-vehicle display 5 that is mounted on the vehicle 2 and can be visually confirmed by the occupants of the vehicle 2; a VR head-mounted display 8; and an external terminal 11 located outside the vehicle 2 and can be visually confirmed by an observer outside the vehicle. By displaying a captured image of the actual scene, in which an image 7 representing a virtual object is synthesized at the location where the virtual object is located, on the in-vehicle display 5 and the VR head-mounted display 8, the occupants of the vehicle can visually confirm the actual scene. By displaying a captured image of the actual scene, in which an image 7 representing a virtual object is synthesized at the location where the virtual object is located, on the external terminal 11, the observer can visually confirm the actual scene. Therefore, both occupants of the vehicle and observers monitoring from the outside without being in the vehicle can visually confirm the actual scene in real time.

[0102] Furthermore, since it is possible to switch between a first visual confirmation mode that visually confirms the actual scene included in the field of view from the vehicle's perspective and a second visual confirmation mode that visually confirms the actual scene included in the field of view from a position different from the vehicle's perspective, it is possible to visually confirm the actual scene with virtual objects configured in real time from various viewpoints in the actual scene without a fixed viewpoint.

[0103] In addition, at least one of the interior and exterior rearview mirrors configured in the vehicle overlaps the image 7 representing the virtual object in the actual scene reflected or displayed by the interior and exterior rearview mirrors, so that the virtual object configured in the actual scene can be visually confirmed in a more realistic manner.

[0104] In addition, virtual objects are configured in the actual scene (S3), so that the vehicle 2, which has virtual sensors 22A to 22L for detecting virtual objects, can drive in the actual scene based on the detection information of virtual sensors 22A to 22L (S5), and collect the driving results of the vehicle. On the other hand, a sensor switching switch 21 is provided to switch the information for autonomous driving assistance between the detection information of virtual sensors 22A to 22L and the detection information of actual sensors. Therefore, compared with the past, preparation work and post-testing cleanup work can be performed more efficiently.

[0105] Furthermore, during the collection of driving results for evaluating the autonomous driving assistance system, the information used for autonomous driving assistance is switched to the detection information of virtual sensors 22A to 22L. After the collection of driving results is completed or during the preparation stage before the collection of driving results, the information used for autonomous driving assistance is switched to the detection information of ultrasonic sensors 19A to 19L, which are actual sensors (S4, S11). Therefore, when the vehicle moves to the starting position of the driving test and moves to the vehicle storage position after the driving test, it can avoid actual obstacles such as walls and other vehicles and move using autonomous driving.

[0106] Furthermore, since the information used for autonomous driving assistance is switched between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L based on signals input from outside the vehicle, the vehicle can also be moved to the starting position of the driving test and the driving test can be started through remote operation from outside.

[0107] In addition, the vehicle has: a driving assistance ECU 20, which calculates control information for driving based on autonomous driving assistance; a first input path, which inputs the detection information of virtual sensors 22A to 22L to the driving assistance ECU 20; and a second input path, which inputs the detection information of ultrasonic sensors 19A to 19L to the driving assistance ECU 20. By selectively cutting off one of the first and second input paths, the information used for autonomous driving assistance in the driving assistance ECU 20 can be switched between the detection information of virtual sensors 22A to 22L and the detection information of ultrasonic sensors 19A to 19L. Therefore, manual operations such as wiring replacement are not required, and the sensor information used for autonomous driving assistance can be switched by electric switch control.

[0108] [Second Implementation]

[0109] Next, based on Figure 12 The vehicle motion simulator system according to the second embodiment will be described below. Furthermore, in the following description, the same as described above... Figures 1 to 11 The same reference numerals as those in the first embodiment of the vehicle motion simulator system 1 indicate parts that are the same as or equivalent to the structure of the vehicle motion simulator system 1, etc., in the first embodiment described above.

[0110] The general structure of the vehicle motion simulator system according to this second embodiment is substantially the same as that of the vehicle motion simulator system 1 according to the first embodiment. Furthermore, the various control processes are also substantially the same as those in the vehicle motion simulator system 1 according to the first embodiment.

[0111] However, the difference lies in, such as Figure 9 As shown, the vehicle motion simulator system 1 according to the first embodiment selectively cuts off one of the first input path and the second input path by the sensor switching switch 21, thereby switching the information for autonomous driving assistance between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L. In contrast, the vehicle motion simulator system according to the second embodiment has a first driving assistance ECU that calculates control information for driving based on autonomous driving assistance by inputting the detection information of virtual sensors and a second driving assistance ECU that calculates control information for driving based on autonomous driving assistance by inputting the detection information of ultrasonic sensors 19A-19L. By switching to obtain control information from either the first driving assistance ECU or the second driving assistance ECU, the information for autonomous driving assistance is switched between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L.

[0112] The following describes the method for switching sensor information input to the driver assistance ECU via the sensor switching switch 21, relating to the vehicle motion simulator system in the second embodiment. Figure 12 As shown, in the vehicle motion simulator system of the second embodiment, the control unit, which calculates control information (control information of various drive units of the vehicle, such as steering wheel, brakes, accelerator, and transmission) for driving based on automated driving assistance, includes a first driving assistance ECU 101 and a second driving assistance ECU 102. Furthermore, it also includes input paths for inputting detection information from virtual sensors 22A-22L to the first driving assistance ECU 101 and input paths for inputting detection information from ultrasonic sensors 19A-19L (actual sensors) to the second driving assistance ECU 102. The first driving assistance ECU 101 calculates the control information for driving based on automated driving assistance based on the detection information from the virtual sensors 22A-22L, and the second driving assistance ECU 102 calculates the control information for driving based on automated driving assistance based on the detection information from the ultrasonic sensors 19A-19L (actual sensors).

[0113] The sensor switching switch 21, by switching the CAN connection of vehicle 2, can switch between the detection information of virtual sensors 22A-22L and the detection information of ultrasonic sensors 19A-19L when autonomous driving assistance is in operation. That is, if the first driver assistance ECU 101 is connected, autonomous driving assistance is performed based on the detection information of virtual sensors 22A-22L; if the second driver assistance ECU 102 is connected, autonomous driving assistance is performed based on the detection information of ultrasonic sensors 19A-19L. Furthermore, the switching control of the sensor switching switch 21 is performed by the simulator device 3, but similarly to the first embodiment, during the evaluation of the autonomous driving assistance system (i.e., during driving tests), it switches to perform autonomous driving assistance based on the detection information of virtual sensors 22A-22L, and after the evaluation is completed or during the preparation stage before the evaluation, it switches to perform autonomous driving assistance based on the detection information of ultrasonic sensors 19A-19L.

[0114] Furthermore, the present invention is not limited to the above-described embodiments, and various improvements and modifications can be made without departing from the spirit of the present invention.

[0115] For example, in the first and second embodiments, virtual sensors 22A to 22L are used instead of ultrasonic sensors 19A to 19L to detect virtual objects. However, instead of a ranging sensor like an ultrasonic sensor, a virtual sensor can be used as an alternative to a camera that uses image recognition to detect objects. In this case, the virtual sensor can detect virtual objects, for example, by using an image obtained by synthesizing the virtual object with an image that is an object for image recognition.

[0116] In addition, in the first and second embodiments, the information used for autonomous driving assistance is switched to the detection information of virtual sensors when the driving test begins, and the information used for autonomous driving assistance is switched to the detection information of actual sensors when the driving test ends. However, it is also possible to switch to the detection information of actual sensors only at the beginning and end of the driving test.

[0117] Furthermore, in the first and second embodiments... Figure 11 The execution entity of the motion evaluation program shown is the simulator device 3 mounted on the vehicle, but the simulator device 3 can also be located outside the vehicle. In this case, the simulator device 3 and the vehicle 2 can communicate wirelessly.

[0118] Furthermore, in both the first and second embodiments, the evaluation object in the driving test is an automated driving assistance system that assists the occupant in driving the vehicle by performing part or all of the occupant's driving operations on the vehicle side. Vehicle 2 is a vehicle equipped with the automated driving assistance system that is the evaluation object. However, vehicle 2 is not necessarily limited to a vehicle equipped with the aforementioned automated driving assistance system; it can also be a vehicle capable only of manual driving. Regarding such a vehicle capable only of manual driving, for example, a driving test is conducted during manual driving to evaluate whether driving assistance such as deceleration control and warnings for obstacles around the vehicle is appropriate. In this case, for example, vehicle control is not performed in S5 above, and the driver manually drives vehicle 2 in a real-world scenario. Furthermore, during the driving, the CPU 41 of the simulator device 3 outputs the detection results of virtual objects detected by the virtual sensors 22A to 22L on the vehicle 2 side to the ECU 20 on the vehicle side, providing driving assistance (e.g., deceleration control and warnings for obstacles). In addition, in S4 and S11 above, the information used in the driving assistance of the above-mentioned vehicle during driving based on manual driving is processed by switching between the detection information of virtual sensors and the detection information of actual sensors.

[0119] [Summary of this implementation method]

[0120] This embodiment has at least the following structure.

[0121] A vehicle motion simulator system (1) includes: a virtual object configuration unit (20) for configuring virtual objects in an actual scene; and a driving result collection unit (20) for collecting driving results of a vehicle (2) having virtual sensors (22A-22L) for detecting the virtual objects in the actual scene, wherein the vehicle has actual sensors (19A-19L) in addition to the virtual sensors, the actual sensors (19A-19L) are sensors for detecting objects actually existing around the vehicle, and the vehicle motion simulator system (1) includes a sensor switching unit (21) for switching driving assistance information for the vehicle between the detection information of the virtual sensors and the detection information of the actual sensors.

[0122] Based on this structure, preparation and post-test cleanup can be carried out more efficiently than before.

[0123] In addition, in this embodiment, it is preferred that the sensor switching unit (21) switches the driving assistance information for the vehicle (2) to the detection information of the virtual sensors (22A-22L) during the period when the driving result collection unit (20) collects the driving results, and switches the driving assistance information for the vehicle to the detection information of the actual sensors (19A-19L) after the collection of driving results by the driving result collection unit is completed or during the preparation stage before the collection of driving results.

[0124] According to this structure, when the vehicle moves to the starting position of the driving test and to the vehicle storage position after the driving test, it can move while avoiding actual obstacles such as walls and other vehicles.

[0125] In addition, in this embodiment, it is preferred that the sensor switching unit (20) switches between the detection information of the virtual sensors (22A-22L) and the detection information of the actual sensors (19A-19L) based on the signal input from outside the vehicle to provide information for driving assistance to the vehicle (2).

[0126] According to this configuration, the vehicle can also be moved to the starting position of the driving test via remote operation from an external source, and the driving test can then begin.

[0127] Explanation of reference numerals in the attached figures

[0128] 1 Vehicle motion simulator system; 2 Vehicle; 3 Simulator device; 4 Evaluator (observer, occupant); 5 In-vehicle display (in-vehicle display device); 6 Real-world image of actual scene; 7 Image of virtual object; 8 VR head-mounted display (in-vehicle display device); 9 Rearview mirror; 11 External terminal (external display device); 19A-19L Ultrasonic sensors (actual sensors); 20 Driver assistance ECU (control unit); 21 Sensor switching switch; 22A-22L Virtual sensors; 40 Virtual control ECU; 41 CPU; 101 First driver assistance ECU; 102 Second driver assistance ECU

Claims

1. A vehicle motion simulator system, wherein, have: The virtual object configuration unit configures virtual objects in the actual scene; as well as The driving result collection unit collects the driving results of the vehicle, equipped with virtual sensors for detecting the virtual objects, in the actual scene. In addition to the virtual sensors, the vehicle also has physical sensors, which are used to detect objects that actually exist around the vehicle. The vehicle motion simulator system has a sensor switching unit that switches between the detection information of the virtual sensor and the detection information of the actual sensor to provide information for driving assistance of the vehicle.

2. The vehicle motion simulator system according to claim 1, wherein, During the period when the driving results are collected by the driving results collection unit, the sensor switching unit switches the information used for driving assistance of the vehicle to the detection information of the virtual sensor. During the preparation phase after the driving results collection by the driving results collection unit has ended or before the driving results collection, the sensor switching unit switches the driving assistance information for the vehicle to the detection information of the actual sensors.

3. The vehicle motion simulator system according to claim 1 or 2, wherein, The sensor switching unit switches driving assistance information for the vehicle between the detection information of the virtual sensor and the detection information of the actual sensor based on signals input from outside the vehicle.

Images