Information presentation method, information presentation device, and recording medium

Abstract

The information prompting device (10) obtains information about the surrounding environment, including the surrounding objects that the vehicle (1) is in contact with, infers the state of the surrounding objects based on the information about the surrounding environment, generates an output signal of at least one of vibration signal, sound signal and image signal based on the state of the surrounding objects, and controls the output device (7-9) based on the output signal.

Description

Technical Field

[0001] This invention relates to information prompting methods, information prompting devices, and recording media. Background Technology

[0002] The following technology is known: acquiring information related to the driving or operation of a vehicle, generating a sound signal representing a sound corresponding to the driving of the vehicle based on the acquired information, and generating a vibration signal representing a vibration corresponding to the driving of the vehicle based on the sound signal (Patent Document 1).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2022-170255 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] However, the technology described in Patent Document 1 provides vibrations that correspond to the engine sound of a vehicle, which has the problem that vehicle users cannot intuitively grasp the surrounding environment, including objects around the vehicle that the vehicle comes into contact with or approaches.

[0008] The problem to be solved by the present invention is to provide a method, a device, and a recording medium for providing information prompts that enable vehicle users to intuitively grasp the status of the surrounding environment, including objects around the vehicle.

[0009] Technical solutions for solving technical problems

[0010] The present invention solves the above-mentioned problem by: obtaining information about the surrounding environment, including surrounding objects that the vehicle is in contact with; estimating the state of the surrounding objects based on the obtained information about the surrounding environment; generating at least one output signal among vibration signal, sound signal and image signal according to the estimated state of the surrounding objects; and controlling the output device based on the generated output signal.

[0011] The present invention solves the above-mentioned problem by: obtaining information about the surrounding environment, including the surrounding objects that the vehicle is approaching; estimating the category of the surrounding objects based on the obtained information about the surrounding environment; generating at least one output signal, either a vibration signal or a sound signal, corresponding to the estimated category of the surrounding objects; and controlling the output device based on the generated output signal.

[0012] Invention Effects

[0013] According to the present invention, vehicle users can intuitively grasp the condition of the surrounding environment, including objects around the vehicle. Attached Figure Description

[0014] Figure 1 This is a block diagram illustrating an example of the structure of the vehicle including the information display device of this embodiment.

[0015] Figure 2 This is a diagram illustrating an example of the structure of the database used in the information display device of this embodiment.

[0016] Figure 3 This is a diagram illustrating a specific example of the output information displayed by the information display method of this embodiment.

[0017] Figure 4 This is a flowchart illustrating the processing flow of the information prompting method executed by the information prompting device of this embodiment. Detailed Implementation

[0018] An embodiment of the information prompting device of the present invention will be described with reference to the accompanying drawings. Figure 1 This is a block diagram illustrating an example of the structure of the vehicle including the information display device of this embodiment. For example... Figure 1 As shown, the vehicle 1 includes a camera 2, a sonar sensor 3, a wheel speed sensor 4, a microphone 5, a vibration sensor 6, a sound output device 7, a vibration output device 8, an image output device 9, and an information prompting device 10. These devices are connected via an in-vehicle LAN such as CAN (Controller Area Network) for information transmission. In this embodiment, the information prompting device 10 obtains various information from the camera 2, sonar sensor 3, wheel speed sensor 4, microphone 5, and vibration sensor 6. Based on the generated output signals, the information prompting device 10 controls the sound output device 7, vibration output device 8, and image output device 9 to prompt output information to the user of the vehicle 1. The user includes occupants of the vehicle 1 such as the driver. The output information includes vibration, sound, and images. The audio output device 7, vibration output device 8, and image output device 9 are examples of the "output devices" described in the claims. In this embodiment, it is sufficient to have at least one of these output devices.

[0019] Camera 2 is installed on vehicle 1 and is a device for capturing images of the surrounding environment of vehicle 1. The images captured by camera 2 include surrounding objects present around vehicle 1. In this embodiment, camera 2 captures images of the surrounding environment, including surrounding objects that are in contact with or close to vehicle 1. Surrounding objects that are in contact with vehicle 1 are those that do not obstruct the movement of vehicle 1, such as the road surface around vehicle 1. Surrounding objects that are close to vehicle 1 include those located at a distance of less than a predetermined distance from vehicle 1. Surrounding objects that are close to vehicle 1 include, for example, walls, curbs, guardrails, utility poles, other vehicles, and other obstacles that obstruct the movement of vehicle 1. Camera 2 may also include a front camera, a rear camera, and / or a side camera of vehicle 1. Images captured by camera 2 are acquired at regular intervals and output to information display device 10.

[0020] Sonar sensor 3 is a sensor installed on vehicle 1 to detect the surrounding environment of vehicle 1. Sonar sensor 3 uses ultrasonic waves to detect objects approaching vehicle 1 and obtains the distance between vehicle 1 and the surrounding objects. Sonar sensor 3 may also include a front sonar sensor for detecting objects in front, a rear sonar sensor for detecting objects behind, and / or a side sonar sensor for detecting objects to the side. The detection information detected by sonar sensor 3 is output to information prompting device 10. Wheel speed sensor 4 detects the rotational speed of the wheels of vehicle 1. For example, wheel speed sensor 4 measures the rotational speed of the drive system of vehicle 1, such as the drive shaft. The wheel speed information of vehicle 1 detected by wheel speed sensor 4 is output to information prompting device 10. Furthermore, the vehicle speed can also be calculated based on the wheel speed detected by wheel speed sensor 4. The wheel speed and vehicle speed information are output as vehicle information to information prompting device 10.

[0021] Microphone 5 is a device for acquiring external sounds from the vehicle 1 as sound information. The sound information acquired by microphone 5 is output to information prompting device 10. Vibration sensor 6 is installed on the body of the vehicle 1 to detect vibrations of the vehicle 1. Vibration sensor 6 is, for example, a G-sensor (accelerometer) or a gyroscope sensor. The detection value detected by vibration sensor 6 is output to information prompting device 10.

[0022] The sound output device 7 is a device that outputs sound to a user. The sound output device 7 is a device installed inside the vehicle 1, such as a speaker. The sound output device 7 outputs sound based on a sound signal. The vehicle 1 may also have multiple sound output devices 7. In this case, the multiple sound output devices 7 are respectively installed in different locations inside the vehicle. For example, the sound output devices 7, viewed from the position of the seats inside the vehicle, are located on the left front side, right front side, left rear side, and right rear side. The installation position of each sound output device 7 corresponds to the position of surrounding objects relative to the vehicle 1. The sound output devices 7 installed on the left front side, right front side, left rear side, and right rear side correspond to the positions of surrounding objects relative to the vehicle 1 as left front side, right front side, left rear side, and right rear side, respectively. For example, if the position of the surrounding object relative to the vehicle 1 is on the left front side, the sound output device 7 located on the left front side inside the vehicle becomes the object of output control. Furthermore, in this embodiment, an example of installing four sound output devices 7 is described, but it is not limited to this; three or fewer, or five or more, sound output devices 7 may be installed in various locations.

[0023] The vibration output device 8 is a device that has a vibration mechanism and generates vibration. The vibration output device 8 generates vibration based on a vibration signal. The vibration output device 8 is disposed within the vehicle 1 in locations that the user comes into contact with. For example, locations that the user comes into contact with include the steering wheel, seats, and the floor under the seats. The vibration output device 8 is provided in the seat cushion. The vehicle 1 may also have multiple vibration output devices 8. When multiple vibration output devices 8 are provided on the steering wheel, the multiple vibration output devices 8 are arranged along the circumference of the steering wheel. For example, the positions of the vibration output devices 8 relative to the center of the steering wheel are the left and right sides of the steering wheel. The position of each vibration output device 8 corresponds to the position of surrounding objects relative to the vehicle 1. The vibration output devices 8 provided on the left and right sides of the steering wheel correspond to the positions of surrounding objects relative to the vehicle 1 being on the left and right, respectively. For example, if the surrounding object is on the left side relative to the vehicle 1, the vibration output device 8 located on the left side of the steering wheel becomes the object of output control.

[0024] When multiple vibration output devices 8 are provided in the seat, the multiple vibration output devices 8 are located at the left front, right front, left rear, and right rear of the seat cushion. The location of each vibration output device 8 corresponds to the location of surrounding objects relative to the vehicle 1. The vibration output devices 8 located at the left front, right front, left rear, and right rear of the seat cushion correspond to the locations of surrounding objects relative to the vehicle 1 as left front, right front, left rear, and right rear, respectively. For example, if the surrounding objects are located at the left front relative to the vehicle 1, the vibration output device 8 located at the left front of the seat cushion becomes the object of output control. Furthermore, when multiple vibration output devices 8 are provided on the floor surface under the seat, the multiple vibration output devices 8 are located at the left front, right front, left rear, and right rear of the floor surface. The location of each vibration output device 8 corresponds to the location of surrounding objects relative to the vehicle 1. The vibration output devices 8 located at the left front, right front, left rear, and right rear of the floor surface correspond to the positions of surrounding objects relative to the vehicle 1 as left front, right front, left rear, and right rear sides, respectively. For example, when the surrounding object is located at the left front side relative to the vehicle 1, the vibration output device 8 located at the left front of the floor surface becomes the object of output control. Furthermore, this embodiment describes an example of installing two or four vibration output devices 8 in each location, but it is not limited to this, and the number of vibration output devices 8 can be changed as needed.

[0025] Image output device 9 is a device for outputting images to a user. Image output device 9 is, for example, a display for displaying images, and is composed of a liquid crystal panel, an organic EL panel, etc. Image output device 9 is provided in the interior of the vehicle 1. For example, image output device 9 is located on the dashboard, for example, below the windshield, on the side forward of the driver's seat. The image displayed on image output device 9 is an image representing the surrounding environment including the vehicle 1. In this embodiment, an image representing the state of surrounding objects is superimposed on the image representing the surrounding environment. The display position of the image representing the state of surrounding objects on the image representing the surrounding environment corresponds to the position of the surrounding objects relative to the position of the vehicle 1. The images representing the state of surrounding objects displayed in the upper left, upper right, lower left, and lower right parts of the image representing the surrounding environment correspond to the positions of the surrounding objects relative to the position of the vehicle 1 as the left front side, right front side, left rear side, and right rear side, respectively. For example, if the surrounding object is a "slippery road surface", and the "slippery road surface" is located on the left front side relative to the vehicle 1, an image of "water ripples" is superimposed on the upper left part of the image representing the surrounding environment as an image representing the state of the surrounding objects.

[0026] The controller 100 is a controller that performs information prompting processing. The controller 100 consists of a CPU 11 that executes an information prompting program for performing information prompting processing, a ROM 12 that stores the information prompting program, and RAM 13 that functions as an accessible storage device. Furthermore, the ROM 12 is an example of the "recording medium" described in the claims. The storage location of the information prompting program is not limited to the ROM 12; it can be any computer-readable recording medium, and the recording medium can be located inside or outside the information prompting device 10.

[0027] As functional blocks, the controller 100 includes: an information acquisition unit 101 that acquires various types of information; an estimation unit 102 that estimates the surrounding environment based on the acquired information; a generation unit 103 that generates an output signal based on the estimated results of the surrounding environment; an output control unit 104 that controls an output device based on the output signal; and a recording unit 105 that records the information acquired in a specified surrounding environment. The CPU 11 implements these functions by executing an information prompting program recorded in the ROM 12. That is, the information prompting program causes the controller 100 to execute each process implemented by the information acquisition unit 101, the estimation unit 102, the generation unit 103, the output control unit 104, and the recording unit 105. Each functional block executes its function through software and hardware cooperation. Furthermore, the functional blocks of the controller 100 do not necessarily require... Figure 1 The function blocks shown are separate, and the number of function blocks is not limited.

[0028] The information acquisition unit 101 performs information acquisition processing to obtain various types of information. When information prompting is initiated, such as when the power to the vehicle 1 is turned on, the information acquisition processing is performed at regular intervals. The acquired information includes information about the surrounding environment of the vehicle 1 and vehicle information. The surrounding environment information includes, for example, information about objects that the vehicle 1 comes into contact with or approaches. During the information acquisition processing, the information acquisition unit 101 obtains information about the road surface around the vehicle 1 and / or information about surrounding objects such as obstacles around the vehicle 1 from sensors including the camera 2, which provide information about the surrounding environment. For example, the information obtained from the camera 2 includes images captured by photographing the surrounding environment of the vehicle 1. The images include the road surface around the vehicle 1. Additionally, the images include obstacles that impede the movement of the vehicle 1, such as walls, curbs, guardrails, and utility poles. Furthermore, the information acquisition unit 101 obtains information about the distance between the vehicle 1 and surrounding objects from the sonar sensor 3.

[0029] Additionally, the information acquisition unit 101 acquires vehicle information of the vehicle 1 from the wheel speed sensor 4. The vehicle information of the vehicle 1 includes information about the wheel speeds of the vehicle 1. Furthermore, the vehicle information of the vehicle 1 may also include information about the vehicle's overall speed.

[0030] Furthermore, when recording output information in a specified surrounding environment, the information acquisition unit 101 can acquire sound information from the microphone 5, vibration detection values ​​from the vibration sensor 6, and images of the road surface around the vehicle 1 captured by the camera 2, for each specified surrounding environment. For example, when the vehicle 1 is traveling on a gravel road, the microphone 5 and vibration sensor 6 acquire the sound and vibration of the vehicle 1 traveling on the gravel road and output them to the information prompting device 10. In addition, the camera 2 acquires images of the gravel road on which the vehicle 1 is traveling and outputs them to the information prompting device 10. The output information in the specified surrounding environment is pre-recorded by the recording unit 105 (described later) before the execution of the information prompting process.

[0031] In addition, in this embodiment, the information acquisition unit 101 can also acquire information about the surrounding environment in the area in front of the vehicle 1 through vehicle-to-vehicle communication with other vehicles around the vehicle 1 and the camera 2. The information acquisition unit 101 can also acquire information about the position of surrounding objects relative to the vehicle 1 as information about the surrounding environment. The position of the surrounding objects includes positions relative to the vehicle 1 at the left front side, right front side, left rear side, and right rear side.

[0032] The estimation unit 102 performs estimation processing based on the acquired information about the surrounding environment to estimate the surrounding environment. The estimated surrounding environment is the state of the surrounding objects that the vehicle 1 comes into contact with, such as the state of the road surface around the vehicle 1. The state of the road surface includes, for example, a slippery state, an icy state, a gravel state, a muddy state, or a state with fine sand, such as desert sand. Alternatively, the estimated surrounding environment can also be the category of surrounding objects that are close to the vehicle 1. Surrounding objects that are close to the vehicle 1 are obstacles, such as walls, curbs on road shoulders, guardrails, etc.

[0033] As an example of a method for estimating the state of the road surface, the estimation unit 102 estimates the state of the road surface based on information about the road surface surrounding the vehicle 1 obtained from the camera 2. For example, the estimation unit 102 performs image recognition processing on an image of the road surface surrounding the vehicle 1 to estimate the state of the road surface. Alternatively, the estimation unit 102 may also estimate the state of the road surface based on information about the road resistance calculated from the slip ratio of the wheels of the vehicle 1. In this embodiment, the information about the road resistance is pre-established to correspond to each state of the road surface. The slip ratio is calculated based on the rotational speed of the wheels obtained from the wheel speed sensor 4.

[0034] Furthermore, as a method for estimating the category of surrounding objects, the estimation unit 102 may also perform image recognition processing on the image of the surrounding objects obtained by the camera 2 to estimate the category of the surrounding objects. Additionally, in this embodiment, the estimation unit 102 may also estimate the surrounding environment in the area ahead based on information about the surrounding environment in the area ahead. For example, the estimation unit 102 may estimate the state of the road surface in the area ahead based on information about the road surface in the area ahead.

[0035] Furthermore, the estimation unit 102 can also estimate the state of the vehicle 1 based on the vehicle information of the vehicle 1. The state of the vehicle 1 includes, for example, a driving state when the vehicle 1 is moving and a stopped state when the vehicle 1 is stationary. In addition, the estimation unit 102 can also estimate the speed of the vehicle 1.

[0036] The generation unit 103 performs output signal generation processing based on the estimated surrounding environment. The output signal is a vibration signal, a sound signal, and an image signal representing the state of the surrounding environment; these are signals representing information about vibration, sound, and image corresponding to the surrounding environment, respectively. When the estimated surrounding environment is a predetermined environment, the generation unit 103 generates the output signal based on at least one of the vibration, sound, and image signals recorded in correspondence with the predetermined environment. Furthermore, in this embodiment, the output signal can be at least one of a vibration signal, a sound signal, and an image signal.

[0037] In this embodiment, vibrations, sounds, and images corresponding to each specified surrounding environment are pre-recorded in the database. The surrounding environment includes the conditions of various road surfaces, such as slippery surfaces, icy surfaces, gravel surfaces, muddy surfaces, and fine sand surfaces. The vibrations, sounds, and images corresponding to the specified surrounding environment represent the condition of that environment. For example, in the case of a "slippery surface," the database records the sound of water droplets falling, vibrations based on the sound of water droplets falling, and images of water ripples. This output information varies depending on the road surface condition; it only needs to evoke the user's perception of the road surface condition through vibrations, sounds, and images, such as vibrations, sounds, and images associated with the road surface condition.

[0038] In addition, in this embodiment, the database may also record the types of surrounding objects such as obstacles that approach the vehicle 1, as well as the vibrations, sounds, and images corresponding to those types of surrounding objects. For example, if the type of surrounding object is "wall," the database may record the scraping sound generated when the vehicle comes into contact with the wall, the vibration based on the scraping sound, and the image of the wall. This output information varies depending on the surrounding object; it is sufficient that the vibrations, sounds, and images evoke the user's perception of the type of surrounding object, such as vibrations, sounds, and images associated with the type of surrounding object.

[0039] Furthermore, in the generation process, the generation unit 103 can also generate an output signal based on the surrounding environment and the state of the vehicle 1. For example, such as Figure 2 As shown, in this embodiment, the database contains pre-recorded sounds, vibrations, and images for each combination of the surrounding environment and the vehicle's state. Figure 2 This is a diagram illustrating an example of the database structure used in the information display device of this embodiment. Figure 2 The database shown records the corresponding types of sounds, vibrations, and images for each combination of the surrounding environment and the vehicle's state. Sounds include, for example, sounds representing onomatopoeic words. Images can be still images or moving images (videos). For example, for the combination of a "slippery road surface" environment and a stationary vehicle, the database records the sound of falling water, vibrations based on the sound of falling water, and images of water ripples. Furthermore, for the combination of a "slippery road surface" environment and the vehicle being in motion, the database records a "splashing" sound, vibrations based on the "splashing" sound, and images of water ripples.

[0040] Here, an example of the generation process corresponding to the surrounding environment including objects in contact with the vehicle 1 will be described. An example of an object in contact with the vehicle 1 is the road surface on which the vehicle 1 is traveling. In the generation process, the generation unit 103 generates vibration signals, sound signals, and image signals based on the road surface condition. For example, if the estimation unit 102 estimates that the road surface around the vehicle 1 is a wet road surface, the generation unit 103 accesses a database, obtains the vibration, sound, and image corresponding to "wet road surface," and generates a vibration signal representing the obtained vibration, a sound signal representing the obtained sound, and an image signal representing the obtained image.

[0041] Next, an example of the generation process corresponding to the surrounding environment containing objects close to the vehicle 1 will be described. Examples of objects close to the vehicle 1 include walls and other similar objects. In the generation process, the generation unit 103 generates vibration signals, sound signals, and image signals based on the category of the surrounding objects. For example, if the estimation unit 102 estimates that the category of the surrounding object is "wall," the generation unit 103 accesses a database, obtains vibration, sound, and image corresponding to "wall," and generates a vibration signal representing the obtained vibration, a sound signal representing the obtained sound, and an image signal representing the obtained image. The generated sound signal is, for example, a signal representing the sound ("scratching" sound, etc.) produced when the vehicle 1 comes into contact with a wall.

[0042] Furthermore, during the generation process, an output signal corresponding to the surrounding environment in the forward region located in front of the vehicle 1 can also be generated. When the estimation unit 102 estimates the surrounding environment in the forward region, the generation unit 103 generates the output signal based on the estimated surrounding environment in the forward region. The specific generation method is the same as described above.

[0043] The output control unit 104 performs output control processing to control the output device based on the output signal. In this output control processing, the output control unit 104 controls the output of at least one of the following devices—the sound output device 7, the vibration output device 8, and the image output device 9—based on the output signal. Specifically, the output control unit 104 sends the generated output signal to the output device that is the object of output control. For example, if a vibration signal is generated by the generation unit 103, the output control unit 104 sends the vibration signal to the vibration output device 8. The vibration output device 8, upon receiving the vibration signal, generates vibration based on the vibration signal. As described above, the user of the vehicle 1 can intuitively grasp the condition of the surrounding environment, including objects that the vehicle 1 comes into contact with or approaches. In particular, in this embodiment, the condition of the surrounding environment is demonstrated in a way that allows the user to grasp the condition of the surrounding environment by reproducing the vibrations accompanying the contact between the vehicle 1 and surrounding objects.

[0044] In addition, when the generation unit 103 generates an output signal corresponding to the surrounding environment in the area ahead, the output control unit 104 can also control the output device based on the output signal before the vehicle 1 reaches the area ahead.

[0045] In this embodiment, the output control unit 104 can also determine the output device to be controlled from among multiple output devices located in various parts of the vehicle 1 based on the position of the surrounding object, and control the output of the determined output device. Specifically, the output control unit 104 sends an output signal to the output device located at a position corresponding to the position of the surrounding object. For example, if the surrounding object is located on the left front side relative to the vehicle 1, the output control unit 104 sends an output signal to the output device located at a position corresponding to the left front side (e.g., the vibration output device 8 located on the left side of the steering wheel). Furthermore, the output control unit 104 can also identify the position of the user's seat and output output information corresponding to the condition of the road surface in front of the door the user is entering or exiting. For example, the output control unit 104 outputs information indicating that the road surface condition is poor due to water accumulation or other reasons in front of the door the user is entering or exiting.

[0046] In addition, in this embodiment, the output control unit 104 can also change the output mode of sound, vibration, and image based on the vehicle speed of the vehicle 1. Changing the output mode can be, for example, changing the number and / or position of the output devices subject to output control. The output control unit 104 obtains the vehicle speed of the vehicle 1 and determines whether the obtained vehicle speed is above a predetermined speed. The predetermined speed is, for example, 20 km / h. When the vehicle speed of the vehicle 1 is above the predetermined speed, the output control unit 104 can also increase the number of sound output devices 7 and / or vibration output devices 8 subject to output control compared to when the vehicle speed of the vehicle 1 is not above the predetermined speed. For example, if the number of sound output devices 7 and vibration output devices 8 subject to output control is one each when the vehicle speed of the vehicle 1 is not above the predetermined speed, the output control unit 104 determines that the number of sound output devices 7 and vibration output devices 8 subject to output control when the vehicle speed of the vehicle 1 is above the predetermined speed is two or more. For example, the output control unit 104 may control not only the sound output device 7 on the left front side, but also the sound output device 7 on the left rear side, and may control not only the vibration output device 8 on the left front side, but also the vibration output device 8 on the left rear side. Furthermore, when the vehicle speed of the vehicle 1 is above a predetermined speed, the output control unit 104 may make the overlap range of the images representing the state of surrounding objects larger than the overlap range of the images representing the state of surrounding objects when the vehicle speed of the vehicle 1 is not above the predetermined speed.

[0047] Here, use Figure 3 Specific examples of the output information displayed by the information prompting method of this embodiment will be described. Figure 3 This diagram illustrates specific examples of the output information displayed by the information display method of this embodiment. Specific examples (A) to (E) of the output information are shown respectively, taking into account the positional relationship between the vehicle V1 and the surrounding object P (e.g., a wet road surface). Figure 3 In the example, the surrounding object P is located on the left front side relative to the vehicle V1 and is in contact with the vehicle V1.

[0048] In this vehicle V1 and the surrounding object P, Figure 3 In that positional relationship, the image output device 9 displays an image as shown in specific example (A). In the image displayed on the image output device 9, an image V1′ representing the vehicle V1 and an image P′ corresponding to the surrounding object P in contact with the vehicle V1 are superimposed on the image representing the environment surrounding the vehicle V1. Figure 3 In this example, the image output device 9 includes four displays: a front display Df, a left-side display Dl, a right-side display Dr, and a bottom display Db. The image output device 9 displays images of the vehicle V1 and its surrounding environment on these displays. For example, display Db displays an image of the road surface surrounding the vehicle V1, superimposed with an image V1′ representing the vehicle V1. Displays Dr and Dl display images representing the surrounding environment to the left and right of the vehicle V1, respectively. The front display Df displays an image including the front of the vehicle V1, the road surface in front of the vehicle V1, and the surrounding environment to the left and right. In specific example (A), an image P′ representing a wet road surface is superimposed on the image representing the surrounding environment, as an image corresponding to a surrounding object P that contacts the vehicle V1. In particular, since the surrounding object P contacts the left front side of the vehicle V1, image P′ is displayed on the left front side relative to image V1′ of the vehicle V1 in the image representing the surrounding environment. Furthermore, the image displayed on the image output device 9 is not limited to such images; it can be any image that includes the surrounding environment of the vehicle V1, for example, it can be an image showing a top view of the vehicle V1.

[0049] Next, a specific example of sound output will be described. As shown in specific example (B), sound output device 7 outputs sound. Specific example (B) represents the interior of the vehicle V1. In specific example (B), sound output device 7 is a speaker located on the left front side relative to the occupant's seat assembly S inside the vehicle. The sound output by sound output device 7 is a sound corresponding to the state of the surrounding object P that is in contact with the vehicle V1, such as the sound of water ripples in a puddle when the vehicle is driving on a wet road surface. Sound output device 7 outputs sound from the left front side.

[0050] Next, specific examples of vibration output will be described. As shown in specific examples (C) to (E), the vibration output device 8 generates vibration. Specific example (C) is an example of a vibration output device 8 installed on the steering wheel SH of the vehicle V1. In specific example (C), the vibration output device 8 is located on the left side of the steering wheel. The vibration output device 8 generates vibration corresponding to the state of the surrounding object P that is in contact with the vehicle V1. For example, the generated vibration is a vibration corresponding to the sound of ripples in a puddle.

[0051] Specific example (D) is an example of a vibration output device 8 installed on the floor surface F of the driver's seat of the vehicle V1. In specific example (D), the vibration output device 8 is located at the left front part of the floor surface F of the driver's seat. The vibration output device 8 generates vibrations corresponding to the state of contact between the vehicle V1 and the surrounding object P. Specific example (E) is an example of a vibration output device 8 installed on the seat S1 of the vehicle V1. In specific example (E), the vibration output device 8 is located at the left front part of the seat S1. The vibration output device 8 generates vibrations corresponding to the state of contact between the vehicle V1 and the surrounding object P.

[0052] The recording unit 105 performs recording processing to record output information according to each specified surrounding environment. For each specified surrounding environment, the recording unit 105 pre-establishes a correspondence between the vibration detected by the vibration sensor 6, the sound detected by the microphone 5, and the images obtained by photographing the road surface around the vehicle 1, and records them accordingly. For example, when the vehicle 1 is driving in the specified surrounding environment, the recording unit 105 records the information obtained from the vibration sensor 6, microphone 5, and camera 2 as output information. Thus, for each surrounding environment, the controller 100 pre-acquires the actual vibration, sound, and actual images generated by the vehicle while driving in that environment and records them as output information in the database. Furthermore, when the vehicle 1 is driving in the same surrounding environment during the execution of information prompt processing, the controller 100 outputs the output information recorded in the database. This provides the user with a more realistic sense of the surrounding environment, allowing the user to intuitively grasp the condition of the surrounding environment.

[0053] For example, when vehicle 1 is traveling on a wet and slippery surface, vibration sensor 6 detects the vibration experienced by vehicle 1 while traveling on the wet and slippery surface, and sends the detected value along with information about the surrounding environment ("wet and slippery surface") to recording unit 105. Recording unit 105 records the information about the surrounding environment ("wet and slippery surface") and the detected value of vibration sensor 6. Additionally, for example, when vehicle 1 is traveling on a wet and slippery surface, microphone 5 acquires the sound generated while traveling on the wet and slippery surface, and sends the acquired sound information along with information about the surrounding environment ("wet and slippery surface") to recording unit 105. Recording unit 105 records the information about the surrounding environment ("wet and slippery surface") and the sound information from microphone 5. Furthermore, for example, when vehicle 1 is traveling on a wet and slippery surface, camera 2 films the wet and slippery surface, and sends the filmed image information along with information about the surrounding environment ("wet and slippery surface") to recording unit 105. Recording unit 105 records the information about the surrounding environment ("wet and slippery surface") and the image information from camera 2. Furthermore, in this embodiment, actual vibrations, sounds, and images in the surrounding environment are recorded and used as output information. However, the output information is not limited to this. It can also be vibrations, sounds, and images that artificially simulate actual vibrations, sounds, and images, or vibrations, sounds, and images that represent the conditions of the surrounding environment.

[0054] Next, refer to Figure 4 This describes an example of an information prompting method performed by the information prompting device of this embodiment. Figure 4 This is a flowchart illustrating the processing flow of the information prompting method executed by the information prompting device of this embodiment. In this embodiment, when the information prompting device 10 is activated and information prompting processing begins, such as when the power to the vehicle 1 is turned on, the process starts from step S11.

[0055] In step S11, if the controller 100 has started information prompting processing, it proceeds to step S12. If the controller 100 has not started information prompting processing, it terminates the control flow. In step S12, the controller 100 acquires information about the surrounding environment, including objects located around the vehicle 1. In step S13, the controller 100 estimates the surrounding environment of the vehicle 1 based on the information about the surrounding environment acquired in step S12. The surrounding environment includes the road surface and obstacles around the vehicle 1. In step S14, the controller 100 estimates the distance between the vehicle 1 and surrounding objects. For example, it estimates the distance between the vehicle 1 and obstacles. In step S15, the controller 100 acquires the speed of the vehicle 1.

[0056] In step S16, the controller 100 generates at least one output signal, including a vibration signal, a sound signal, and an image signal, based on the estimated surrounding environment. For example, based on the distance between the vehicle 1 and surrounding objects, the controller 100 generates an output signal corresponding to the type of surrounding object if there is an object approaching the vehicle 1. Additionally, the controller 100 determines the output method for outputting the output information based on the vehicle 1's speed. In step S17, the controller 100 controls the output of at least one of the sound output device 7, the vibration output device 8, and the image output device based on the generated output signal. Specifically, the controller 100 sends the generated output signal to each output device. Each output device, upon receiving the output signal from the controller 100, performs output based on the output signal. In step S18, the controller 100 determines whether the information prompting process is complete. If the information prompting process is complete, the controller 100 ends the control flow. If the information prompting process is not complete, the controller 100 returns to step S12 and repeats the following process.

[0057] also, Figure 4 The flowchart shown illustrates one example of the information prompting method of this embodiment. Including all steps is not mandatory; appropriate steps can be selected as needed. For example, estimating the distance between vehicle 1 and surrounding objects in step S14 and obtaining the speed of vehicle 1 in step S15 are not mandatory and can be selected as needed.

[0058] As described above, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller acquires information about the surrounding environment, including objects in contact with the vehicle, infers the state of the surrounding objects based on the acquired information, generates at least one output signal among vibration signal, sound signal, and image signal based on the inferred state of the surrounding objects, and controls the output device based on the generated output signal. Thus, the vehicle user can intuitively grasp the condition of the surrounding environment, including the surrounding objects.

[0059] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller acquires information about the road surface around the vehicle as information about the surrounding environment, infers the state of the road surface based on the acquired information about the road surface around the vehicle, generates a vibration signal corresponding to the inferred state of the road surface, and controls the output device based on the generated vibration signal. Thus, the vehicle user can intuitively grasp the state of the road surface around the vehicle.

[0060] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller obtains information about the road surface surrounding the vehicle from sensors, including a camera, mounted on the vehicle as information about the surrounding environment. Therefore, the condition of the surrounding environment can be inferred based on the information about the road surface surrounding the vehicle.

[0061] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller, acting as the surrounding environment, estimates the state of the road surface based on information about the road surface around the vehicle obtained from sensors, including a camera, mounted on the vehicle, and / or information about the road surface resistance calculated based on the wheel slip ratio of the vehicle. The slip ratio is calculated based on the wheel rotation speed. Therefore, the state of the road surface around the vehicle can be estimated as the vehicle's surrounding environment.

[0062] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller generates at least one output signal among vibration signals, sound signals, and image signals that represents the estimated condition of the surrounding environment. Thus, the user can intuitively grasp the condition of the surrounding environment, including objects in contact with the vehicle.

[0063] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller, based on the generated output signal, controls the output of at least one of the following devices: a sound output device, a vibration output device, and an image output device. The sound output device and the image output device are installed inside the vehicle, and the vibration output device is disposed inside the vehicle at a location accessible to the user. Thus, visual, auditory, and tactile information can be used to prompt the user about the surrounding environment.

[0064] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller pre-establishes a correspondence and records for each specified surrounding environment, including vibrations detected by a vibration sensor mounted on the vehicle, sounds detected by a microphone mounted on the vehicle, and images obtained by photographing the road surface around the vehicle. When the estimated surrounding environment is the specified surrounding environment, an output signal is generated based on at least one of the vibrations, sounds, and images recorded in correspondence with the specified surrounding environment. Thus, the user can be prompted about the surrounding environment using vibrations, sounds, and road surface images actually generated in the same environment as the estimated surrounding environment.

[0065] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller obtains information about the surrounding environment in the area in front of the vehicle by communicating with other vehicles around the vehicle and using a camera mounted on the vehicle. Based on the obtained information about the surrounding environment in the area in front, the controller infers the surrounding environment in the area in front, generates an output signal based on the inferred surrounding environment, and controls the output device based on the generated output signal before the vehicle reaches the area in front. Thus, the user of the vehicle can be informed in advance of the condition of the surrounding environment in front of the vehicle.

[0066] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller acquires information about the position of surrounding objects relative to the vehicle as information about the surrounding environment. Based on the position of the surrounding objects, it determines the output device to be controlled from among multiple output devices located within the vehicle, and controls the output of the determined output device. Thus, the vehicle user can intuitively grasp the position of surrounding objects in contact with the vehicle.

[0067] Furthermore, in the information prompting method, information prompting device, and recording medium of this embodiment, the controller acquires information about the surrounding environment, including objects that the vehicle is approaching, infers the category of the surrounding objects based on the acquired information, generates at least one output signal, either a vibration signal or a sound signal, corresponding to the inferred category of the surrounding objects, and controls the output device based on the generated output signal. Thus, the vehicle user can intuitively grasp the status of the surrounding environment, including objects that the vehicle is approaching.

[0068] Furthermore, the recording medium in this embodiment is a recording medium for recording the information prompting program executed by the controller. The information prompting program causes the controller to perform a process including the following steps: acquiring information about the surrounding environment, including surrounding objects that the vehicle is in contact with; estimating the state of the surrounding objects based on the acquired information about the surrounding environment; generating at least one output signal among vibration signal, sound signal, and image signal based on the estimated state of the surrounding objects; and controlling the output device based on the generated output signal. Thus, the vehicle user can intuitively grasp the condition of the surrounding environment, including the surrounding objects.

[0069] Furthermore, the embodiments described above are provided for ease of understanding of the present invention and are not intended to limit the present invention. Therefore, the essence of the elements disclosed in the above embodiments also includes all design modifications and equivalents that fall within the technical scope of the present invention.

[0070] Explanation of reference numerals in the attached figures

[0071] 1: This vehicle

[0072] 2: Camera

[0073] 3: Sonar sensor

[0074] 4: Wheel speed sensor

[0075] 5: Microphone

[0076] 6: Vibration sensor

[0077] 7: Sound output device

[0078] 8: Vibration output device

[0079] 9: Image output device

[0080] 10: Information Prompt Device

[0081] 100: Controller

[0082] 101: Information Acquisition Department

[0083] 102: Presumption Department

[0084] 103: Production Department

[0085] 104: Output Control Unit

[0086] 105: Recording Department

Claims

1. An information prompting method, executed by a controller, characterized in that, The controller acquires information about the surrounding environment, including objects in contact with the vehicle, and estimates the state of the surrounding objects based on the acquired information. Based on the estimated state of the surrounding objects, it generates at least one output signal among vibration signal, sound signal, and image signal, and controls the output device based on the generated output signal.

2. The information prompting method according to claim 1, characterized in that, The controller acquires information about the road surface around the vehicle as information about the surrounding environment, estimates the state of the road surface based on the acquired information about the road surface around the vehicle, generates a vibration signal corresponding to the estimated state of the road surface, and controls the output device based on the generated vibration signal.

3. The information prompting method according to claim 1 or 2, characterized in that, The controller obtains information about the road surface around the vehicle from sensors, including a camera mounted on the vehicle, as information about the surrounding environment.

4. The information prompting method according to any one of claims 1 to 3, characterized in that, The controller infers the state of the road surface based on information about the road surface around the vehicle obtained from sensors, including a camera mounted on the vehicle, and / or information about the road surface resistance calculated based on the slip ratio of the vehicle's wheels, and uses this information as the surrounding environment. The slip ratio is calculated based on the rotational speed of the wheel.

5. The information prompting method according to any one of claims 1 to 4, characterized in that, The controller generates at least one of the vibration signal, the sound signal, and the image signal as an output signal representing the presumed condition of the surrounding environment.

6. The information prompting method according to any one of claims 1 to 5, characterized in that, Based on the generated output signal, the controller controls the output of at least one of the following devices: a sound output device, a vibration output device, and an image output device. The sound output device and the image output device are located inside the vehicle. The vibration output device is located inside the vehicle in a place where the user comes into contact with it.

7. The information prompting method according to any one of claims 1 to 6, characterized in that, The controller pre-establishes and records correspondences between vibrations detected by vibration sensors mounted on the vehicle, sounds detected by microphones mounted on the vehicle, and images of the road surface around the vehicle, according to each specified surrounding environment. If the presumed surrounding environment is the specified surrounding environment, the output signal is generated based on at least one of the vibration, the sound, and the image that corresponds to and is recorded in relation to the specified surrounding environment.

8. The information prompting method according to any one of claims 1 to 7, characterized in that, The controller obtains information about the surrounding environment in the area in front of the vehicle by communicating with other vehicles around the vehicle and with a camera mounted on the vehicle. Based on the obtained information about the surrounding environment in the area in front, the surrounding environment in the area in front is inferred. The output signal is generated based on the estimated surrounding environment in the foreground region. Before the vehicle reaches the area ahead, the output device is controlled based on the generated output signal.

9. The information prompting method according to any one of claims 1 to 8, characterized in that, The controller obtains information about the position of the surrounding objects relative to the vehicle as information about the surrounding environment. Based on the position of the surrounding objects, it determines the output device that will be the object of output control from the output devices located in multiple parts of the vehicle, and controls the output of the determined output device.

10. An information prompting method, executed by a controller, characterized in that, The controller acquires information about the surrounding environment, including objects that the vehicle is approaching, and based on the acquired information about the surrounding environment, infers the category of the surrounding objects, generates at least one output signal, either a vibration signal or a sound signal, corresponding to the inferred category of the surrounding objects, and controls the output device based on the generated output signal.

11. An information prompting device, comprising a controller, characterized in that, The controller acquires information about the surrounding environment, including objects in contact with the vehicle, and estimates the state of the surrounding objects based on the acquired information. Based on the estimated state of the surrounding objects, it generates at least one output signal among vibration signal, sound signal, and image signal, and controls the output device based on the generated output signal.

12. A recording medium recording an information prompting program executed by a controller, characterized in that, The information prompting program causes the controller to perform a process including the following steps: Obtain information about the surrounding environment, including objects in contact with the vehicle. Based on the information obtained about the surrounding environment, the state of the surrounding objects is inferred; Based on the presumed state of the surrounding objects, at least one of the following output signals is generated: a vibration signal, a sound signal, and an image signal. The output device is controlled based on the generated output signal.

Images