Information processing equipment, information processing methods, computer program products, mobile devices and information processing systems
By installing multiple sensors on the vehicle and combining them with warning sound control technology, the accuracy of recognizing the direction of objects around the vehicle is improved, enhancing the driver's object perception ability.
Patent Information
- Application Number
- CN202180087735.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-01-04
- Filing Date
- 2021-12-21
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-12-21
AI Technical Summary
In existing technologies, the accuracy of orientation recognition of objects around a vehicle is insufficient, making it difficult to effectively improve.
By installing sensors such as cameras, radar, LiDAR, and ultrasonic sensors on the vehicle, combined with identification components and warning sound control components, the characteristic quantity and sound image position of the warning sound are controlled according to the object's location to indicate the object's presence.
It improves the accuracy of recognizing the orientation of objects around the vehicle, enhancing the driver's ability to perceive the position of objects.
Smart Images

Figure CN116670006B_ABST
Abstract
Description
Technical Field
[0001] This technology relates to information processing devices, information processing methods, programs, mobile devices, and information processing systems, and particularly to information processing devices, information processing methods, programs, mobile devices, and information processing systems suitable for use in situations where warning sounds are output to objects around a mobile device. Background Technology
[0002] Previously, techniques have been proposed that use acoustic localization control to localize the acoustic image of a warning sound in the direction of an object present around the vehicle, thereby enabling the driver to identify the direction of the object's presence (e.g., see PTL 1).
[0003] [List of Citations]
[0004] [Patent Literature]
[0005] [PTL 1]
[0006] Japanese Patent Publication No. 2007-253721 Summary of the Invention
[0007] [Technical Issues]
[0008] In techniques such as PTL 1 that use sound localization control to position the acoustic image of a warning sound in the direction of an object present around the vehicle, there is a need to improve the accuracy of the object's orientation.
[0009] This technology was developed in view of the situation and aims to improve the accuracy of recognizing the orientation of objects around mobile devices such as vehicles.
[0010] [Solution to the problem]
[0011] An information processing apparatus according to a first aspect of the present technology includes: an identification component configured to perform detection processing on an object around a mobile device; and a warning sound control component configured to control a feature quantity of a first warning sound and the position of the acoustic image of the first warning sound based on the position of the detected object, the first warning sound indicating the presence of the object.
[0012] The information processing method according to a first aspect of the present technology includes the following steps: performing detection processing on objects around a mobile device; and controlling the feature quantity of a warning sound and the position of the acoustic image of the warning sound based on the position of the detected object, the warning sound indicating the presence of the object.
[0013] The procedure according to a first aspect of the present technology causes a computer to perform processing, the processing including performing detection processing on objects around a mobile device; and controlling the characteristic quantity of a warning sound and the position of the acoustic image of the warning sound based on the position of the detected object, the warning sound indicating the presence of the object.
[0014] In a first aspect of this technology, detection processing of objects around the mobile device is performed, and the feature quantity of the warning sound, the warning sound indicating the presence of the object, and the position of the sound image of the warning sound are controlled according to the position of the detected object.
[0015] A mobile device according to a second aspect of the present technology includes: an identification component configured to perform detection processing on objects around the mobile device; and a warning sound control component configured to control the feature quantity of a warning sound and the position of the acoustic image of the warning sound based on the position of the detected object, the warning sound indicating the presence of the object.
[0016] In a second aspect of this technology, detection processing of surrounding objects is performed, and the characteristic quantity of the warning sound, the warning sound indicating the presence of the object, and the position of the sound image of the warning sound are controlled according to the position of the detected object.
[0017] An information processing system according to a third aspect of the present technology includes: a sensor unit configured to sense the vicinity of a mobile device; an identification component configured to perform detection processing on an object in the vicinity of the mobile device with reference to the sensing result of the sensor unit; a warning sound control component configured to control a feature quantity of a warning sound and the position of the sound image of the warning sound according to the position of the detected object, the warning sound indicating the presence of the object; and a speaker configured to output a warning sound under the control of the warning sound control component.
[0018] In a third aspect of this technology, sensing of the area around the mobile device is performed, object detection processing of the area around the mobile device is performed with reference to the sensing results, and the position of the warning sound's characteristic quantity, the warning sound indicating the presence of the object, and the sound image of the warning sound are controlled based on the position of the detected object, and the warning sound is output. Attached Figure Description
[0019] Figure 1 This is a block diagram illustrating an example configuration of a vehicle control system.
[0020] Figure 2 This is a diagram showing an example of the mounting locations of a camera, LiDAR, radar, and ultrasonic sensors.
[0021] Figure 3 This is a schematic diagram of the vehicle's interior viewed from right to left and looking down.
[0022] Figure 4 This is a schematic diagram of the vehicle's interior as viewed from the front.
[0023] Figure 5 This is a block diagram showing the configuration associated with the warning sound control component.
[0024] Figure 6 This is a flowchart to illustrate the warning sound control process.
[0025] Figure 7 This is a diagram showing an example of a region in a planar direction.
[0026] Figure 8 This is a diagram showing the direction in which warning sounds are heard for objects in each area.
[0027] Figure 9 This is a diagram illustrating the warning sounds for vehicles approaching from the left rear.
[0028] Figure 10 This is a diagram illustrating the warning sounds for vehicles approaching from the left rear.
[0029] Figure 11 This is a diagram illustrating the warning sounds for vehicles approaching from the left rear.
[0030] Figure 12 This is a diagram showing an example of a region setting in the height direction.
[0031] Figure 13 This is a diagram showing an example of a region in a planar direction.
[0032] Figure 14 This is a diagram showing an example of a region in a planar direction.
[0033] Figure 15 This is a diagram showing an example of a region in a planar direction.
[0034] Figure 16 This is a diagram illustrating an example of how the characteristic quantity of a warning sound is controlled based on the area where an object is present.
[0035] Figure 17 This is a diagram illustrating an example of how the characteristic quantity of a warning sound is controlled based on the area where an object is present.
[0036] Figure 18 This is a diagram showing an example of a computer configuration. Detailed Implementation
[0037] The following describes the patterns used to implement this technology. The descriptions will be presented in the following order.
[0038] 1. Configuration example of vehicle control system
[0039] 2. Example
[0040] 3. Modification
[0041] 4. Other
[0042] <<1. Configuration Example of Vehicle Control System>>
[0043] Figure 1 This is a block diagram illustrating an example configuration of a vehicle control system 11, which is an example of a mobile device control system to which this technology is applied.
[0044] The vehicle control system 11 is installed in the vehicle 1 and performs processing related to driving support and autonomous driving of the vehicle 1.
[0045] The vehicle control system 11 includes a vehicle control ECU (electronic control unit) 21, a communication unit 22, a map information accumulation unit 23, a location information acquisition unit 24, an external identification sensor 25, an in-vehicle sensor 26, a vehicle sensor 27, a storage unit 28, a driving support and automatic driving control unit 29, a DMS (driver monitoring system) 30, an HMI (human-machine interface) 31, and a vehicle control unit 32.
[0046] The vehicle control ECU 21, communication unit 22, map information accumulation unit 23, location information acquisition unit 24, external identification sensor 25, in-vehicle sensor 26, vehicle sensor 27, storage unit 28, driving support and automatic driving control unit 29, driver monitoring system (DMS) 30, human-machine interface (HMI) 31, and vehicle control unit 32 are communicatively connected to each other via communication network 41. Communication network 41 includes, for example, in-vehicle communication networks conforming to digital two-way communication standards, buses, etc., such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), FlexRay (registered trademark), or Ethernet (registered trademark). Different communication networks 41 can be used depending on the type of data transmitted. For example, CAN can be used for data related to vehicle control, while Ethernet can be used for high-volume data. Incidentally, components of the vehicle control system 11 can be directly connected to each other wirelessly, for example, assuming relatively short-range communication, such as short-range wireless communication (NFC (Near Field Communication)) or Bluetooth (registered trademark), without the intervention of communication network 41.
[0047] Incidentally, in the following text, when the components of the vehicle control system 11 communicate with each other via the communication network 41, the description of the communication network 41 will be omitted. For example, when the vehicle control ECU 21 and the communication unit 22 communicate with each other via the communication network 41, a simple description will be given of the vehicle control ECU 21 and the communication unit 22 communicating with each other.
[0048] The vehicle control ECU 21 includes, for example, various processors, such as CPU (central processing unit) and MPU (microprocessor unit). The vehicle control ECU 21 controls all or part of the functions of the vehicle control system 11.
[0049] The communication unit 22 communicates with various devices inside and outside the vehicle, other vehicles, servers, base stations, etc., and sends and receives various data. At this time, the communication unit 22 can use multiple communication systems to perform communication.
[0050] The communication with the outside of the vehicle that can be performed by the communication unit 22 will be briefly described. The communication unit 22 communicates, for example, with a server (hereinafter referred to as an external server) existing on an external network via a wireless communication system such as 5G (Fifth Generation Mobile Communication System), LTE (Long Term Evolution), or DSRC (Dedicated Short Range Communication) through a base station or access point. The external network on which the communication unit 22 performs communication is, for example, the Internet, a cloud network, or a network specific to an enterprise entity. The communication system used by the communication unit 22 to perform communication on the external network is not particularly limited to any type, as long as the communication system is a wireless communication system that allows for two-way digital communication at a predetermined communication speed or higher and over a predetermined distance or greater.
[0051] Additionally, the communication unit 22 can, for example, use P2P (peer-to-peer) technology to communicate with terminals located near the vehicle. Terminals located near the vehicle could be, for example, terminals worn by mobile entities moving at relatively low speeds (such as pedestrians, bicycles, etc.), terminals fixed and installed in locations such as shops, or MTC (machine-to-computer communication) terminals. Furthermore, the communication unit 22 can also perform V2X communication. V2X communication refers to communication between the vehicle and other things, such as vehicle-to-vehicle communication with other vehicles, road-to-vehicle communication with roadside units, vehicle-to-infrastructure communication, communication with houses, or pedestrian-to-vehicle communication with terminals carried by pedestrians.
[0052] The communication unit 22 can, for example, receive programs (via air) from an external source for updating the software controlling the operation of the vehicle control system 11. The communication unit 22 can also receive map information, traffic information, and information about the surroundings of the vehicle 1 from an external source. Furthermore, the communication unit 22 can, for example, send information about the vehicle 1 and information about the surroundings of the vehicle 1 to an external source. The information about the vehicle 1 sent to the external source via the communication unit 22 includes, for example, data indicating the status of the vehicle 1 and the identification results from the identification unit 73. In addition, the communication unit 22 can, for example, perform communications supporting vehicle emergency notification systems such as eCall.
[0053] Communication unit 22 may receive, for example, electromagnetic waves transmitted by a road traffic information communication system (VICS (Vehicle Information and Communication System) (registered trademark)) such as a radio beacon, optical beacon, or FM multiplex broadcast.
[0054] The communication with the vehicle interior that can be performed by the communication unit 22 will be briefly described. The communication unit 22 can communicate with various devices within the vehicle, for example, using wireless communication. The communication unit 22 can perform wireless communication with devices within the vehicle via a communication system such as wireless LAN, Bluetooth, NFC, or WUSB (Wireless USB), enabling bidirectional digital communication at a predetermined communication speed or higher. Not limited to this, the communication unit 22 can also communicate with various devices within the vehicle using wired communication. For example, the communication unit 22 can communicate with various devices within the vehicle via a cable connected to a connection terminal not shown. The communication unit 22 can communicate with various devices within the vehicle via wired communication such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (registered trademark), or MHL (Mobile High-Definition Link), enabling bidirectional digital communication at a predetermined communication speed or higher.
[0055] Here, "devices within the vehicle" refers to, for example, devices not connected to the vehicle's communication network 41. Examples of devices within the vehicle include, for instance, mobile devices or wearable devices carried by occupants such as the driver, information devices carried into the vehicle and temporarily installed therein.
[0056] Map information accumulation unit 23 accumulates one or both of externally obtained maps and maps generated by vehicle 1. For example, map information accumulation unit 23 accumulates a three-dimensional high-precision map, a global map with lower precision than the high-precision map but covering a wide area, etc.
[0057] High-precision maps include, for example, dynamic maps, point cloud maps, and vector maps. Dynamic maps are maps that include four layers: dynamic information, quasi-dynamic information, quasi-static information, and static information. Dynamic maps are provided to vehicles from external servers, etc. Point cloud maps are maps that include point clouds (point cluster data). Vector maps are maps that, for example, associate traffic information such as the positions of lanes and traffic lights with point cloud maps and adapt them for ADAS (Advanced Driver Assistance Systems) or AD (Autonomous Driving) systems.
[0058] Point cloud maps and vector maps can be provided from external servers, or generated in vehicle 1 by referencing results sensed by cameras 51, radar 52, LiDAR 53, etc., and are used as maps for matching with the local map described later, and are accumulated in map information accumulation unit 23. Alternatively, when a high-precision map is provided from an external server, map data of a few hundred meters square related to the planned route that vehicle 1 will subsequently travel is obtained from the external server to reduce communication overhead.
[0059] The location information acquisition unit 24 receives GNSS signals from GNSS (Global Navigation Satellite System) satellites to obtain location information about the vehicle 1. The acquired location information is provided to the driving support and automatic driving control unit 29. Incidentally, the location information acquisition unit 24 is not limited to systems using GNSS signals, and location information can be obtained, for example, by using beacons.
[0060] The external identification sensor 25 includes various sensors for identifying the external conditions of the vehicle 1. The external identification sensor 25 provides sensor data from each sensor to various parts of the vehicle control system 11. The type and number of sensors included in the external identification sensor 25 are optional.
[0061] External identification sensor 25 includes, for example, a camera 51, radar 52, LiDAR (light detection and ranging, laser imaging detection and ranging) 53, and an ultrasonic sensor 54. It is not limited thereto, and external identification sensor 25 may be configured to include one or more of the following sensors: camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54. The number of camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54 is not particularly limited to any specific number, as long as it is a number that can actually be installed in vehicle 1. Furthermore, the types of sensors included in external identification sensor 25 are not limited to this example, and external identification sensor 25 may include other types of sensors. Examples of the sensing areas of the various sensors included in external identification sensor 25 will be described later.
[0062] Note that the imaging system of camera 51 is not particularly limited to any kind. For example, cameras of various imaging systems capable of performing distance measurement, such as ToF (Time-of-Flight) cameras, stereo cameras, monocular cameras, and infrared cameras, can be used as camera 51 as needed. However, it is not limited to this; camera 51 may not involve distance measurement and may simply be a camera for acquiring images.
[0063] Additionally, for example, the external identification sensor 25 may include an environmental sensor for detecting the environment of the vehicle 1. This environmental sensor is used to detect environmental factors including atmospheric conditions, weather, brightness, etc. The environmental sensor may include, for example, various sensors such as rain sensors, fog sensors, sunlight sensors, snow sensors, and illuminance sensors.
[0064] In addition, the external identification sensor 25 includes, for example, a microphone for detecting sounds around the vehicle 1 and the location of sound sources.
[0065] The in-vehicle sensors 26 include various sensors for detecting information about the interior of the vehicle. The in-vehicle sensors 26 provide sensor data from each sensor to various parts of the vehicle control system 11. The types and number of sensors included in the in-vehicle sensors 26 are not particularly limited to any particular type or number, as long as they are those that can actually be installed in the vehicle 1.
[0066] For example, the in-vehicle sensor 26 may include one or more sensors selected from cameras, radar, seating sensors, steering wheel sensors, microphones, and biosensors. As a camera included in the in-vehicle sensor 26, it may be a camera from various photographic systems capable of ranging, such as ToF cameras, stereo cameras, SLR cameras, and infrared cameras. However, it is not limited to this; the camera included in the in-vehicle sensor 26 may not involve ranging but may simply be a camera for acquiring photographic images. For example, biosensors included in the in-vehicle sensor 26 may be placed in seats, steering wheels, etc., to detect various biometric information about occupants such as the driver.
[0067] Vehicle sensor 27 includes various sensors for detecting the state of vehicle 1. Vehicle sensor 27 provides sensor data from each sensor to various parts of vehicle control system 11. The types and number of various sensors included in vehicle sensor 27 are not particularly limited to any type or number, as long as the type and number are those that can actually be installed in vehicle 1.
[0068] For example, vehicle sensor 27 includes a speed sensor, an acceleration sensor, an angular velocity sensor (gyroscope sensor), and an inertial measurement unit (IMU) integrating these sensors. For example, vehicle sensor 27 includes a steering angle sensor for detecting the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor for detecting the amount of operation of the accelerator pedal, and a brake sensor for detecting the amount of operation of the brake pedal. For example, vehicle sensor 27 includes a rotation sensor for detecting the rotational speed of the engine or electric motor, a tire pressure sensor for detecting tire pressure, a slip rate sensor for detecting tire slip ratio, and a wheel speed sensor for detecting the rotational speed of the wheels. For example, vehicle sensor 27 includes a battery sensor for detecting the remaining battery capacity and temperature, and an impact sensor for detecting impacts from external sources.
[0069] Storage unit 28 includes at least one of non-volatile storage medium and volatile storage medium. Storage unit 28 stores data and programs. Storage unit 28 is used as, for example, EEPROM (Electrically Erasable Programmable Read-Only Memory) and RAM (Random Access Memory). Magnetic storage devices such as HDD (Hard Disk Drive), semiconductor storage devices, optical storage devices, and magneto-optical storage devices can be used as storage media. Storage unit 28 stores various programs and data used by various parts of the vehicle control system 11. For example, storage unit 28 includes EDR (Event Data Recorder) and DSSAD (Data Storage System for Automated Driving), and stores information about vehicle 1 and information obtained by in-vehicle sensors 26 before and after events such as accidents.
[0070] The driving support and automatic driving control unit 29 controls the driving support and automatic driving of the vehicle 1. The driving support and automatic driving control unit 29 includes, for example, an analysis unit 61, an action planning unit 62, and an operation control unit 63.
[0071] The analysis unit 61 performs analysis and processing of the vehicle 1 and its surrounding conditions. The analysis unit 61 includes a self-position estimation unit 71, a sensor fusion unit 72, and an identification unit 73.
[0072] The self-position estimation unit 71 estimates the vehicle 1's own position by referencing sensor data from the external identification sensor 25 and the high-precision map accumulated in the map information accumulation unit 23. For example, the self-position estimation unit 71 generates a local map by referring to sensor data from the external identification sensor 25 and estimates the vehicle 1's own position by matching the local map with the high-precision map. For example, the center of the rear wheel axle is set as the reference for the vehicle 1's position.
[0073] Local maps can be, for example, high-precision 3D maps or occupancy grid maps generated using technologies such as SLAM (Simultaneous Localization and Mapping). High-precision 3D maps include, for example, point cloud maps. Occupancy grid maps divide the 3D or 2D space surrounding vehicle 1 into grids of predetermined size and represent the occupancy status of objects in grid units. For example, the occupancy status of an object is indicated by its presence, absence, or probability. Local maps are also used, for example, in the detection and recognition processing of the external conditions of component 73 on vehicle 1.
[0074] Incidentally, the self-position estimation component 71 can estimate the self-position of the vehicle 1 by referring to the position information obtained by the position information acquisition unit 24 and the sensor data from the vehicle sensor 27.
[0075] The sensor fusion unit 72 performs sensor fusion processing, which obtains new information by combining data from multiple different types of sensors (e.g., image data from camera 51 and sensor data from radar 52). Methods for combining different types of sensor data include integration, fusion, correlation, etc.
[0076] The identification component 73 performs detection processing of the external condition of the vehicle 1 and identification processing of the external condition of the vehicle 1.
[0077] For example, the identification component 73 refers to information from the external identification sensor 25, information from its own position estimation component 71, and information from the sensor fusion component 72 to perform detection and identification processing of the external conditions of the vehicle 1.
[0078] Specifically, for example, the identification component 73 performs detection and identification processing of objects around the vehicle 1. Object detection processing includes, for example, detecting the presence or absence of an object, its size, shape, position, movement, etc. Object identification processing includes, for example, identifying attributes such as the type of object and identifying a specific object. However, detection and identification processing are not necessarily clearly separated from each other and may overlap.
[0079] For example, the identification component 73 detects objects around the vehicle 1 by performing clustering, which classifies point clouds based on sensor data such as radar 52, LiDAR 53, etc., into each cluster of point groups. Thus, the presence, size, shape, and location of objects around the vehicle 1 are detected.
[0080] For example, the recognition component 73 detects the movement of objects around vehicle 1 by performing tracking of the movement of clusters of points classified by clustering. Thus, the speed and direction of travel (movement vector) of objects around vehicle 1 are detected.
[0081] For example, the recognition component 73 can detect or recognize vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road markings, etc., by referring to image data provided by the camera 51. Additionally, the recognition component 73 can identify the types of objects around the vehicle 1 by performing recognition processing such as semantic segmentation.
[0082] For example, the recognition component 73 can refer to the map accumulated in the map information accumulation unit 23, the self-position estimation result of the self-position estimation component 71, and the recognition result of the recognition component 73 on objects around the vehicle 1 to perform traffic rule recognition processing around the vehicle 1. Through this processing, the recognition component 73 can recognize the position and status of traffic lights, the content of traffic signs and road markings, the content of traffic regulations, and the lanes that can be driven.
[0083] For example, the recognition component 73 can perform recognition processing on the surrounding environment of the vehicle 1. Assume that the surrounding environment to be recognized by the recognition component 73 is weather, temperature, humidity, brightness, road conditions, etc.
[0084] Action planning component 62 generates an action plan for vehicle 1. For example, action planning component 62 generates an action plan by performing route planning and route tracking processes.
[0085] Incidentally, path planning (global path planning) is the process of planning a general path from the start to the destination. This path planning also includes a process called trajectory planning, which takes into account the motion characteristics of vehicle 1 and performs trajectory generation (local path planning) to enable safe and smooth travel in the vicinity of vehicle 1 along the planned path.
[0086] Path following is a process for planning operations to ensure safe and accurate travel within a planned time along a path planned by the path planner. The action planning component 62, for example, can calculate the target speed and target angular velocity of vehicle 1 by referring to the results of the path following process.
[0087] Operation control unit 63 controls the operation of vehicle 1 in order to implement the action plan generated by action plan unit 62.
[0088] For example, the operation control unit 63 controls the steering control unit 81, braking control unit 82, and drive control unit 83 included in the vehicle control unit 32 (described later), thereby performing acceleration / deceleration control and directional control to ensure that the vehicle 1 travels on a trajectory calculated by the trajectory plan. For example, the operation control unit 63 performs coordinated control to achieve ADAS functions such as collision avoidance or impact mitigation, following, speed maintenance, collision warning, and lane departure warning. For example, the operation control unit 63 performs coordinated control for autonomous driving that does not rely on driver operation.
[0089] The DMS 30 references sensor data from the in-vehicle sensor 26 and input data to the HMI 31, which will be described later, to perform driver authentication processing, driver state recognition processing, etc. The driver's state to be recognized includes, for example, assumed physical condition, alertness, concentration, fatigue, gaze direction, intoxication level, driving actions, posture, etc.
[0090] Incidentally, DMS 30 can perform authentication processing for (one or more) occupants other than the driver and status recognition processing for (one or more) occupants. Additionally, for example, DMS 30 can refer to sensor data from in-vehicle sensor 26 to perform vehicle interior condition recognition processing. Suppose the vehicle interior conditions to be recognized are, for example, atmospheric temperature, humidity, brightness, odor, etc.
[0091] HMI 31 inputs various data and instructions, and presents these data to the driver.
[0092] The data input of HMI 31 will be briefly described. HMI 31 includes input devices for human data input. HMI 31 generates input signals based on data, instructions, etc., input through the input devices and provides these input signals to various parts of the vehicle control system 11. HMI 31 includes, for example, operating elements as input devices, such as touch panels, buttons, switches, and joysticks. Not limited to this, HMI 31 may also include input devices that allow information input via methods other than manual operation (i.e., via voice, gestures, etc.). Furthermore, HMI 31 may, for example, use remote control devices utilizing infrared or radio waves, or external connection devices such as mobile devices or wearable devices that support the operation of the vehicle control system 11, as input devices.
[0093] The presentation of data by HMI 31 will be briefly described. HMI 31 generates visual, auditory, and tactile information for occupants or the outside of the vehicle. Furthermore, HMI 31 performs output control, controlling the output, content, timing, and method of each generated information. HMI 31 generates and outputs visual information, such as information represented by images or light, including operation screens, displays of vehicle 1's status, warning displays, and monitoring images indicating the surrounding conditions of vehicle 1. Additionally, HMI 31 generates and outputs auditory information, such as information represented by sound, including voice guidance, warning sounds, and warning messages. Moreover, HMI 31 generates and outputs tactile information, provided to occupants through tactile sensations such as force, vibration, and movement.
[0094] The output device for HMI 31 that outputs visual information can be, for example, a display device that presents visual information by displaying images on its own, or a projector device that presents visual information by projecting images. Incidentally, in addition to display devices with ordinary displays, the display device can also be a device that displays visual information within the occupant's field of vision; such devices include, for example, head-up displays, transmissive displays, and wearable devices with AR (augmented reality) functionality. Furthermore, HMI 31 can also use display devices included in the navigation system, dashboard, CMS (camera monitoring system), electronic mirrors, lights, etc., installed in vehicle 1 as the output device to which visual information will be output.
[0095] The output device for HMI 31 to output auditory information can be, for example, an audio speaker, headphones, and earphones.
[0096] For example, a haptic element using haptic technology can be used as an output device for the HMI 31 to output haptic information. The haptic element is, for example, located in parts of the vehicle 1 that are in contact with passengers, such as the steering wheel or seat.
[0097] The vehicle control unit 32 controls various parts of the vehicle 1. The vehicle control unit 32 includes a steering control component 81, a braking control component 82, a drive control component 83, a body system control component 84, a lighting control component 85, and a horn control component 86.
[0098] The steering control unit 81 detects and controls the state of the steering system of the vehicle 1. The steering system includes, for example, a steering mechanism with a steering wheel, an electric steering device, etc. The steering control unit 81 includes, for example, a steering ECU that controls the steering system, and an actuator that drives the steering system.
[0099] The brake control unit 82 detects and controls the status of the braking system of the vehicle 1. This braking system includes, for example, a brake mechanism with a brake pedal, ABS (anti-lock braking system), regenerative braking mechanism, etc. The brake control unit 82 includes, for example, a brake ECU that controls the braking system, and actuators that drive the braking system.
[0100] The drive control unit 83 performs tasks such as detecting and controlling the state of the drive system of the vehicle 1. The drive system includes, for example, an accelerator pedal, a drive force generating device (e.g., an internal combustion engine or drive motor) for generating drive force, and a drive force transmission mechanism for transmitting drive force to the wheels. The drive control unit 83 includes, for example, a drive ECU for controlling the drive system and actuators for driving the drive system.
[0101] The body system control unit 84 detects and controls the status of the vehicle 1's body system. The body system includes, for example, a keyless entry system, a smart key system, power windows, power seats, air conditioning, airbags, seat belts, and a gearshift lever. The body system control unit 84 includes, for example, a body system ECU that controls the body system and actuators that drive the body system.
[0102] The lighting control unit 85 detects and controls the status of various lights in the vehicle 1. Assuming the lights to be controlled include, for example, headlights, reverse lights, fog lights, turn signals, brake lights, projectors, bumper indicators, etc. The lighting control unit 85 includes a lighting ECU for controlling the lights and actuators for driving the lights.
[0103] The horn control unit 86 detects and controls the status of the car horn of vehicle 1. The horn control unit 86 includes, for example, a horn ECU that controls the car horn and an actuator that drives the car horn.
[0104] << 2. Example>>
[0105] Next, we will refer to Figures 2 to 17 An embodiment of this technology is described.
[0106] <Example of the configuration of external identification sensor 25 and in-vehicle sensor 26>
[0107] Figure 2 An example of the mounting positions of the camera 51, radar 52, LiDAR 53, and ultrasonic sensor 54 included in the external identification sensor 25 of vehicle 1, and the camera included in the in-vehicle sensor 26 are shown. In this example, the external identification sensor 25 includes cameras 51FC1 to 51BC3, radar 52FC to 52BR, LiDAR 53F to 53B, and ultrasonic sensors 54FL1 to 54BR. The in-vehicle sensor 26 includes cameras 101L to 102.
[0108] Camera 51FC1 is positioned near the center of the front end of vehicle 1. Camera 51FC1 takes pictures along the front of vehicle 1. For example, a fisheye camera is used as camera 51FC1. Camera 51FC1 is used, for example, for ADAS and surround view. Surround view is a function that displays, for example, images of the surroundings of vehicle 1 and a bird's-eye view of the surroundings of vehicle 1 viewed from above.
[0109] Camera 51FC2 is positioned near the front center of the interior of vehicle 1. Camera 51FC2 captures images along the front of vehicle 1 through the windshield. Camera 51FC2 is used, for example, in ADAS (Advanced Driver Assistance Systems). Images captured by camera 51FC2 are recorded, for example, in a driver's recorder.
[0110] Cameras 51FL and 51FR are arranged laterally at a predetermined distance from each other inside the front of vehicle 1. Cameras 51FL and 51FR constitute a stereo camera. Cameras 51FL and 51FR capture images along the front of vehicle 1 through the windshield. Cameras 51FL and 51FR are used, for example, in ADAS (Advanced Driver Assistance Systems). For example, cameras with a higher resolution than camera 51FC2 may be used as cameras 51FL and 51FR.
[0111] Cameras 51SL1 to 51SL4 are mounted on the left side surface of vehicle 1, near the front end of the driver's seat door.
[0112] Camera 51SL1 takes pictures from the left-facing front direction of vehicle 1. Camera 51SL1 is used, for example, in ADAS.
[0113] Camera 51SL2 takes pictures from the left side of vehicle 1. For example, camera 51SL2 may be a fisheye camera with a wider angle than cameras 51SL1, 51SL3, and 51SL4. For example, camera 51SL2 may be used for ADAS and surround view.
[0114] Camera 51SL3 performs shooting in a left-rear direction from vehicle 1. For example, a camera with a higher resolution than camera 51SL1 is used as camera 51SL3. Camera 51SL3 is used, for example, in ADAS.
[0115] Camera 51SL4 takes pictures from the left-rear angle of vehicle 1. The optical axis of camera 51SL4 points further back towards the rear of vehicle 1 than the optical axis of camera 51SL3. In other words, the optical axis of camera 51SL3 points further to the left (lateral) of vehicle 1 than the optical axis of camera 51SL4. Camera 51SL4 is used, for example, in a CMS (Content Management System).
[0116] Cameras 51SR1 to 51SR4 are mounted on the right side surface of vehicle 1 and are located near the front end of the door of the front passenger seat.
[0117] Camera 51SR1 performs shooting in the direction of right-southeast of vehicle 1. Camera 51SR1 is used, for example, in ADAS.
[0118] Camera 51SR2 performs shooting in the right direction of vehicle 1. For example, camera 51SR2 may be a fisheye camera with a wider angle than cameras 51SR1, 51SR3, and 51SR4. For example, camera 51SR2 may be used for ADAS and surround view.
[0119] Camera 51SR3 performs shooting in a right-rear-angle direction from vehicle 1. For example, a camera with a higher resolution than camera 51SR1 is used as camera 51SR3. Camera 51SR3 is used, for example, in ADAS.
[0120] Camera 51SR4 takes pictures from the right-rear angle of vehicle 1. The optical axis of camera 51SR4 points closer to the rearward direction of vehicle 1 than the optical axis of camera 51SR3. In other words, the optical axis of camera 51SR3 points closer to the right (lateral) direction of vehicle 1 than the optical axis of camera 51SR4. Camera 51SR4 is used, for example, in CMS.
[0121] A camera 51BC1 is positioned near the center of the rear end of vehicle 1. Camera 51BC1 takes pictures along the rear of vehicle 1. For example, a fisheye camera is used as camera 51BC1. For example, camera 51BC1 is used for ADAS and surround view.
[0122] Cameras 51BC2 and 51BC3 are installed near the center rear of the interior of vehicle 1. Cameras 51BC2 and 51BC3 capture images along the rear of vehicle 1 through the rear window. For example, camera 51BC2 is used for CMS (Conditional Management System). Camera 51BC3 is used, for example, for ADAS (Advanced Driver Assistance Systems). For example, the images captured by camera 51BC3 are recorded in a driver's recorder.
[0123] The radar 52FC is positioned near the center of the front of vehicle 1. The radar 52FC performs sensing at the front of vehicle 1.
[0124] Radar 52FL is located near the left end of the front of vehicle 1. Radar 52FL performs sensing in the left-southeast direction of vehicle 1.
[0125] Radar 52FR is located near the right end of the front of vehicle 1. Radar 52FR performs sensing in the right-front diagonal direction of vehicle 1.
[0126] Radar 52BC is positioned near the center of the rear end of vehicle 1. Radar 52BC senses along the rear of vehicle 1.
[0127] Radar 52BL is located near the left end of the rear end of vehicle 1. Radar 52BL performs sensing along the left rearward direction of vehicle 1.
[0128] The radar 52BR is located near the right end of the rear end of vehicle 1. The radar 52BR senses along the right rearward direction of vehicle 1.
[0129] A LiDAR 53F is positioned near the center of the front of vehicle 1. The LiDAR 53F performs sensing along the front of vehicle 1.
[0130] A LiDAR 53L is positioned in front of the left side surface of vehicle 1. The LiDAR 53L performs sensing along the left side of vehicle 1.
[0131] A LiDAR 53R is positioned in front of the right side surface of vehicle 1. The LiDAR 53R performs sensing along the rightward direction of vehicle 1.
[0132] A LiDAR 53B is positioned near the center of the rear end of vehicle 1. The LiDAR 53B performs sensing along the rearward direction of vehicle 1.
[0133] The ultrasonic sensor 54FL1 is positioned slightly to the left of the center of the front end of vehicle 1. The ultrasonic sensor 54FL1 performs sensing in the direction slightly to the left of the front end of vehicle 1.
[0134] The ultrasonic sensor 54FL2 is located near the left end of the front of the vehicle 1. The ultrasonic sensor 54FL1 performs sensing in the left-southeast direction of the vehicle 1.
[0135] The ultrasonic sensor 54FR1 is positioned slightly to the right of the center of the front of vehicle 1. The ultrasonic sensor 54FR1 performs sensing in the direction slightly to the right of the front of vehicle 1.
[0136] The ultrasonic sensor 54FR2 is located near the right end of the front of vehicle 1. The ultrasonic sensor 54FR1 performs sensing in the right-front diagonal direction of vehicle 1.
[0137] The ultrasonic sensor 54SL1 is positioned at the front of the left side surface of the vehicle body 1. The ultrasonic sensor 54SL1 performs sensing in the leftward direction at the front of the vehicle 1.
[0138] The ultrasonic sensor 54SL2 is located at the rear of the left side surface of the vehicle body 1. The ultrasonic sensor 54SL2 performs sensing in a leftward direction at the rear of the vehicle 1.
[0139] The ultrasonic sensor 54SR1 is positioned at the front of the right side surface of the vehicle body 1. The ultrasonic sensor 54SR1 performs sensing in the rightward direction at the front of the vehicle 1.
[0140] The ultrasonic sensor 54SR2 is located at the rear of the right side surface of the vehicle body 1. The ultrasonic sensor 54SR2 performs sensing in a rightward direction at the rear of the vehicle 1.
[0141] The ultrasonic sensor 54BL is located near the left end of the rear end of vehicle 1. The ultrasonic sensor 54BL performs sensing along the left-rear direction of vehicle 1.
[0142] The ultrasonic sensor 54BR is located near the right end of the rear end of vehicle 1. The ultrasonic sensor 54BR senses in the right-rear direction of vehicle 1.
[0143] Camera 101L is positioned diagonally above and to the right of the driver's seat inside the vehicle. Camera 101L captures images of the driver's seat and its vicinity. For example, a ToF camera can be used as camera 101L.
[0144] Camera 101R is positioned diagonally above and to the left of the front passenger seat inside the vehicle. Camera 101R captures images of the front passenger seat and its vicinity. For example, a ToF camera may be used as camera 101R.
[0145] Camera 102 is positioned slightly to the left of the center on the dashboard inside the vehicle. Camera 102 captures images of the driver's seat and its vicinity.
[0146] <Speaker installation location>
[0147] The following will describe an example of the speaker installation location within vehicle 1.
[0148] like Figure 3 As shown, in the door 202FL on the driver's seat 201FL side, a speaker 203FL is embedded slightly in front of the backrest of the driver's seat 201FL. Although not shown in the figure, in the door 202FR on the front passenger seat 201FR side, a speaker 203FR is embedded slightly in front of the backrest of the front passenger seat 201FR. In the door 202BL on the left rear seat 201BL side, a speaker 203BL is embedded slightly in front of the backrest of the rear seat 201BL. Although not shown in the figure, in the door 202BR on the right rear seat 201BR side, a speaker 203BR is embedded slightly in front of the backrest of the rear seat 201BR.
[0149] like Figure 4 As shown, seat speaker 204FL is embedded below the headrest of the driver's seat 201FL. Seat speaker 204FR is embedded below the headrest of the front passenger seat 201FR. Seat speaker 204BL is embedded in the left rear seat 201BL. Seat speaker 204BR is embedded in the right rear seat 201BR.
[0150] Incidentally, in the following text, unless it is necessary to separately distinguish between the driver's seat 201FL to the rear seat 201BR, the driver's seat 201FL to the rear seat 201BR will be referred to as seat 201. In the following text, unless it is necessary to separately distinguish between the doors 202FL to 202BR, the doors 202FL to 202BR will be referred to as door 202. Below, unless it is necessary to separately distinguish between the speakers 203FL to 203BR, the speakers 203FL to 203BR will be referred to as speaker 203. Below, unless it is necessary to separately distinguish between the seat speakers 204FL to 204BR, the seat speakers 204FL to 204BR will be referred to as seat speaker 204.
[0151] Speakers 203 embedded in each door 202 are used, for example, to output sound to the entire interior of the vehicle (all occupants inside the vehicle). In addition, 360 Real Audio (registered trademark) is implemented through the speakers 203 (hereinafter, 360 Real Audio will be referred to as 360RA).
[0152] Here, 360RA is a technology that arranges sound images (virtual sound sources) of various sounds at desired locations within a spherical space and controls the sound output so that the listener can hear each sound, with each sound emanating from the direction of its respective sound image. By using 360RA, occupants can, for example, enjoy motion graphics and music with realistic sound inside a vehicle. Furthermore, occupants can identify the positions of objects and other objects present around the vehicle.
[0153] On the other hand, the seat speakers 204 of each seat 201 are primarily used, for example, to output private sounds targeted at each occupant sitting in each seat 201. That is, the sound output from each seat speaker 204 is controlled individually.
[0154] Incidentally, the shape of each seat 201 and the position of each seat speaker 204 are adjusted so that people of various heights (seating heights) can clearly hear the sound of the seat speaker 204 of each seat 201.
[0155] Furthermore, this speaker arrangement is an example and can be changed. For instance, the speakers could be mounted on the dashboard at the front of vehicle 1.
[0156] <Warning Sound Control Component>
[0157] Figure 5 An example of the configuration associated with the warning sound control unit 251 as part of the functionality of HMI 31 is shown.
[0158] The warning sound control unit 251 controls the output of the warning sound by referring to the recognition result of the recognition unit 73, the action plan generated by the action plan unit 62, and the control status of the vehicle control unit 32 on each part of the vehicle 1, and controls the speaker 203 and the seat speaker 204.
[0159] Here, warning sounds include, for example, warning sounds used to notify the occupants of vehicle 1 of a dangerous situation and warning sounds used to notify objects around vehicle 1 (e.g., bicycles, pedestrians, etc.) of a dangerous situation. The former warning sound is, for example, a warning sound used to indicate the presence of an object or dangerous location around vehicle 1 and to warn the occupants inside the vehicle. The latter warning sound is, for example, a warning sound used to indicate the presence of vehicle 1 to objects around vehicle 1 and to warn those objects.
[0160] Additionally, as will be described later, the warning sound control unit 251 controls the position of the sound image (virtual sound source) of the warning sound and the characteristic quantities of the warning sound. Here, the characteristic quantities of the warning sound include, for example, volume (amplitude), pitch (frequency), tone (waveform and frequency components), rhythm (rhythm), melody (melody), harmony (harmony), etc.
[0161] <Warning Sound Control Processing>
[0162] Next, refer to Figure 6 The flowchart illustrates the warning sound control processing performed on vehicle 1.
[0163] This process begins, for example, when starting and driving the vehicle 1 is performed (e.g., when the ignition switch, power switch, starter switch, etc. of the vehicle 1 are turned on). Conversely, this process ends, for example, when an operation to end driving the vehicle 1 is performed (e.g., when the ignition switch, power switch, starter switch, etc. of the vehicle 1 are turned off).
[0164] In step S1, the recognition unit 73 performs object detection processing on objects around the vehicle 1. For example, as described above, the recognition unit 73 performs object detection processing on objects around the vehicle 1 by referring to information from the external recognition sensor 25, information from its own position estimation unit 71, information from the sensor fusion unit 72, etc. Thus, the recognition unit 73 detects the positions of objects around the vehicle 1. Incidentally, this object detection processing may include the object recognition processing described above.
[0165] Here, the recognition component 73 can use the curvature and slope information of the road on which the vehicle 1 travels to correct the detection results of the object position, thereby improving the detection accuracy.
[0166] For example, when vehicle 1 is traveling on a curved road, the recognition unit 73 estimates the curvature of the road by referring to the input image from camera 51. Alternatively, for example, when vehicle 1 is traveling on a curved road, the recognition unit 73 detects the actual curvature of the road by referring to a high-precision map such as a dynamic map. Then, the recognition unit 73 corrects the position detection results of other vehicles traveling on the same curved road based on the road curvature.
[0167] For example, when vehicle 1 is traveling on a slope, the identification unit 73 estimates the road gradient by referring to sensor data from the accelerometer included in vehicle sensor 27. Alternatively, for example, when vehicle 1 is traveling on a slope, the identification unit 73 detects the actual road gradient by referring to a high-precision map such as a dynamic map. Then, the identification unit 73 corrects the detected position in the pitch direction of other vehicles traveling on the same slope based on the road gradient.
[0168] Additionally, as needed, the identification component 73 detects the movement and speed of objects (e.g., speed relative to vehicle 1). Furthermore, as needed, the identification component 73 identifies attributes such as the type, size, and shape of objects surrounding vehicle 1.
[0169] In step S2, the identification component 73 determines whether an object has been detected by referring to the processing result of step S1. If it is determined that no object has been detected, the process returns to step S1.
[0170] After that, the processes of steps S1 and S2 are repeated until an object is detected in step S2.
[0171] On the other hand, if it is determined in step S2 that an object has been detected, the process proceeds to step S3.
[0172] In step S3, the identification component 73 determines whether an object exists in the area surrounding the vehicle 1.
[0173] Specifically, the identification component 73 defines an area around the vehicle 1 to define the range of the object from which the warning sound is output and the direction of the output warning sound.
[0174] Figure 7 An example of a zone setting method is shown. In this example, vehicle 1 is traveling on a road with at least three or more lanes on one side, and zones ZL1 to ZR4 are set around vehicle 1.
[0175] Incidentally, in the following text, it is assumed that the axis pointing in the direction of travel of vehicle 1 is the X-axis, and the axis perpendicular to the X-axis and pointing to the left of vehicle 1 is the Y-axis.
[0176] Zones ZL1 to ZR4 are rectangular areas of approximately the same size. Zones ZC1, ZC2, and ZC4 are located in the driving lane of vehicle 1. Zone ZC2 is located immediately behind vehicle 1. Zone ZC1 is located behind zone ZC2. Zone ZC4 is located immediately in front of vehicle 1.
[0177] Zones ZL1 to ZL4 are located in the lane to the left of the driving lane of vehicle 1. Zones ZR1 to ZR4 are located in the lane to the right of the driving lane of vehicle 1. Zones ZL1 and ZR1 are located at the same position as Zone ZC1 in the X-axis direction. Zones ZL2 and ZR2 are located at the same position as Zone ZC2 in the X-axis direction. Zones ZL3 and ZR3 are located at the same position as vehicle 1 in the X-axis direction. Zones ZL4 and ZR4 are located at the same position as Zone ZC4 in the X-axis direction.
[0178] For example, if the detected object is located outside the range of regions ZL1 to ZR4, the identification unit 73 determines that the object does not exist in each region, and the process returns to step S1.
[0179] Subsequently, the process of steps S1 to S3 is repeated until it is determined in step S3 that an object exists in the area surrounding vehicle 1.
[0180] On the other hand, in step S3, when the detected object is located within one of the regions ZL1 to ZR4, the identification unit 73 determines that the object exists in the area around the vehicle 1, and the process proceeds to step S4.
[0181] In step S4, vehicle 1 outputs a warning sound from the direction of the area where the object is located. Specifically, the identification unit 73 provides the warning sound control unit 251 with information indicating the area where the object is located, as well as the type and size of the object.
[0182] The warning sound control unit 251 generates warning sound data for outputting a warning sound for the detected object. The warning sound control unit 251 generates a warning sound output by providing the generated warning sound data to the speakers 203FL to 203BR.
[0183] At this time, for example, such as Figure 8 As indicated by the arrows in each area, the warning sound control unit 251 sets the position of the warning sound image (virtual sound source) so that the warning sound is heard from the direction of the area where the object is detected at a reference position (e.g., the driver's head) within the vehicle 1. This allows the driver to visually identify the direction of the detected object.
[0184] Figures 9 to 11An example of a method for controlling a warning sound for a vehicle 301 approaching from the left rear direction is shown.
[0185] Figure 9 The sound field A1 of the warning sound is shown when vehicle 301 is present in region ZL1. Figure 10 The sound field A1 of the warning sound is shown when vehicle 301 is present in area ZL2. Figure 11 The sound field A1 of the warning sound is shown when vehicle 301 is present in area ZL3.
[0186] Incidentally, the vertex of the cone-shaped sound field A1 represents the position of the sound image (virtual sound source). The direction in which the cone extends from the vertex of sound field A1 represents the direction of sound field expansion.
[0187] As described above, the orientation of the sound field A1 is set in a direction substantially the same as the direction of the driver's head when viewing vehicle 1 from vehicle 301. Furthermore, the sound image of the warning sound is positioned at a predetermined distance from the driver's head along the direction of vehicle 301. Therefore, the sound field A1 is moved in such a way that it points in a direction substantially the same as the direction of the driver's head when viewing vehicle 1 from vehicle 301. Moreover, the sound field A1 is moved such that the distance between the sound image of the warning sound and the driver's head is substantially proportional to the distance between vehicle 301 and vehicle 1.
[0188] Therefore, the driver can recognize the approach of vehicle 301 through the warning sound. In addition, the driver can visually identify the speed, distance, and direction of vehicle 301 by observing the movement of the warning sound.
[0189] Incidentally, such as Figure 12 As shown, regions can be divided not only in the planar direction but also in the vertical direction. For example, Figure 12 An example is shown where region ZL1 is divided into regions ZL1L to ZL1H in the height direction. Furthermore, the warning sound control unit 251 sets the position of the acoustic image of the warning sound for the detected object in the height direction based on the size, which is one of the attributes of the detected object in region ZL1.
[0190] Here, the size of an object is represented by at least one of the object's width, depth, and height.
[0191] For example, when the object is small, such as a motorcycle, bicycle, or pedestrian, the sound image position of the warning sound is set so that the warning sound is heard from the direction of zone ZL1L at the reference position. For example, when the object is medium-sized, such as a passenger vehicle, the sound image position of the warning sound is set so that the warning sound is heard from the direction of zone ZL1M at the reference position. For example, when the object is large, such as a large vehicle like a truck or bus, the sound image position of the warning sound is set so that the warning sound is heard from the direction of zone ZL1H at the reference position.
[0192] Therefore, even when an object is detected in the same area in the planar direction, the position of the warning sound image in the height direction changes depending on the size of the object. Thus, the driver can not only visually identify the location of the detected object, but also its size.
[0193] Furthermore, drivers can predict the degree of danger of an object by recognizing its size. Here, the degree of danger refers to the risk that vehicle 1 might be harmed by the object or the risk that vehicle 1 might cause harm to the object. For example, a driver can predict that the higher the position of the warning sound's sound image, the higher the risk that vehicle 1 might be harmed by the object due to a collision, and the lower the risk that vehicle 1 might cause harm to the object. Conversely, a driver can predict that the lower the position of the warning sound's sound image, the lower the risk that vehicle 1 might be harmed by the object due to a collision, and the higher the risk that vehicle 1 might cause harm to the object.
[0194] After that, the process returns to step S1 and the processing from step S1 onwards is executed.
[0195] As described above, the driver can hear the warning sound from the direction of the detected object, thereby improving the accuracy of the object's orientation. Furthermore, because the sound image of the warning sound moves not only in the planar direction but also in the vertical direction, the driver can identify the size of the object and thus determine its level of danger.
[0196] <3. Revision>
[0197] Modifications to the embodiments of the present technology described above will be described below.
[0198] <Region-related modifications>
[0199] The area around vehicle 1 that the driver should pay attention to changes depending on the situation of vehicle 1. Therefore, the recognition component 73 can change the setting of the area according to the situation of vehicle 1. In other words, the recognition component 73 can change the setting of the area according to the area that the driver should pay attention to.
[0200] Here, changing the settings of an area means changing its location, number, size, shape, etc. Additionally, the conditions of vehicle 1 include, for example, the location where vehicle 1 is traveling, the surrounding environment of vehicle 1, the driving operation of vehicle 1, and the driving status of vehicle 1.
[0201] For example, when vehicle 1 wants to change lanes, the driver's need to pay attention to vehicles in vehicle 1's lane is reduced. Therefore, when vehicle 1 wants to change lanes, the recognition component 73 may, for example, not be required. Figure 7 The driving lanes of vehicle 1 in the region are region ZC1, region ZC2 and region ZC4.
[0202] For example, when vehicle 1 is traveling on a road with two lanes on one side and there are lanes arranged therein... Figure 7 If the lanes in zones ZR1 to ZR4 are oncoming lanes, the recognition unit 73 may not need to set zones ZR1 to ZR4. Alternatively, for example, if the driver needs to pay more attention to vehicles in the oncoming lane due to narrow road width, poor visibility, etc., the recognition unit 73 may set zones ZR1 to ZR4.
[0203] For example, when vehicle 1 is driving in an urban area, the speed of vehicle 1 and surrounding vehicles is relatively low, and there are many pedestrians. Therefore, the driver needs to pay more attention to pedestrians crossing the road ahead rather than behind. Therefore, as... Figure 13 As shown, for example, the identification component 73 may be provided in areas ZL11 to ZR14 around the vehicle 1.
[0204] Regions ZL11 to ZR14 are rectangular areas of approximately the same size. Regions ZC11, ZC13, and ZC14 are located within the driving lane of vehicle 1. Region ZC11 is positioned immediately behind vehicle 1. Region ZC13 is positioned immediately in front of vehicle 1. Region ZC14 is positioned in front of region ZC13.
[0205] Zones ZL11 to ZL14 are located in the lane to the left of the driving lane of vehicle 1. Zones ZR11 to ZR14 are located in the lane to the right of the driving lane of vehicle 1. Zones ZL11 and ZR11 are located at the same position as Zone ZC11 in the X-axis direction. Zones ZL12 and ZR12 are located at the same position as vehicle 1 in the X-axis direction. Zones ZL13 and ZR13 are located at the same position as Zone ZC13 in the X-axis direction. Zones ZL14 and ZR14 are located at the same position as Zone ZC14 in the X-axis direction.
[0206] Therefore, in Figure 13 In the example, with Figure 7 Compared to the previous example, the area behind vehicle 1 is reduced, while the area in front of vehicle 1 is increased.
[0207] For example, when vehicle 1 is reversing, more attention needs to be paid to the area behind vehicle 1. Therefore, as Figure 14 As shown, for example, the identification component 73 may be provided in areas ZL21 to ZR25 around the vehicle 1.
[0208] Zones ZL21 to ZR25 are rectangular areas of approximately the same size. Zones ZC21 to ZC23 and Zone ZC25 are located in the driving lane of vehicle 1. Zone ZC23 is located immediately behind vehicle 1. Zone ZC22 is located behind zone ZC23. Zone ZC21 is located behind zone ZC22. Zone ZC25 is located immediately in front of vehicle 1.
[0209] Zones ZL21 to ZL25 are located in the lane to the left of the driving lane of vehicle 1. Zones ZR21 to ZR25 are located in the lane to the right of the driving lane of vehicle 1. Zones ZL21 and ZR21 are located in the same position as Zone ZC21 in the X-axis direction. Zones ZL22 and ZR22 are located in the same position as Zone ZC22 in the X-axis direction. Zones ZL23 and ZR23 are located in the same position as Zone ZC23 in the X-axis direction. Zones ZL24 and ZR24 are located in the same position as vehicle 1 in the X-axis direction. Zones ZL25 and ZR25 are located in the same position as Zone ZC25 in the X-axis direction.
[0210] Therefore, in Figure 14 In the example, with Figure 7 Compared to the example, the area behind vehicle 1 is increased.
[0211] For example, when vehicle 1 is parked in a parking lot, other vehicles are more likely to approach from the side than from in front of or behind vehicle 1, so more attention needs to be paid to the side of vehicle 1. Therefore, as Figure 15As shown, the identification component 73 can, for example, set an area ZL31A to ZR32C around the vehicle 1.
[0212] Regions ZL31A to ZR32C are rectangular regions of approximately the same size. Region ZL31A is located on the left side of vehicle 1. Region ZL31B is located to the left of region ZL31A. Region ZL31C is located to the left of region ZL31B. Region ZR31A is located on the right side of vehicle 1. Region ZR31B is located to the right of region ZR31A. Region ZR31C is located to the left of region ZR31B. Region ZC32 is located immediately in front of vehicle 1. Region ZL32A is located to the left of region ZC32. Region ZL32B is located to the left of region ZL32A. Region ZL32C is located to the left of region ZL32B. Region ZR32A is located to the right of region ZC32. Region ZR32B is located to the right of region ZR32A. Region ZR32C is located to the left of region ZR32B.
[0213] Therefore, in Figure 15 In the example, the area is more extensive on the side of vehicle 1 compared to other examples.
[0214] As described above, the zone setting can be changed according to the situation of vehicle 1. Therefore, it is possible to output warning sounds for objects around vehicle 1 more appropriately according to the situation.
[0215] Incidentally, although examples of making each region the same size and shape have been shown in the above description, it is possible to make each region different in size and shape.
[0216] Alternatively, for example, the warning sound control unit 251 can simply set the position of the sound image of the warning sound based on the position of the detected object, without setting each zone.
[0217] <Modifications related to warning sounds>
[0218] For example, the identification component 73 can estimate the degree of danger of the detected object, and the warning sound control component 251 can control the characteristic quantity of the warning sound according to the estimated degree of danger of the object.
[0219] For example, the warning sound control unit 251 changes the characteristic quantity of the warning sound so that the closer the area where the object is detected is, that is, the closer the detected object is to vehicle 1, the stronger the alarm is to the driver. Therefore, the higher the degree of danger of the detected object, the more confidently the driver can identify the presence and direction of the object and direct their attention to it.
[0220] For example, Figure 16 The area ZL41 located to the left rear of vehicle 1 and Figure 16The area ZR41 located to the right rear of vehicle 1 and Figure 17 The area ZL51 located to the left rear of vehicle 1 and Figure 17 A comparison of region ZR51, located to the right rear of vehicle 1, shows that regions ZL41 and ZR41 are closer to vehicle 1 than regions ZL51 and ZR51. Therefore, in the case where an object is present in region ZL41 or region ZR41, the warning sound control unit 251, for example, makes the warning sound higher (raised to a higher frequency) compared to the case where an object is present in region ZL51 or region ZR51.
[0221] For example, the higher the relative speed of an object with respect to vehicle 1 in the direction in which the object approaches vehicle 1, the higher the risk of vehicle 1 colliding with the object. Therefore, even when an object is detected in the same area, for example, the warning sound control unit 251 increases the volume of the warning sound as the relative speed of the detected object in the direction in which it approaches vehicle 1 increases.
[0222] Furthermore, for example, the warning sound control unit 251 can change the characteristic quantity of the warning sound between a situation where the vehicle 1 is more likely to be damaged by the detected object and a situation where the vehicle 1 is more likely to cause damage to the detected object. For example, if the detected object is a truck, bus, passenger car, etc., it is determined that the vehicle 1 is more likely to be damaged. For example, if the detected object is a motorcycle, bicycle, pedestrian, etc., it is determined that the vehicle 1 is more likely to cause damage.
[0223] Thus, the driver can intuitively identify whether their vehicle (Vehicle 1) is the victim of danger or the cause of danger. Then, the driver can take more appropriate action against the detected object based on whether their vehicle is the victim of danger or the cause of danger.
[0224] Furthermore, for example, if vehicle 1 has a speaker or similar device that outputs sound to the surrounding area, the warning sound control unit 251 can control the output of a warning sound indicating the presence of vehicle 1 to the detected object based on the degree of danger that vehicle 1 may cause to the detected object. This allows for warnings to, for example, motorcycles, bicycles, pedestrians, etc., around vehicle 1, thereby preventing accidents.
[0225] Furthermore, for example, the warning sound control unit 251 can change the characteristic quantity of the warning sound according to the properties of the detected object. For example, the warning sound control unit 251 can change the characteristic quantity of the warning sound according to the type or size of the detected object. As a result, the driver can intuitively identify the properties of the detected object.
[0226] Furthermore, when multiple objects are detected simultaneously, the warning sound control unit 251 can output multiple warning sounds to the corresponding objects at the same time. In this case, the warning sound control unit 251 can change the characteristic values of each warning sound to facilitate identification of the warning sound for the corresponding object. For example, as described above, the warning sound control unit 251 can change the characteristic values of the warning sounds output to each object according to the degree of danger of each object.
[0227] Incidentally, for example, when multiple detected objects are moving in a line, the warning sound control unit 251 may assume that the multiple objects moving in a line are one object and output a warning sound to that assumed object.
[0228] Additionally, for example, when vehicle 1 performs a predetermined operation, the warning sound control unit 251 may output a warning sound when an object is detected in the area.
[0229] For example, when vehicle 1 intends to change lanes or change direction (e.g., when the turn signal of vehicle 1 is activated), the warning sound control unit 251 may output a warning sound when an object is detected in the area. In this case, the warning sound is used as a function of the lane change decision support system ((LCDAS) lane change decision assistance system).
[0230] For example, when vehicle 1 wants to reverse (e.g., when vehicle 1 is set to reverse), the warning sound control unit 251 can output a warning sound when an object is detected in the area.
[0231] Furthermore, for example, if the identification component 73 detects a dangerous location, the warning sound control component 251 can output a warning sound for the dangerous location in a manner similar to that used when the aforementioned object is detected.
[0232] Here, we assume that a hazardous location is one where accidents frequently occur or where the probability of an accident is high. For example, a location with a high probability of an accident might be near the entrance to a park or school, or near a construction site.
[0233] In addition, for example, when vehicle 1 is performing autonomous driving, vehicle 1 basically avoids dangers and drives safely. Therefore, as mentioned above, it is not necessarily necessary to output a warning sound for detected objects.
[0234] On the other hand, for example, when vehicle 1 performs operations such as changing lanes, changing direction, parking, overtaking, or leaving during autonomous driving, it is assumed that an object constituting an obstacle is detected and the operation stops. In this case, for example, the warning sound control unit 251 can output a warning sound for the detected object. For example, if vehicle 1 does not turn right even when a turn signal is issued indicating that vehicle 1 intends to turn right, the warning sound control unit 251 can output a warning sound for an object that has caused vehicle 1 to stop turning right. Therefore, it is possible for one or more occupants to recognize the reason why vehicle 1 has stopped operating and to provide the occupants with a sense of security.
[0235] Additionally, for example, when vehicle 1 detects an object posing a high danger and automatically performs a hazard avoidance maneuver such as a pre-collision safety operation, the warning sound control unit 251 can output a warning sound for the object that has become the cause of the hazard avoidance maneuver. Therefore, occupants can identify the reason for vehicle 1 performing the hazard avoidance maneuver and receive a sense of security.
[0236] Furthermore, for example, through road-to-vehicle communication, the identification unit 73 can detect objects at intersections and even in areas that are blind spots, making detection by the external identification sensor 25 difficult. Similarly, in this case, the warning sound control unit 251 can output a warning sound for the detected object using a method similar to that described above.
[0237] Furthermore, for example, occupants may be annoyed when warning sounds are continuously emitted to vehicles around vehicle 1 during traffic congestion. Therefore, for example, the identification unit 73 can identify vehicles during traffic congestion, and the warning sound control unit 251 can exclude vehicles during traffic congestion from the list of objects receiving warning sounds. Additionally, if an object other than a vehicle during traffic congestion is detected in the area, the warning sound control unit 251 can output a warning sound to the object. Therefore, for example, during traffic congestion, warning sounds can be output to motorcycles or similar vehicles passing by on either side of the vehicle.
[0238] In addition, for example, a warning sound can be output only to a specific occupant (e.g., the driver) by using a corresponding seat speaker 204.
[0239] <Other modifications>
[0240] This technology can use acoustic imaging positioning technologies other than 360RA.
[0241] This technology is not specifically limited to any type of vehicle. Furthermore, it can be applied not only to vehicles, but also to mobile devices such as personal mobile devices, transport robots, aircraft, ships, construction machinery, and agricultural machinery.
[0242] <<4. Others>>
[0243] <Example of computer configuration>
[0244] The aforementioned series of processes can be performed by hardware or by software. In the case where the processes are performed by software, the program constituting the software is installed on the computer. Here, "computer" includes computers integrated into dedicated hardware, such as general-purpose personal computers capable of performing various functions by installing various programs on them.
[0245] Figure 18 This is a block diagram illustrating an example of the hardware configuration of a computer that performs the above series of processes through a program.
[0246] In computer 1000, CPU (Central Processing Unit) 1001, ROM (Read-Only Memory) 1002 and RAM (Random Access Memory) 1003 are interconnected via bus 1004.
[0247] The input / output interface 1005 is further connected to the bus 1004. The input / output interface 1005 is connected to the input unit 1006, the output unit 1007, the recording unit 1008, the communication unit 1009, and the driver 1010.
[0248] Input unit 1006 includes input switches, buttons, microphones, imaging elements, etc. Output unit 1007 includes displays, speakers, etc. Recording unit 1008 includes hard disks, non-volatile memory, etc. Communication unit 1009 includes network interfaces, etc. Driver 1010 drives removable media 1011 such as disks, optical disks, magneto-optical disks, semiconductor memories, etc.
[0249] In the computer 1000 configured as described above, the CPU 1001 loads a program recorded in the recording unit 1008 into the RAM 1003 via, for example, the input-output interface 1005 and the bus 1004, and executes the program. Thus, the series of processes described above are performed.
[0250] For example, a program to be executed by a computer 1000 (CPU 1001) can be provided in a state recorded on a removable medium 1011, such as a packaging medium. Alternatively, the program can be provided via a wired or radio transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
[0251] In computer 1000, a program can be installed in recording unit 1008 via input-output interface 1005 by loading removable medium 1011 into drive 1010. Alternatively, the program can be received by communication unit 1009 via wired or radio transmission medium and installed in recording unit 1008. Furthermore, the program can be pre-installed in ROM 1002 or recording unit 1008.
[0252] Incidentally, a program executed by a computer can be a program in which processing is performed sequentially in the order described in this specification, or a program in which processing is performed, for example, in parallel or at necessary time intervals when called.
[0253] Furthermore, in this specification, a system refers to a collection of multiple constituent elements (devices, modules (components), etc.), and it is not important whether all constituent elements are housed in the same housing. Therefore, multiple devices housed in separate housings and interconnected via a network, and a device having multiple modules housed in one housing, are both systems.
[0254] Furthermore, the embodiments of this technology are not limited to the foregoing embodiments, and various modifications can be made without departing from the spirit of this technology.
[0255] For example, this technology can employ a cloud computing configuration, where multiple devices share and jointly process a function via a network.
[0256] Furthermore, each step described in the flowchart above can be performed not only by one device, but also by multiple devices.
[0257] Furthermore, in cases where a step includes multiple processes, these processes can be performed not only by one device but also shared and executed by multiple devices.
[0258] <Examples of configuration combinations>
[0259] This technology can also be configured as follows.
[0260] (1) An information processing device, comprising:
[0261] The recognition component is configured to perform detection processing on objects around the mobile device; and
[0262] A warning sound control unit is configured to control the characteristic quantity of a first warning sound and the position of the sound image of the first warning sound based on the position of the detected object, the first warning sound indicating the presence of the object.
[0263] (2) The information processing device according to (1) above, wherein
[0264] The identification component detects the relative speed of the object with respect to the mobile device, and
[0265] The warning sound control unit also controls the characteristic quantity of the first warning sound based on the relative speed of the object.
[0266] (3) The information processing device according to (1) or (2) above, wherein
[0267] Identify components to identify object attributes, and
[0268] The warning sound control unit also controls at least one of the characteristic quantity of the first warning sound and the position of the sound image of the first warning sound based on the properties of the object.
[0269] (4) The information processing equipment according to (3) above, wherein
[0270] An object's properties include its size, and
[0271] The warning sound control unit controls the position of the sound image of the first warning sound in the height direction according to the size of the object.
[0272] (5) An information processing device according to any one of (1) to (4) above, wherein
[0273] Identify components to estimate the degree of danger of an object, and
[0274] The warning sound control unit also controls the characteristic quantity of the first warning sound according to the degree of danger of the object.
[0275] (6) The information processing device according to (5) above, wherein
[0276] The identification component estimates which is higher: the probability that the mobile device will cause damage to an object, or the probability that the mobile device will be damaged by an object.
[0277] The warning sound control unit changes the characteristic quantity of the first warning sound between a situation where the mobile device is more likely to cause damage to an object and a situation where the mobile device is more likely to be damaged by an object.
[0278] (7) The information processing device according to (5) or (6) above, wherein
[0279] The warning sound control unit controls the output of a second warning sound to the object to indicate the presence of the mobile device, based on the degree of danger that the mobile device poses to the object.
[0280] (8) An information processing device according to any one of (1) to (7) above, wherein,
[0281] When multiple objects are detected, the warning sound control unit performs control to output multiple first warning sounds simultaneously, each first warning sound indicating the presence of a corresponding object.
[0282] (9) The information processing device according to (8) above, wherein
[0283] Identify components to estimate the hazard level of each object, and
[0284] The warning sound control unit controls the corresponding characteristic quantity of the first warning sound according to the degree of danger of each object.
[0285] (10) The information processing device according to (8) or (9) above, wherein,
[0286] When the multiple objects are moving in a queue, the warning sound control unit treats the multiple objects as a single object and executes control to output a first warning sound.
[0287] (11) An information processing device according to any one of (1) to (10) above, wherein
[0288] The recognition component defines multiple zones around the mobile device and detects the areas where objects are present.
[0289] The warning sound control unit controls the characteristic quantity of the first warning sound and the position of the sound image of the first warning sound based on the area where the object is located.
[0290] (12) The information processing device according to (11) above, wherein
[0291] The identification component changes the area settings based on the mobile device's location.
[0292] (13) An information processing device according to any one of (1) to (12) above, wherein
[0293] The warning sound control unit controls the output of a first warning sound when an object is detected while the mobile device is performing a predetermined operation.
[0294] (14) An information processing device according to any one of (1) to (13) above, wherein
[0295] The warning sound control unit controls the output of a first warning sound when the mobile device stops a predetermined operation due to a detected object while it is in autonomous driving mode.
[0296] (15) An information processing device according to any one of (1) to (14) above, wherein
[0297] The identification component uses road-to-vehicle communication to identify objects around the mobile device, and
[0298] The warning sound control unit controls the output of a first warning sound when an object is detected in the blind spot of the mobile device via road-to-vehicle communication.
[0299] (16) An information processing device according to any one of (1) to (15) above, wherein
[0300] The identification component obtains information about the road the mobile device is traveling on based on at least one of the sensors contained in the mobile device and a map of the area surrounding the mobile device, and corrects the position of the object with reference to the information about the road.
[0301] (17) An information processing method, comprising the following steps:
[0302] Perform detection processing on objects around the mobile device; and
[0303] The location of the warning sound's characteristic quantity and acoustic image is controlled based on the location of the detected object, and the warning sound indicates the presence of the object.
[0304] (18) A program for causing a computer to perform a process, the process comprising:
[0305] Perform detection processing on objects around the mobile device; and
[0306] The location of the warning sound's characteristic quantity and acoustic image is controlled based on the location of the detected object, and the warning sound indicates the presence of the object.
[0307] (19) A mobile device, comprising:
[0308] The recognition component is configured to perform detection processing on objects around the mobile device; and
[0309] The warning sound control unit is configured to control the characteristic quantity of the warning sound and the position of the sound image of the warning sound based on the position of the detected object, the warning sound indicating the presence of the object.
[0310] (20) An information processing system, comprising:
[0311] The sensor unit is configured to sense the surroundings of the mobile device;
[0312] The identification component is configured to perform detection processing on objects around the mobile device based on the sensing results of the sensor unit.
[0313] A warning sound control unit is configured to control the characteristic quantity of a warning sound and the position of the sound image of the warning sound based on the position of a detected object, the warning sound indicating the presence of the object; and
[0314] The speaker is configured to output a warning sound under the control of the warning sound control unit.
[0315] It should be noted that the effects described in this specification are merely illustrative and not limiting, and may have other effects as well.
[0316] [List of Reference Symbols]
[0317] 1: Vehicle
[0318] 11: Vehicle Control System
[0319] 25: External identification sensor
[0320] 31: HMI
[0321] 51: Camera
[0322] 52: Radar
[0323] 53: LiDAR
[0324] 54: Ultrasonic sensor
[0325] 73: Identification Components
[0326] 62: Action Plan Component
[0327] 63: Operation control unit
[0328] 202FL to 202BR: Speakers
[0329] 251: Alarm sound control unit
Claims
1. An information processing device, comprising: The recognition component is configured to perform detection processing on objects around the mobile device; as well as A warning sound control unit is configured to control the characteristic quantity of a first warning sound and the position of the sound image of the first warning sound based on the position of a detected object, the first warning sound indicating the presence of the object. Among them, the identification component estimates whether the probability of the mobile device causing damage to an object is higher, or the probability of the mobile device being damaged by an object. The warning sound control unit changes the characteristic quantity of the first warning sound between a situation where the mobile device is more likely to cause damage to an object and a situation where the mobile device is more likely to be damaged by an object.
2. The information processing device according to claim 1, wherein... The identification component detects the relative speed of the object with respect to the mobile device, and The warning sound control unit also controls the characteristic quantity of the first warning sound based on the relative speed of the object.
3. The information processing device according to claim 1, wherein... Identify components to identify object attributes, and The warning sound control unit also controls at least one of the characteristic quantity of the first warning sound and the position of the sound image of the first warning sound based on the properties of the object.
4. The information processing device according to claim 3, wherein An object's properties include its size, and The warning sound control unit controls the position of the sound image of the first warning sound in the height direction according to the size of the object.
5. The information processing device according to claim 1, wherein... Identify components to estimate the degree of danger of an object, and The warning sound control unit also controls the characteristic quantity of the first warning sound according to the degree of danger of the object.
6. The information processing device according to claim 1, wherein... The warning sound control unit controls the output of a second warning sound to the object to indicate the presence of the mobile device, based on the degree of danger that the mobile device poses to the object.
7. The information processing device according to claim 1, wherein, When multiple objects are detected, the warning sound control unit performs control to output multiple first warning sounds simultaneously, each first warning sound indicating the presence of a corresponding object.
8. The information processing device according to claim 7, wherein Identify components to estimate the hazard level of each object, and The warning sound control unit controls the corresponding characteristic quantity of the first warning sound according to the degree of danger of each object.
9. The information processing device according to claim 7, wherein, When the multiple objects are moving in a queue, the warning sound control unit treats the multiple objects as a single object and executes control to output a first warning sound.
10. The information processing device according to claim 1, wherein The recognition component defines multiple zones around the mobile device and detects the areas where objects are present. The warning sound control unit controls the characteristic quantity of the first warning sound and the position of the sound image of the first warning sound based on the area where the object is located.
11. The information processing device according to claim 10, wherein... The identification component changes the area settings based on the mobile device's location.
12. The information processing device according to claim 1, wherein The warning sound control unit controls the output of a first warning sound when an object is detected while the mobile device is performing a predetermined operation.
13. The information processing device according to claim 1, wherein... The warning sound control unit controls the output of a first warning sound when the mobile device stops a predetermined operation due to a detected object while it is in autonomous driving mode.
14. The information processing device according to claim 1, wherein... The identification component uses road-to-vehicle communication to identify objects around the mobile device, and The warning sound control unit controls the output of a first warning sound when an object is detected in the blind spot of the mobile device via road-to-vehicle communication.
15. The information processing device according to claim 1, wherein... The identification component obtains information about the road the mobile device is traveling on based on at least one of the sensors contained in the mobile device and a map of the area surrounding the mobile device, and corrects the position of the object with reference to the information about the road.
16. An information processing method, comprising the following steps: Perform detection processing on objects around the mobile device; The location of the warning sound's characteristic quantity and acoustic image is controlled based on the location of the detected object, and the warning sound indicates the presence of the object; Estimate which is higher: the probability that the mobile device will cause damage to the object or the probability that the mobile device will be damaged by the object. as well as The characteristic value of the first warning sound is changed between a situation where the mobile device is more likely to cause damage to an object and a situation where the mobile device is more likely to be damaged by an object.
17. A computer program product comprising instructions that, when executed by a processor, perform the following processing: Perform detection processing on objects around the mobile device; The location of the warning sound's characteristic quantity and acoustic image is controlled based on the location of the detected object, and the warning sound indicates the presence of the object; Estimate which is higher: the probability that the mobile device will cause damage to the object or the probability that the mobile device will be damaged by the object. as well as The characteristic value of the first warning sound is changed between a situation where the mobile device is more likely to cause damage to an object and a situation where the mobile device is more likely to be damaged by an object.
18. A mobile device, comprising: The recognition component is configured to perform detection processing on objects around the mobile device; as well as The warning sound control unit is configured to control the characteristic quantity of the warning sound and the position of the sound image of the warning sound based on the position of the detected object, the warning sound indicating the presence of the object. Among them, the identification component estimates whether the probability of the mobile device causing damage to an object is higher, or the probability of the mobile device being damaged by an object. The warning sound control unit changes the characteristic quantity of the first warning sound between a situation where the mobile device is more likely to cause damage to an object and a situation where the mobile device is more likely to be damaged by an object.
19. An information processing system, comprising: The sensor unit is configured to sense the surroundings of the mobile device; The identification component is configured to perform detection processing on objects around the mobile device based on the sensing results of the sensor unit. A warning sound control unit is configured to control the characteristic quantity of a warning sound and the position of the sound image of the warning sound based on the position of a detected object, the warning sound indicating the presence of the object; as well as The speaker is configured to output a warning sound under the control of the warning sound control unit. Among them, the identification component estimates whether the probability of the mobile device causing damage to an object is higher, or the probability of the mobile device being damaged by an object. The warning sound control unit changes the characteristic quantity of the first warning sound between a situation where the mobile device is more likely to cause damage to an object and a situation where the mobile device is more likely to be damaged by an object.
Citation Information
Patent Citations
Driving support device
JP2007253721A
Three-dimensional sound-filed alarm device
JP1993126948A
Vehicle warning device
JP2007328603A
Proximity notification device and proximity alarm program
JP2011118753A
