Information processing apparatus, information processing method, and computer program product

Abstract

This invention provides an information processing apparatus, an information processing method, and a computer program product. It improves the visibility of the surrounding conditions of the vehicle. The information processing apparatus (10) includes a display control unit (22C). The display control unit (22C) displays a rear view image captured by a rear view camera (13) that captures the rear of the vehicle (1) on an electronic rearview mirror (18) installed in the vehicle (1). If a first condition is met—that a two-wheeled vehicle exists that is not captured in the rear view image but is captured in the rear side view image—the display control unit (22C) displays a second display image obtained by superimposing the rear side view image onto a portion of the rear view image on the electronic rearview mirror (18). This rear side view image is captured by rear side view cameras (12) that capture the rear of the left and right sides of the vehicle (1), respectively.

Description

Technical Field

[0001] This disclosure relates to information processing apparatus, information processing method, and computer program products. Background Technology

[0002] Previously, the introduction of functions to notify drivers of other vehicles in the vicinity of their vehicle has been progressing. For example, the following functions have been disclosed: when the distance between a vehicle and a vehicle behind is greater than a specified distance, the image captured by the rearview camera is displayed on the electronic rearview mirror; when the distance is less than the specified distance, the image captured by the side-view camera is displayed on the electronic rearview mirror.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-69862 Summary of the Invention

[0006] The problem the invention aims to solve

[0007] However, in existing technology, even when the driver needs to confirm the image captured by the rearview camera, the electronic rearview mirror display is sometimes switched to show only the image from the side-view camera. Furthermore, in existing technology, even when the driver needs to confirm the image captured by the side-view camera, sometimes only the image from the rearview camera is displayed on the electronic rearview mirror. That is, in existing technology, the visibility of the vehicle's surroundings is sometimes reduced.

[0008] The problem to be solved by this disclosure is to provide an information processing device, information processing method, and computer program product that can improve the visibility of the surrounding conditions of the vehicle.

[0009] Solution for solving the problem

[0010] The information processing apparatus disclosed herein includes a display control unit. The display control unit performs the following processing: displays a rear view captured by a rear-view camera that captures the rear of the vehicle on a display device installed in the vehicle; and, if a first condition is met—that there is a two-wheeled vehicle not captured in the rear view but captured in a rear-side view—displays a second display image obtained by superimposing the rear-side view onto a portion of the rear view on the display device, wherein the rear-side view is an image captured by a rear-side camera that captures the rear of the left side and the rear of the right side of the vehicle, respectively.

[0011] The effects of the invention

[0012] According to the information processing apparatus, information processing method and information processing program disclosed herein, it is possible to improve the visibility of the surrounding conditions of the vehicle. Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating an example of the functional structure of an information processing device.

[0014] Figure 2 This is a schematic diagram illustrating an example of the driving environment of this vehicle.

[0015] Figure 3 This is a schematic diagram showing the visual observation from inside the vehicle's passenger compartment towards the front of the vehicle.

[0016] Figure 4 This is a hardware structure diagram of an example of an information processing device.

[0017] Figure 5A This is a schematic diagram of an example of an image displayed in an electronic rearview mirror.

[0018] Figure 5B This is a schematic diagram of an example of an image displayed in an electronic rearview mirror.

[0019] Figure 5C This is a schematic diagram of an example of a second display image shown in an electronic rearview mirror.

[0020] Figure 5D This is a schematic diagram of an example of a second display image shown in an electronic rearview mirror.

[0021] Figure 6A This is a flowchart illustrating an example of the information processing flow performed by an information processing device.

[0022] Figure 6B This is a flowchart illustrating an example of the information processing flow performed by an information processing device. Detailed Implementation

[0023] The embodiments of the information processing apparatus, information processing method, and information processing program involved in this disclosure will now be described with reference to the accompanying drawings.

[0024] Figure 1 This is a block diagram illustrating an example of the functional structure of the information processing apparatus 10 in this embodiment.

[0025] The information processing device 10 is a device used to improve the visibility of the surrounding conditions of other vehicles and the like around the vehicle 1. In this embodiment, the information processing device 10 is mounted on the vehicle 1 as an example for explanation.

[0026] Vehicle 1 is a vehicle equipped with the information processing device 10. Other vehicles are vehicles other than those equipped with the information processing device 10.

[0027] In this embodiment, vehicle 1 is described as a four-wheeled vehicle as an example. Other vehicles include two-wheeled vehicles.

[0028] Two-wheeled vehicles are vehicles with two wheels. Examples of two-wheeled vehicles include motorcycles, scooters, electric bicycles, bicycles, and electric skateboards. Four-wheeled vehicles are vehicles with four or more wheels. Examples of four-wheeled vehicles include regular cars, small cars, medium-sized cars, large cars, light cars, and special-purpose vehicles.

[0029] This vehicle 1 is equipped with an information processing device 10, a communication unit 11, a rear side camera unit 12, a rear camera unit 13, a sensor 14, a UI (user interface) unit 15, a driving control unit 16, an operation unit 17, an electronic rearview mirror 18, and a storage unit 19.

[0030] The communication unit 11, the rear side camera unit 12, the rear camera unit 13, the sensor 14, the UI unit 15, the driving control unit 16, the operation unit 17, the electronic rearview mirror 18, the storage unit 19, and the information processing device 10 are connected in a communicable manner via the bus 20, etc. At least a part of the bus 20 may also use a local area network such as CAN (Controller Area Network).

[0031] The communication unit 11 is a communication interface for communicating with external devices. For example, the communication unit 11 communicates with external information processing devices via a network. Additionally, the communication unit 11 can also communicate with other vehicles via vehicle-to-vehicle communication, such as V2V (vehicle-to-vehicle) communication.

[0032] The rear-side camera unit 12 is a camera device that captures images of the rear side of the vehicle 1 and obtains captured image data. The rear-side camera unit 12 is, for example, a camera. The rear-side camera unit 13 is a camera device that captures images of the rear of the vehicle 1 and obtains captured image data. The rear-side camera unit 13 is, for example, a camera. Hereinafter, the captured image data will be simply referred to as captured images. In this embodiment, the rear-side camera unit 12 and the rear-side camera unit 13 continuously capture images over time and sequentially output the captured images obtained through the capture to the information processing device 10.

[0033] Figure 2 This is a schematic diagram illustrating an example of the driving environment of vehicle 1. Figure 3 This is a schematic diagram showing the visual observation from inside the passenger compartment of vehicle 1 towards the front of the vehicle.

[0034] The rear-side camera unit 12 captures images of the rear left side and the rear right side of the vehicle 1, respectively, and obtains rear-side images V, i.e., rear-side images VS, of the rear left and right sides of the vehicle 1. The rear-side camera unit 12 includes a right rear-side camera unit 12R and a left rear-side camera unit 12L.

[0035] The right rear-side camera unit 12R is a rear-side camera unit 12 that takes the rear right side of the vehicle 1 as its shooting area. The right rear-side camera unit 12R is pre-adjusted in terms of shooting angle, setting position, shooting direction, etc., so as to be able to shoot the rear right side of the vehicle 1's surroundings as far as the vehicle 1 is concerned. In detail, the right rear-side camera unit 12R shoots the right rear side and a portion of the right side of the vehicle 1 by pointing the shooting optical axis (lens) toward the right rear of the vehicle 1. The right rear-side camera unit 12R is provided, for example, in the side rearview mirror 24 of the vehicle 1, which is arranged on the right side relative to the steering wheel 23.

[0036] As an example, the right rear-side imaging unit 12R lacks a mechanism for changing the orientation of the imaging optical axis, the lens has a fixed focus, and the shooting angle is set to a relatively wide angle. Therefore, the right rear-side imaging unit 12R captures a relatively wide-angle and fixed shooting angle shooting area GR on the right rear side of the vehicle 1, obtaining a right rear-side image VSR as the captured image of this shooting area GR. The right rear-side image VSR is a rear-side image VS captured by the right rear-side imaging unit 12R in the rear-side imaging unit 12. The rear-side image VS is an example of an image V captured by the rear-side imaging unit 12 and the rear imaging unit 13.

[0037] The left rear-side camera unit 12L is a rear-side camera unit 12 that takes the rear left side of the vehicle 1 as its shooting area. The left rear-side camera unit 12L is pre-adjusted in terms of shooting angle, setting position, shooting direction, etc., so as to be able to shoot the rear left side of the vehicle 1's perimeter. In detail, the left rear-side camera unit 12L shoots the left rear side and a portion of the left side of the vehicle 1 by pointing the shooting optical axis (lens) towards the left rear of the vehicle 1. The left rear-side camera unit 12L is provided, for example, in the side rearview mirror 24 of the vehicle 1, which is positioned on the left side relative to the steering wheel 23.

[0038] As an example, the left rear-side imaging unit 12L does not have a mechanism for changing the orientation of the imaging optical axis, the lens has a fixed focus, and the shooting angle is set to a relatively wide wide-angle. Therefore, the left rear-side imaging unit 12L captures a relatively wide-angle and fixed shooting angle shooting area GL on the left rear side of the vehicle 1, and obtains a left rear-side image VSL as the captured image of the shooting area GL. The left rear-side image VSL is the rear-side image VS captured by the left rear-side imaging unit 12L in the rear-side imaging unit 12.

[0039] The rear camera unit 13 is a camera unit that uses the rear side (rear side) of the perimeter of the vehicle 1 as its shooting area. The rear camera unit 13 is pre-adjusted in terms of shooting angle, setting position, shooting direction, etc., so as to be able to shoot the rear side of the perimeter of the vehicle 1 as its rear side. In detail, the rear camera unit 13 shoots the rear side of the vehicle 1 by pointing the shooting optical axis (lens) towards the rear of the vehicle 1.

[0040] For example, the rear camera unit 13 is positioned at the rear of the trunk lid of vehicle 1, with its optical axis (lens) pointing towards the rear of vehicle 1, capturing the rear side of vehicle 1. Furthermore, the rear camera unit 13 can be positioned where it can capture the rear side; it can be positioned near the rear bumper of vehicle 1 or positioned to capture the rear side of vehicle 1 through the rear windshield. Regarding the rear camera unit 13, the lens is set to a fixed focal point, and no mechanism is provided for changing the orientation of the optical axis. It captures a fixed-viewing-angle area GB to obtain a rear image VB, which is the captured image of that area GB. The rear image VB is an example of an image V captured by the rear side camera unit 12 and the rear camera unit 13.

[0041] return Figure 1 Let me continue explaining.

[0042] Sensor 14 is mounted on the vehicle 1 to detect the internal and external conditions of the vehicle 1. Sensor 14 includes an external sensor and an internal sensor.

[0043] An external sensor is mounted on the vehicle 1 to detect the external conditions of the vehicle 1. In this embodiment, the external sensor detects the presence or absence of other vehicles 2, the distance to other vehicles 2, etc. Examples of external sensors include distance sensors, sonar sensors that detect objects using sound waves, and ultrasonic sensors. Examples of distance sensors include millimeter-wave radar and laser sensors. Examples of laser sensors include two-dimensional LiDAR (Laser Imaging Detection and Range) sensors and three-dimensional LiDAR sensors arranged parallel to the horizontal plane. In this embodiment, the external sensor is positioned to detect other vehicles 2 present at least behind the vehicle 1. Furthermore, the external sensor is pre-adjusted to detect other vehicles 2 at least behind the vehicle 1; the number and arrangement of the external sensors on the vehicle 1 are not limited.

[0044] Internal sensors are sensors that detect the state of vehicle 1. Internal sensors detect the position, speed, acceleration, throttle opening, steering angle of the steering system, and brake pedal depressure of vehicle 1. Examples of internal sensors include GPS (Global Positioning System), speed sensors, acceleration sensors such as IMU (Inertial Measurement Unit) that detect acceleration acting on vehicle 1 in at least the longitudinal direction, throttle opening sensors that detect throttle opening, steering angle sensors that detect steering angle of the steering system, and sensors that detect brake pedal depressure.

[0045] UI unit 15 is a user interface used for user input and information output. UI unit 15 may include functions for outputting images and sound, and functions for receiving user input instructions. Output functions may include, for example, a combination instrument cluster displaying information to the driver, a touch panel, and notification tone generating devices such as a buzzer and a speaker. UI unit 15 may also be part of an HMI (Human Machine Interface) or IVI (In-Vehicle Infotainment) system.

[0046] The driving control unit 16 is an ECU (Engine Control Unit) that controls the driving of the vehicle 1. The driving control unit 16 performs control of the vehicle 1's engine, motor and other drive devices, as well as the vehicle 1's transmission system devices, based on the passenger's operation information received from the operation unit 17 and the detection results of the sensor 14.

[0047] The operating unit 17 is operated by the occupants of the vehicle 1. The operating unit 17 includes, for example, an ignition switch, a gear lever, a steering wheel 23, a direction indicator, an accelerator pedal, and a brake pedal. However, the operating unit 17 mounted on the vehicle 1 is not limited to these.

[0048] The driving control unit 16 controls the drive unit and transmission system of the vehicle 1 based on the operation information of the ignition switch, the gear position information of the gear lever, the operation information of the direction indicator, the steering angle indicated by the steering operation amount, the accelerator pedal operation information of the accelerator pedal, and the brake pedal information of the brake pedal.

[0049] The electronic rearview mirror 18 is a device that displays the surrounding images V of the vehicle 1 on its mirror surface. Depending on the manufacturer of the vehicle 1, the electronic rearview mirror 18 is sometimes referred to as a smart rearview mirror (SRVM), smart room mirror, intelligent room mirror, etc. The electronic rearview mirror 18 is an example of a display device.

[0050] like Figure 3 As shown, the electronic rearview mirror 18 is positioned on the upper side of the windshield of the vehicle and approximately at the center in the width direction of the vehicle 1. The electronic rearview mirror 18 is constructed, for example, of an LCD panel and is capable of displaying an image V facing inwards towards the interior of the vehicle 1. The electronic rearview mirror 18 displays an image V captured by at least one of the rear side camera unit 12 and the rear camera unit 13, through a process described later. The electronic rearview mirror 18 functions in place of the interior rearview mirror, which is an optical mirror, allowing the driver to visually confirm the surrounding conditions of the vehicle 1, centered on the rear of the vehicle 1, by visually confirming the image V displayed on the electronic rearview mirror 18.

[0051] return Figure 1 Let me continue explaining.

[0052] Storage unit 19 stores various types of data. Alternatively, at least a portion of the data contained in storage unit 19 may be stored in an external storage device that is communicatively connected to information processing device 10.

[0053] Next, the information processing device 10 will be described in detail.

[0054] Figure 4 This is a hardware structure diagram of an example of an information processing device 10.

[0055] The information processing device 10 interconnects the CPU (Central Processing Unit) 11A, ROM (Read Only Memory) 11B, RAM (Random Access Memory) 11C, and I / F (Interface) 11D via bus 11E, thus utilizing the hardware structure of a typical computer.

[0056] CPU 11A is a computing unit that controls the information processing apparatus 10 of this embodiment. ROM 11B stores programs that implement various processes performed by CPU 11A. RAM 11C stores data required for various processes performed by CPU 11A. I / F 11D is an interface for sending and receiving data.

[0057] The information processing device 10 can also be implemented using a computer such as an ECU (Electronic Control Unit), a CDC (Cockpit Domain Controller) integrating multiple ECUs, or an OBU (On Board Unit) installed inside the vehicle 1. In this case, the information processing device 10 can transmit and receive information with other ECUs installed in the vehicle 1 and devices connected to the vehicle 1 via an in-vehicle network including CAN (Controller Area Network), Ethernet, USB (Universal Serial Bus), etc., or it can communicate with external information processing devices via a network such as the Internet. Furthermore, the information processing device 10 according to the embodiment can be implemented using a single ECU, or it can be implemented using other ECUs, DCUs, or OBUs that control various functions of the vehicle 1, such as power steering control, throttle control, braking control, other charging control, and automatic driving control, or it can be implemented in cooperation with them.

[0058] The information processing device 10 is not limited to the CPU 11A, but may also have various processors such as GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), and FPGA (Field Programmable Gate Array).

[0059] In addition, the information processing device 10 may also have various recording media and recording devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), and Flash memory as auxiliary storage devices.

[0060] Programs for performing the various processes described above executed by the information processing device 10 are provided pre-loaded into ROM 11B or the like. Programs for performing the vehicle control method executed in this embodiment may also be provided as files that can be installed into or executed by these devices, stored on computer-readable recording media such as CD-ROM, floppy disk (FD), CD-R, or DVD (Digital Versatile Disc).

[0061] Alternatively, the program executed by the information processing apparatus 10 according to this embodiment may be stored on a computer connected to a network such as the Internet, and provided by downloading it via the network. Alternatively, the program executed by the information processing apparatus 10 according to this embodiment may be provided or distributed via a network such as the Internet.

[0062] Alternatively, the program executed by the information processing device 10 according to this embodiment may be pre-loaded into the ROM 11B or the like for provision or distribution.

[0063] For example, at least one processor (e.g., CPU 11A) of the information processing device 10 loads a program stored in an auxiliary storage device (e.g., ROM 11B) into a main storage device (e.g., RAM 11C) and executes the loaded program, thereby realizing the various functions of the information processing device 10. Alternatively, at least one processor of the information processing device 10 may also be configured as a dedicated hardware circuit.

[0064] return Figure 1 Let me continue explaining.

[0065] The information processing apparatus 10 includes a processing unit 22. The processing unit 22 performs various information processing tasks. For example, the CPU 11A reads a program from the ROM 11B into the RAM 11C and executes it, thereby implementing the various functional units of the processing unit 22 described later on the computer.

[0066] The processing unit 22 includes an acquisition unit 22A, a driving status export unit 22B, and a display control unit 22C. Part or all of the acquisition unit 22A, the driving status export unit 22B, and the display control unit 22C can be implemented, for example, by executing a program (i.e., software) using a processing device such as a CPU 11A, or by implementing hardware such as an IC (Integrated Circuit), or by combining software and hardware. Alternatively, at least one of the acquisition unit 22A, the driving status export unit 22B, and the display control unit 22C can be mounted on an external information processing device that is communicatively connected to the information processing device 10 via a network or the like.

[0067] The acquisition unit 22A acquires images V from the rear-side imaging unit 12 and the rear-side imaging unit 13. Specifically, the acquisition unit 22A acquires the right rear-side image VSR from the right rear-side imaging unit 12R, the left rear-side image VSL from the left rear-side imaging unit 12L, and the rear-side image VB from the rear-side imaging unit 13. Additionally, the acquisition unit 22A acquires the detection results from the sensor 14. Furthermore, the driving control unit 16 acquires information indicating the driving control content of the driving control unit 16.

[0068] The driving status export unit 22B exports the driving status of this vehicle 1.

[0069] Driving conditions refer to information including at least one of the following: the types of other vehicles 2 included in the rear view VB and the rear side view VS respectively; the road width W of the road R on which this vehicle 1 is traveling (also refer to...). Figure 2 The following information is included: the speed of vehicle 1; the distance between vehicle 1 and other vehicles 2 following behind vehicle 1; the total number of other vehicles 2 captured in the rear view VB; the type of adjacent area adjacent to the lane of vehicle 1; the relative speed between vehicle 1 and the detected two-wheeled vehicle 2A; the distance between vehicle 1 and the detected two-wheeled vehicle 2A; whether other vehicles 2 captured in the rear view VB are vehicles to be watched; the position of vehicles to be watched in the rear view VB; and whether vehicle 1 is in a state where it cannot change lanes or turn left or right.

[0070] The other vehicle type 2 refers to either a two-wheeled vehicle 2A or a four-wheeled vehicle 2B. The driving status derivation unit 22B uses known methods such as pattern matching and AI (Artificial Intelligence) learned through deep learning to analyze the rear image VB and the rear side image VS respectively, in order to determine the other vehicles 2 captured in the rear image VB and the rear side image VS respectively, and to determine the type of the other vehicles 2.

[0071] The road width W of the road R on which vehicle 1 is traveling represents the width of the road on which vehicle 1 is traveling (refer to...). Figure 2 Furthermore, the road width W of the driving road R of this vehicle 1 can also be any one of the width of the lane in which this vehicle 1 is traveling, and the combined width of the lane in which this vehicle 1 is traveling and other lanes that are adjacent to this lane and extend in the same direction as the extension direction of this lane.

[0072] The driving status export unit 22B can export the road width W of the driving road R of the vehicle 1 using the detection results of the sensor 14, the image V, etc., through known methods. Alternatively, the road width W can be exported by obtaining the road width W of the driving road R of the vehicle 1 from the driving control unit 16 via CAN or the like. Furthermore, the driving status export unit 22B can also export the road width W of the driving road R of the vehicle 1 from an external information processing device such as a communication unit 11, using map data or the like.

[0073] The vehicle speed of vehicle 1 refers to the current speed of vehicle 1. The driving status derivation unit 22B determines the vehicle speed of vehicle 1 by acquiring the vehicle speed detected by the internal sensors. Alternatively, the driving status derivation unit 22B can also determine the vehicle speed of vehicle 1 by acquiring the vehicle speed of vehicle 1 from the driving control unit 16 via CAN or the like.

[0074] The distance between vehicle 1 and other vehicles 2 following vehicle 1 refers to the distance between vehicle 1 and the closest other vehicle 2 following vehicle 1 that is traveling behind vehicle 1. Other vehicles 2 following vehicle 1 refer to other vehicles 2 captured in the rear view VB.

[0075] The driving status derivation unit 22B determines the shortest distance between the vehicle 1 and other vehicles 2 detected by the external sensor as the vehicle distance from the vehicle 1 to the other vehicle 2. The external sensor uses the area GB encompassing the image frame VB as its detection range. Alternatively, for example, the driving status derivation unit 22B can also simulate the distance from the vehicle 1 to other vehicles 2 following behind the vehicle 1 using known methods, based on the lower position and lateral width of objects, by combining the rear image VB with object detection using AI learned through deep learning or similar methods. Alternatively, the driving status derivation unit 22B can also calculate the difference between the position of the vehicle 1 detected by the internal sensor and the position of the other vehicle 2 received from the other vehicle 2 via vehicle-to-vehicle communication, etc., as the vehicle distance between the vehicle 1 and the other vehicle 2.

[0076] The total number of other vehicles 2 captured in the rear view VB refers to the total number of all other vehicles 2 captured in the rear view VB.

[0077] The driving status derivation unit 22B derives the total number of other vehicles 2 captured in the rear view image VB by combining the rear view image VB with object detection using AI learned through deep learning or similar methods. Alternatively, the driving status derivation unit 22B can also derive the total number of other vehicles 2 captured in the rear view image VB by obtaining the total number of other vehicles 2 present in the shooting area GB of the rear view image VB of the vehicle 1 from the driving control unit 16 via CAN or the like. Furthermore, the driving status derivation unit 22B can also derive the total number of other vehicles 2 captured in the rear view image VB by receiving information from other vehicles 2 present in the shooting area GB via vehicle-to-vehicle communication or the like.

[0078] The type of adjacent area adjacent to the driving lane of this vehicle 1 indicates information that the area adjacent to the driving road R of this vehicle 1 is either an area where vehicles are prohibited or have difficulty driving, or an area where vehicles are permitted or can drive. An area where vehicles are prohibited or have difficulty driving is, for example, a sidewalk. An area where vehicles are permitted or can drive is, for example, a roadway. In this embodiment, the adjacent area type is described as either a roadway or a sidewalk as an example.

[0079] The driving condition derivation unit 22B determines whether an area adjacent to the driving road R of the vehicle 1 is a roadway or a pedestrian walkway by combining the rear side image VS and the rear image VB with area detection using AI learned through deep learning or similar methods. Alternatively, for example, the driving condition derivation unit 22B can also determine whether an adjacent area is a roadway or a pedestrian walkway by obtaining the type of adjacent areas of the vehicle 1 from the driving control unit 16 via CAN or the like. Furthermore, for example, the driving condition derivation unit 22B can also receive the type of adjacent areas of the vehicle 1 derived using map data from an external information processing device such as the communication unit 11, thereby determining whether the type of adjacent areas adjacent to the driving lane of the vehicle 1 is a roadway or a pedestrian walkway.

[0080] The relative speed between vehicle 1 and the detected two-wheeled vehicle 2A represents the relative speed of the detected two-wheeled vehicle 2A relative to vehicle 1. The detected two-wheeled vehicle 2A refers to the two-wheeled vehicle 2A that is captured in the rear side image VS but not in the rear image VB. Specifically, the detected two-wheeled vehicle 2A is the two-wheeled vehicle 2A that satisfies the first condition described later. Figure 2 In the indicated state, the detected two-wheeled vehicle 2A is two-wheeled vehicle 2A1 among two-wheeled vehicles 2A1 and 2A2 traveling around the vehicle 1. The driving status derivation unit 22B derives the relative speed of the two-wheeled vehicle 2A relative to the vehicle 1, for example, based on changes in at least one of the position and size of the two-wheeled vehicle 2A in the rear-view image VS, using a known method. Alternatively, the driving status derivation unit 22B can also derive the relative speed using the speed of the vehicle 1 detected by its internal sensors and the speed of the two-wheeled vehicle 2A obtained from the two-wheeled vehicle 2A via vehicle-to-vehicle communication.

[0081] The distance between the vehicle 1 and the detected two-wheeled vehicle 2A represents the distance between the vehicle 1 and the detected two-wheeled vehicle 2A. The definition of the detected two-wheeled vehicle 2A is the same as described above. The driving condition derivation unit 22B determines the shortest distance between the vehicle 1 and the two-wheeled vehicle 2A detected by the external sensor as the distance from the vehicle 1 to the two-wheeled vehicle 2A. The external sensor uses the range of at least one of the shooting area GL and the shooting area GR of the rear side image VS as the detection range. Alternatively, for example, the driving condition derivation unit 22B can also estimate the distance from the vehicle 1 to the two-wheeled vehicle 2A by simulating the distance using known methods, such as the combination of the rear side image VS and object detection using AI learned through deep learning, etc. Alternatively, for example, the driving status output unit 22B can calculate the difference between the position of the vehicle 1 detected by the internal sensor and the position of the two-wheeled vehicle 2A received from the two-wheeled vehicle 2A through vehicle-to-vehicle communication, etc., as the distance between the vehicle 1 and the two-wheeled vehicle 2A.

[0082] The question of whether other vehicles 2 captured in the rear view VB are designated as "pay attention vehicles" indicates whether they are pre-defined vehicles that require attention. "Pay attention vehicles" refers to other vehicles 2 that may need to be alerted to the driver of vehicle 1. Examples of vehicles requiring attention include emergency vehicles such as fire trucks and ambulances, and vehicles driven aggressively.

[0083] The driving status export unit 22B pre-stores feature information indicating the characteristics of vehicles to be noticed. Other vehicles 2 in the rear view image VB that match or are similar to this feature information are identified as vehicles to be noticed. Alternatively, the driving status export unit 22B can also analyze the rear view image VB using known methods to determine whether other vehicles 2 in the rear view image VB are vehicles to be noticed.

[0084] The position of the vehicle being observed in the rear view VB indicates its position within that rear view VB. The driving status export unit 22B determines the position of the vehicle being observed in the rear view VB.

[0085] The vehicle 1 is in a state where it cannot change lanes or turn left or right, for example, if the road R in which the vehicle 1 is traveling is a one-way single lane, and it is difficult to enter other driving paths, parking lots, or change lanes by turning right or left. The driving status derivation unit 22B determines whether the vehicle 1 is in a state where it cannot change lanes or turn left or right by combining the rear side image VS and the rear image VB with area detection using AI learned through deep learning or the like. Alternatively, for example, the driving status derivation unit 22B can also determine whether the vehicle 1 is in a state where it cannot change lanes or turn left or right by obtaining the surrounding conditions of the vehicle 1 from the driving control unit 16 via CAN or the like. Alternatively, for example, the driving status derivation unit 22B can also receive information from an external information processing device, such as the communication unit 11, indicating whether the vehicle 1 is in a state where it cannot change lanes or turn left or right by using map data, and thereby derive this information.

[0086] Next, the display control unit 22C will be described.

[0087] The display control unit 22C displays the rear view image VB captured by the rear view camera 13, which captures the rear view of the vehicle 1, on the electronic rearview mirror 18. As described above, the electronic rearview mirror 18 is an example of a display device.

[0088] In addition, if the first condition is met, the display control unit 22C will display a second display image obtained by superimposing the rear side image VS on a portion of the rear image VB on the electronic rearview mirror 18.

[0089] The first condition indicates that there is a two-wheeled vehicle 2A that is not captured in the rear image VB but is captured in the rear side image VS, wherein the rear side image VS is an image captured by the rear side imaging unit 12 that captures the rear of the left side and the rear of the right side of the vehicle 1 respectively.

[0090] The rear-side image VS captured by the rear-side camera unit 12, which captures the rear of the left side and the rear of the right side of the vehicle 1 respectively, refers to being captured in either the left rear-side image VSL captured by the left rear-side camera unit 12L or the right rear-side image VSR captured by the right rear-side camera unit 12R.

[0091] use Figure 2 Explanation will be provided. In Figure 2In the situation shown, two-wheeled vehicles 2A1 and 2A2, which are located behind and to the rear of vehicle 1, are not included in the shooting area GB of the rear camera unit 13, but are included in the shooting area GL of the left rear side camera unit 12L. Therefore, two-wheeled vehicle 2A1 is not captured in the rear side image VS captured by the rear camera unit 13, but is captured in the left rear side image VSL captured by the left rear side camera unit 12L. Therefore, Figure 2 The situation shown is the situation that satisfies the first condition. In other words, the two-wheeled vehicle 2A1 is the two-wheeled vehicle 2A that satisfies the first condition.

[0092] On the other hand, the two-wheeled vehicle 2A2 is included in the shooting area GB of the rear shooting unit 13, and is therefore captured in the rear image VB captured by the rear shooting unit 13. Therefore, the two-wheeled vehicle 2A2 is a two-wheeled vehicle 2A that does not satisfy the first condition.

[0093] If a two-wheeled vehicle 2A that meets the first condition exists, the display control unit 22C will display a second display image on the electronic rearview mirror 18 by superimposing the rear side image VS on a portion of the rear image VB.

[0094] Figure 5A and Figure 5B This is a schematic diagram of an example of the image V displayed in the electronic rearview mirror 18.

[0095] exist Figure 5A The image shows an example where only the rear view image VB is displayed on the electronic rearview mirror 18, without showing the rear side image VS. Figure 5B The image shown is an example of the state of the second display image V2 displayed on the electronic rearview mirror 18.

[0096] In the absence of a two-wheeled vehicle 2A that satisfies the first condition, such as Figure 5A As shown, the display control unit 22C only displays the rear view image VB on the electronic rearview mirror 18. On the other hand, if a two-wheeled vehicle 2A (hereinafter sometimes referred to as two-wheeled vehicle 2A1) that meets the first condition exists, the display control unit 22C switches the image V displayed on the electronic rearview mirror 18 to the second display image V2.

[0097] The second display image V2 is obtained by superimposing the rear side image VS on a portion of the rear image VB.

[0098] The display control unit 22C displays a second display image V2, obtained by superimposing a portion of the rear image VB onto the rear side image VS of the two-wheeled vehicle 2A1 that meets the first condition, on the electronic rearview mirror 18. That is, the display control unit 22C displays a second display image V2, obtained by superimposing the rear side image VS of the left rear side image VSL and the right rear side image VSR that meets the first condition, onto the rear image VB, on the electronic rearview mirror 18.

[0099] Therefore, the display control unit 22C can display the rear view image VB of the vehicle 1 behind the driver and the rear side image VS of the two-wheeled vehicle 2A1 that is not captured in the rear view image VB on the electronic rearview mirror 18 in a manner that the driver can confirm.

[0100] Preferably, the display control unit 22C displays a second display image V2 on the electronic rearview mirror 18 by superimposing the area in the rear image VB corresponding to the detection direction of the two-wheeled vehicle 2A1 relative to the detection direction of the vehicle 1 with the rear side image VS corresponding to the detection direction of the two-wheeled vehicle 2A1.

[0101] The display control unit 22C uses the analysis of the rear side image VS and the detection results of sensor 14 to determine the detection direction of the two-wheeled vehicle 2A1 that meets the first condition. Figure 2 In the state shown, the two-wheeled vehicle 2A1 that meets the first condition is located on the right rear side relative to the vehicle 1. Therefore, the display control unit 22C can display the second display image V2, which is obtained by superimposing the right rear side image VSR on the right end side of the display surface of the electronic rearview mirror 18 when it is visually confirmed by the driver of the vehicle 1, on the electronic rearview mirror 18.

[0102] Therefore, the display control unit 22C can display the rear side image VS of the two-wheeled vehicle 2A1 that was captured in the rear image VB but not captured on the detection direction side of the two-wheeled vehicle 2A1, and can provide the rear side image VS in a way that the driver can easily identify the direction in which the two-wheeled vehicle 2A1 exists.

[0103] Alternatively, preferably, the display control unit 22C displays a second display image V2 on the electronic rearview mirror 18, which is obtained by superimposing the rear side image VS on the area in the rear image VB that is outside the specified area of ​​the specified viewing angle range and corresponds to the detection direction of the detected two-wheeled vehicle 2A1 relative to the vehicle 1.

[0104] use Figure 2 Please provide an explanation.

[0105] The prescribed field of view (GS) refers to the range of vision that a driver must be able to confirm using rearview mirrors or other means specified for each vehicle type. Specifically, the prescribed field of view (GS) is the range that meets the field-of-view requirements of Rule 46 (UN-R46: United Nations Regulation 46). The field-of-view requirements of UN-R46 refer to the agreement rules established by the United Nations concerning indirect field-of-view references, which define the field of view as "a flat, horizontal portion of the road at least 20 meters wide, centered on the vertical longitudinal center plane of the vehicle, that enables the driver to see the road, and the field of view extending from 60 meters behind the driver's eye point to the horizon."

[0106] The display control unit 22C will overlay the rear side image VS of the two-wheeled vehicle 2A1 onto the area outside the designated area ES of the designated viewing angle area GS in the rear image VB of the rear image area GB of the rear imaging unit 13, which corresponds to the detection direction of the two-wheeled vehicle 2A1 detected relative to the vehicle 1. Figure 2 The second display image V2, obtained from the right rear side image (VSR) in the current situation, is displayed on the electronic rearview mirror 18.

[0107] Figure 5C and Figure 5D This is a schematic diagram of an example of the second display image V2 shown on the electronic rearview mirror 18.

[0108] like Figure 5C As shown, the display control unit 22C displays a second display image V2 on the electronic rearview mirror 18, which is obtained by superimposing the right rear side image VSR of the two-wheeled vehicle 2A1 onto the area outside the predetermined area ES of the predetermined viewing angle area GS in the rear image VB, corresponding to the detection direction of the two-wheeled vehicle 2A1 relative to the vehicle 1. Therefore, the predetermined area ES of the rear image VB is displayed in the electronic rearview mirror 18, and the right rear side image VSR containing the two-wheeled vehicle 2A1 that meets the first condition is displayed in the detection direction of the two-wheeled vehicle 2A1.

[0109] Therefore, it is possible to prevent the designated area ES in the rear view image VB from being superimposed on the rear side view image VS of the two-wheeled vehicle 2A1, thus preventing it from being displayed. In other words, the display control unit 22C can display the rear view image VB of the vehicle 1, which includes at least the designated area ES, and the rear side view image VS of the two-wheeled vehicle 2A1 that is not captured in the rear view image VB, on the electronic rearview mirror 18 in a manner that the driver can easily confirm. That is, the display control unit 22C can display the two-wheeled vehicle 2A1 present in the rear side view image VS in a manner that complies with rule 46 and is easily confirmed by the driver.

[0110] Furthermore, the display control unit 22C can position the center C' of the rear view image VB at a position offset from the center C of the display area of ​​the electronic rearview mirror 18, and display the second display image V2 obtained by superimposing the rear side image VS of the two-wheeled vehicle 2A1, which satisfies the first condition, onto the area of ​​the rear view image VB excluding the designated area ES, on the electronic rearview mirror 18. The center C' of the rear view image VB refers to the center of the width direction Y of the rear view image VB. More specifically, the center C' of the rear view image VB refers to the center position of the rear view image VB in the width direction Y2. The center C of the display area of ​​the electronic rearview mirror 18 refers to the center of the display area of ​​the electronic rearview mirror 18 in the width direction Y. More specifically, the center C of the display area of ​​the electronic rearview mirror 18 refers to the center position of the display surface of the electronic rearview mirror 18 in the width direction Y1. The width directions Y, Y1, and Y2 are directions consistent with the vehicle width direction of the vehicle 1.

[0111] The display control unit 22C displays the center C' of the rear image VB at a position offset from the center C of the display area of ​​the electronic rearview mirror 18, thereby enabling the display of the rear side image VS captured by the two-wheeled vehicle 2A1 and suppressing at least a portion of the specified area ES contained in the rear image VB from being undisplayed due to the superimposed display of the rear side image VS.

[0112] like Figure 5D As shown, the display control unit 22C can display a second display image V2 on the electronic rearview mirror 18 by cropping the area of ​​the two-wheeled vehicle 2A1 that meets the first condition from the rear side image VS and superimposing it on the area of ​​the rear image VB excluding the specified area ES.

[0113] The display control unit 22C captures the area of ​​the two-wheeled vehicle 2A1 captured in the rear side image VS and overlays it onto the rear image VB. This prevents at least a portion of the specified area ES contained in the rear image VB from being undisplayed due to the overlay display of the rear side image VS captured in the two-wheeled vehicle 2A1.

[0114] return Figure 2 Let me continue explaining.

[0115] The display control unit 22C can also display the second display image V2 on the electronic rearview mirror 18 when both the first and second conditions are met.

[0116] The second condition indicates at least one of the following: the road width W of the road R on which the vehicle 1 is traveling is greater than or equal to the first width; the speed of the vehicle 1 is less than the first speed; the distance between the vehicle 1 and other vehicles 2 following behind the vehicle 1 is less than the first distance; the total number of other vehicles 2 captured in the rear image VB is greater than or equal to the first predetermined number; and the adjacent area adjacent to the lane on which the vehicle 1 is traveling is not a roadway, and the speed of the vehicle 1 is less than the second speed.

[0117] For example, if the first condition is met and the road width W of the driving road R of the vehicle 1 is greater than or equal to the first width, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear image VB to the second display image V2.

[0118] The first width is the minimum width of the road R that allows the two-wheeled vehicle 2A1 to pass and overtake the vehicle 1 traveling on the road R. The first width can be preset based on the width of the vehicle 1. Alternatively, it can be set to be able to change appropriately based on the user's operation instructions on the UI unit 15, etc.

[0119] The display control unit 22C determines whether the road width W of the driving road R of the vehicle 1 derived by the driving condition derivation unit 22B is greater than or equal to a first width. Furthermore, if the display control unit 22C determines that the first condition is met and the road width W of the driving road R of the vehicle 1 is greater than or equal to the first width, it switches the display of the electronic rearview mirror 18 from the rear image VB to the second display image V2. Through this switching process from the rear image VB to the second display image V2, the display control unit 22C can display the second display image V2, obtained by superimposing the rear image VB with the rear side image VS of the two-wheeled vehicle 2A1 (which is not captured in the rear image VB but is captured in the rear side image VS), on the electronic rearview mirror 18 when there is a high probability that the two-wheeled vehicle 2A1 will pass through the side of the vehicle 1.

[0120] In addition, when the first condition is met and the vehicle speed of the vehicle 1 is less than the first speed, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear image VB to the second display image V2.

[0121] The first speed is the maximum speed at which the two-wheeled vehicle 2A1 can pass and overtake the vehicle 1 from the side. The first speed can be preset based on the drivable speed of the vehicle 1. Alternatively, it can be set to allow the first speed to be appropriately changed based on user operation instructions on the UI unit 15, etc.

[0122] The display control unit 22C determines whether the vehicle speed of the vehicle 1 derived by the driving status derivation unit 22B is less than a first speed. Furthermore, if the display control unit 22C determines that the first condition is met and the vehicle speed of the vehicle 1 is less than the first speed, it switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2. Through this switching process from the rear view image VB to the second display image V2, the display control unit 22C can display the second display image V2, obtained by superimposing the rear side image VS of the two-wheeled vehicle 2A1 (which is not captured in the rear view image VB but is captured in the rear side image VS), on the electronic rearview mirror 18 when there is a high probability that the two-wheeled vehicle 2A1 will pass by and overtake the vehicle 1 from the side.

[0123] In addition, when the first condition is met and the distance between the display control unit 22C and other vehicles 2 following behind the vehicle 1 is less than the first distance, the display of the electronic rearview mirror 18 is switched from the rear image VB to the second display image V2.

[0124] The first distance is the maximum distance that ensures that other vehicles 2 traveling behind vehicle 1 are largely captured in the rear view image VB, while other vehicles 2 besides those vehicles are not captured in the rear view image VB. The first distance can be preset. Alternatively, it can be set to be adjustable based on user instructions on the UI unit 15, etc.

[0125] The display control unit 22C determines whether the distance between the vehicle 1 and other vehicles 2 following behind it, as derived by the driving status derivation unit 22B, is less than a first distance. Furthermore, if the first condition is met and the distance between the vehicle 1 and other vehicles 2 following behind it is less than the first distance, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2. Through this switching process from the rear view image VB to the second display image V2, the display control unit 22C can prevent a two-wheeled vehicle 2A that might be weaving in from the side of the vehicle 1 from being obstructed by another vehicle 2 traveling closest to the vehicle 1, thus preventing the driver from not being able to see the two-wheeled vehicle 2A.

[0126] In addition, if the first condition is met and the total number of other vehicles 2 captured in the rear view image VB is more than the first predetermined number, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2.

[0127] The first predetermined number is the minimum number of vehicles 2A1 that could potentially be buried by other vehicles 2 if there are many other vehicles 2 following behind vehicle 1, thus satisfying the first condition. This first predetermined number can be preset. Alternatively, it can be set to be adjustable based on user instructions to the UI unit 15, etc.

[0128] The display control unit 22C determines whether the total number of other vehicles 2 captured in the rear view image VB, exported by the driving status export unit 22B, is greater than or equal to a first predetermined number. Furthermore, if the first condition is met and the total number of other vehicles 2 captured in the rear view image VB is greater than or equal to the first predetermined number, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2. Through this switching process from the rear view image VB to the second display image V2, the display control unit 22C can prevent a situation where at least a portion of the two-wheeled vehicles 2A traveling around the vehicle 1 are obstructed by multiple other vehicles 2 traveling behind the vehicle 1, making it impossible for the driver to see them.

[0129] In addition, if the first condition is met, the adjacent area adjacent to the driving lane of the vehicle 1 is not a roadway, and the speed of the vehicle 1 is less than the second speed, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear image VB to the second display image V2.

[0130] The second speed is the maximum speed among the speed at which vehicle 1 slows down for careful driving, the speed when turning right, and the speed when turning left. The second speed can be preset. Alternatively, it can be configured to change the second speed appropriately based on user input instructions from the UI unit 15, etc.

[0131] The display control unit 22C determines whether the adjacent area adjacent to the driving lane of the vehicle 1, derived by the driving condition derivation unit 22B, is a pedestrian walkway and whether the vehicle 1's speed is less than the second speed. Furthermore, if the first condition is met, and the adjacent area adjacent to the driving lane of the vehicle 1 is a pedestrian walkway instead of a roadway, and the vehicle 1's speed is less than the second speed, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2. Through this switching process from the rear view image VB to the second display image V2, the display control unit 22C can provide the rear view image VB when the vehicle 1 decelerates to a speed less than the second speed to enter an adjacent area that is not a roadway, and provides a visually verifiable image of a two-wheeled vehicle 2A1 that is highly likely to cut in from the side of the vehicle 1.

[0132] Thus, preferably, when both the first and second conditions are met, the display control unit 22C displays the second display image V2 on the electronic rearview mirror 18.

[0133] Furthermore, preferably, when the first and second conditions are met, the faster the relative speed of the two-wheeled vehicle 2A1 relative to the vehicle 1 is, the faster the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear image VB to the second display image V2 within a shorter time after determining that the first and second conditions are met.

[0134] The display control unit 22C switches the display of the electronic rearview mirror 18 from the rear image VB to the second display image V2 within a shorter time from the timer from when the first and second conditions are determined to be met, based on the relative speed between the vehicle 1 and the two-wheeled vehicle 2A1 that meets the first condition, derived by the driving condition derivation unit 22B.

[0135] For example, suppose the display control unit 22C switches the rear view image VB to the second display image V2 after a reference elapsed time (e.g., a few milliseconds) has elapsed since the timing when the first and second conditions are determined to be met. In this case, the faster the relative speed, the shorter the elapsed time relative to the reference elapsed time after the timing when the display control unit 22C determines that the first and second conditions are met, and the faster the electronic rearview mirror 18 can switch the display from the rear view image VB to the second display image V2.

[0136] The display control unit 22C switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2 at a timing corresponding to the relative speed. This allows the user to be alerted to the presence of the two-wheeled vehicle 2A1 meeting the first condition more quickly the relative speed increases. Conversely, the lower the relative speed, the less necessary it is to promptly alert the driver to the presence of the two-wheeled vehicle 2A1 meeting the first condition. Therefore, the display control unit 22C can provide the presence of the two-wheeled vehicle 2A1 meeting the first condition in a manner that alerts the driver at appropriate timings.

[0137] Next, the switching control from the second display image V2 to the rear image VB performed by the display control unit 22C will be explained.

[0138] If the third condition is met, the display control unit 22C switches the second display image V2 to the rear image VB and displays it on the electronic rearview mirror 18. Specifically, if the display control unit 22C determines that the third condition is met, it will... Figures 5B-5C The second display image V2, obtained by overlaying the rear side image VS of the two-wheeled vehicle 2A1 that meets the first condition onto the rear image VB shown, is switched to... Figure 5AThe image shown is only the rear view VB and is displayed on the electronic rearview mirror 18.

[0139] The third condition is at least one of the following: the distance between the vehicle 2A1 and the vehicle 2A1 captured in the rear side image VS changes from a predetermined distance to less than the predetermined distance, and the vehicle 2A1 is no longer captured in the rear side image VS; the road width W of the road R on which the vehicle 1 is traveling is less than the first width; the speed of the vehicle 1 is greater than or equal to the first speed; the distance between the vehicle 1 and other vehicles 2 following behind the vehicle 1 is greater than or equal to the first distance; the other vehicles 2 captured in the rear image VB are pre-defined vehicles to be watched; the vehicles to be watched captured in the rear image VB are located in the superimposed area of ​​the rear image VB contained in the second display image V2, which is superimposed on the rear side image VS; and the vehicle 1 cannot change lanes or turn left or right.

[0140] For example, while the electronic rearview mirror 18 is displaying the second display image V2, if the distance between the electronic rearview mirror 18 and the two-wheeled vehicle 2A1 captured in the rear side image VS changes from a predetermined distance to less than the predetermined distance and the two-wheeled vehicle 2A1 is no longer captured in the rear side image VS, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB.

[0141] The specified distance can be set to a distance that allows the determination that a two-wheeled vehicle 2A1, satisfying the first condition, has passed by and completely overtaken the vehicle 1, and is now traveling in front of the vehicle 1. The specified distance is, for example, 5 meters, but is not limited to this distance. Alternatively, the specified distance can be adjusted appropriately based on user instructions to the UI unit 15, etc.

[0142] The display control unit 22C determines whether the distance between the vehicle 2A1 and the vehicle (as determined by the driving status derivation unit 22B) has changed from a state where the distance is greater than or equal to a predetermined distance to a state where the distance is less than the predetermined distance, and whether the vehicle 2A1 is no longer captured in the rear side image VS. Furthermore, while the electronic rearview mirror 18 is displaying the second display image V2, if the distance between the vehicle 2A1 captured in the rear side image VS changes from greater than or equal to a predetermined distance to less than the predetermined distance, and the vehicle 2A1 is no longer captured in the rear side image VS, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB. Through this switching process from the second display image V2 to the rear image VB, the display control unit 22C can switch the image V displayed on the electronic rearview mirror 18 from the second display image V2 to the rear image VB when the risk posed by the vehicle 2A1 to the vehicle 1 is reduced, satisfying the first condition.

[0143] In addition, if the display control unit 22C determines that the road width W of the driving road R of the vehicle 1 is less than the first width while the electronic rearview mirror 18 is displaying the second display image V2, it switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB.

[0144] The display control unit 22C determines whether the road width W of the driving path R of the vehicle 1, derived by the driving status derivation unit 22B, is less than the first width. Furthermore, if the display control unit 22C determines that the road width W of the driving path R of the vehicle 1 is less than the first width, it switches the display on the electronic rearview mirror 18 from the second display image V2 to the rear view image VB. Through this switching process from the second display image V2 to the rear view image VB, the display control unit 22C can switch the second display image V2 to the rear view image VB and display it on the electronic rearview mirror 18 when the possibility of the two-wheeled vehicle 2A1 weaving through the side of the vehicle 1 is low.

[0145] Furthermore, while the electronic rearview mirror 18 is displaying the second display image V2, if the vehicle speed of the vehicle 1 is above the first speed, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB.

[0146] The display control unit 22C determines whether the vehicle speed of the vehicle 1 derived by the driving status derivation unit 22B is above the first speed. Furthermore, if the display control unit 22C determines that the vehicle speed of the vehicle 1 is above the first speed while the electronic rearview mirror 18 is displaying the second display image V2, it switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear view image VB. Through this switching process from the second display image V2 to the rear view image VB, the display control unit 22C can switch the second display image V2 to the rear view image VB and display it on the electronic rearview mirror 18 when the possibility of a two-wheeled vehicle 2A1, which is not captured in the rear view image VB but is captured in the rear side view image VS, cutting in from the side of the vehicle 1 is low.

[0147] Furthermore, while the electronic rearview mirror 18 is displaying the second display image V2, if the distance between the electronic rearview mirror 18 and another vehicle 2 following behind the vehicle 1 is greater than a first distance, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB.

[0148] The display control unit 22C determines whether the distance between the vehicle 1 and other vehicles 2 following behind it, as derived by the driving status derivation unit 22B, is greater than or equal to a first distance. Furthermore, if the distance between the vehicle 1 and other vehicles 2 following behind it is greater than or equal to the first distance, the display control unit 22C switches the display on the electronic rearview mirror 18 from the second display image V2 to the rear view image VB. Through this switching process from the second display image V2 to the rear view image VB, the display control unit 22C can switch the second display image V2 to the rear view image VB and display it on the electronic rearview mirror 18 when the distance between the vehicle 1 and the two-wheeled vehicle 2A1 is greater than the first distance and easily visible to the driver.

[0149] Furthermore, while the electronic rearview mirror 18 is displaying the second display image V2, if another vehicle 2 captured in the rear view image VB is a pre-defined vehicle to be watched, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear view image VB.

[0150] When the driving status export unit 22B exports emergency vehicles such as fire trucks and ambulances, or vehicles driven aggressively, into the rear view image VB, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear view image VB. Through this switching process from the second display image V2 to the rear view image VB, the display control unit 22C can provide notice of vehicles captured in the rear view image VB in a manner that allows the driver to easily visually confirm their presence.

[0151] Additionally, while the electronic rearview mirror 18 is displaying the second display image V2, if the vehicle being observed in the rear image VB is located in the superimposed area of ​​the rear image VB included in the second display image V2 and superimposed with the rear side image VS, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB.

[0152] The display control unit 22C determines whether the position of the vehicle being observed in the rear view image VB, derived by the driving status deriving unit 22B, is within the superimposed area of ​​the rear side view image VS in the currently displayed second display image V2. Furthermore, if the vehicle being observed in the rear view image VB exists within the superimposed area of ​​the rear view image VB included in the second display image V2, the display control unit 22C switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear view image VB. Through this switching process from the second display image V2 to the rear view image VB, the display control unit 22C can provide information about the vehicle being observed in the rear view image VB in a manner that allows the driver to easily visually confirm its presence.

[0153] In addition, if the display control unit 22C determines that the vehicle 1 cannot change lanes or turn left or right while the electronic rearview mirror 18 is displaying the second display image V2, it switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear image VB.

[0154] The display control unit 22C determines whether the vehicle 1 is unable to change lanes or turn left or right by acquiring information derived from the driving status derivation unit 22B indicating whether the vehicle 1 is in a state where lane changes or left / right turns are prohibited. Furthermore, if the display control unit 22C determines that the vehicle 1 is unable to change lanes or turn left or right while the electronic rearview mirror 18 is displaying the second display image V2, it switches the display of the electronic rearview mirror 18 from the second display image V2 to the rear view image VB. Through this switching process from the second display image V2 to the rear view image VB, the display control unit 22C can switch the display of the second display image V2 to the rear view image VB when the possibility of the two-wheeled vehicle 2A1 weaving through the side of the vehicle 1 is low.

[0155] Next, an example of the information processing flow performed by the information processing apparatus 10 of this embodiment will be described.

[0156] Figure 6A and Figure 6B This is a flowchart illustrating an example of the information processing flow performed by the information processing apparatus 10 in this embodiment.

[0157] The display control unit 22C displays the rear view image VB captured by the rear view camera 13 (which captures the rear view of the vehicle 1) on the electronic rearview mirror 18 (step S100).

[0158] The display control unit 22C determines whether a two-wheeled vehicle 2A1 exists that is not captured in the rear image VB but is captured in the rear side image VS (step S102). When a negative judgment is made in step S102 (step S102: "No"), the process proceeds to step S130 described later. When a positive judgment is made in step S102 (step S102: "Yes"), the process proceeds to step S104.

[0159] In step S104, the display control unit 22C determines whether the road width W of the driving road R of the vehicle 1 is greater than or equal to the first width (step S104). When a positive determination is made in step S104 (step S104: "Yes"), the process proceeds to step S114. When a negative determination is made in step S104 (step S104: "No"), the process proceeds to step S106.

[0160] In step S106, the display control unit 22C determines whether the speed of the vehicle 1 is less than the first speed (step S106). When a positive judgment is made in step S106 (step S106: "Yes"), the process proceeds to step S114. When a negative judgment is made in step S106 (step S106: "No"), the process proceeds to step S108.

[0161] In step S108, the display control unit 22C determines whether the distance between itself and other vehicles 2 following behind vehicle 1 is less than a first distance (step S108). If a positive determination is made in step S108 (step S108: "Yes"), the process proceeds to step S114. If a negative determination is made in step S108 (step S108: "No"), the process proceeds to step S110.

[0162] In step S110, the display control unit 22C determines whether the total number of other vehicles 2 captured in the rear image VB is greater than or equal to a first predetermined number (step S110). If an affirmative determination is made in step S110 (step S110: "Yes"), the process proceeds to step S114. If a negative determination is made in step S110 (step S110: "No"), the process proceeds to step S112.

[0163] In step S112, the display control unit 22C determines whether the adjacent area adjacent to the driving lane of the vehicle 1 is not a roadway and whether the speed of the vehicle 1 is less than the second speed (step S112). When a positive determination is made in step S112 (step S112: "Yes"), the process proceeds to step S114. When a negative determination is made in step S112 (step S112: "No"), the process proceeds to step S130.

[0164] In step S114, the display control unit 22C switches the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2 (step S114). Preferably, the faster the relative speed of the two-wheeled vehicle 2A1 relative to the vehicle 1, satisfying the first condition, the shorter the time it takes for the display control unit 22C to switch the display of the electronic rearview mirror 18 from the rear view image VB to the second display image V2 from the time it determines that the first and second conditions are met.

[0165] Then, proceed to step S116.

[0166] In step S116, the display control unit 22C determines whether the two-wheeled vehicle 2A1, which met the first condition as determined in step S102, has completely passed by the side of the vehicle 1 (step S116). The display control unit 22C performs the determination in step S116 by determining whether the distance between the two-wheeled vehicle 2A1 and the vehicle captured in the rear side image VS changes from a predetermined distance to less than the predetermined distance and whether the two-wheeled vehicle 2A1 is no longer captured in the rear side image VS. When a positive determination is made in step S116 (step S116: "Yes"), the process proceeds to step S130. When a negative determination is made in step S116 (step S116: "No"), the process proceeds to step S118.

[0167] In step S118, the display control unit 22C determines whether the road width W of the driving road R of the vehicle 1 is less than the first width (step S118). When a positive judgment is made in step S118 (step S118: "Yes"), the process proceeds to step S130. When a negative judgment is made in step S118 (step S118: "No"), the process proceeds to step S120.

[0168] In step S120, the display control unit 22C determines whether the speed of the vehicle 1 is above the first speed (step S120). When a positive determination is made in step S120 (step S120: "Yes"), the process proceeds to step S130. When a negative determination is made in step S120 (step S120: "No"), the process proceeds to step S122.

[0169] In step S122, the display control unit 22C determines whether the distance between itself and other vehicles 2 following behind vehicle 1 is greater than a first distance (step S122). If an affirmative determination is made in step S122 (step S122: "Yes"), the process proceeds to step S130. If a negative determination is made in step S122 (step S122: "No"), the process proceeds to step S124.

[0170] In step S124, the display control unit 22C determines whether the other vehicle 2 in the rear image VB captured is a vehicle to be noticed (step S124). When an affirmative determination is made in step S124 (step S124: "Yes"), the process proceeds to step S126.

[0171] In step S126, the display control unit 22C determines whether the vehicle in the captured rear image VB exists in the superimposed area of ​​the rear image VB included in the second display image V2, which is superimposed on the rear side image VS (step S126). When a positive judgment is made in step S126 (step S126: "Yes"), the process proceeds to step S130. When a negative judgment is made in step S126 (step S126: "No"), the process proceeds to step S128.

[0172] In step S128, the display control unit 22C determines whether the vehicle 1 is in a state where it cannot change lanes or turn left or right (step S128). If a positive judgment is made in step S128 (step S128: "Yes"), the process proceeds to step S130. If a negative judgment is made in step S128 (step S128: "No"), the process returns to step S116.

[0173] In step S130, the display control unit 22C determines whether the vehicle 1 has finished driving (step S130). For example, the display control unit 22C performs the determination in step S130 by determining whether it has received a signal indicating that the engine of the vehicle 1 has been stopped and the power supply to each part of the vehicle 1 has been cut off. When a negative determination is made in step S130 (step S130: "No"), the process returns to step S100. When a positive determination is made in step S130 (step S130: "Yes"), the routine ends.

[0174] As described above, the information processing device 10 of this embodiment includes a display control unit 22C. The display control unit 22C displays the rear view image VB captured by the rear view camera 13 that captures the rear of the vehicle 1 on the electronic rearview mirror 18 (display device) installed in the vehicle 1. If a first condition is met, that there is a two-wheeled vehicle 2A1 that is not captured in the rear view image VB but is captured in the rear side view image VS, the display control unit 22C displays a second display image V2 obtained by superimposing the rear side view image VS on a portion of the rear view image VB on the electronic rearview mirror 18 (display device), wherein the rear side view image VS is an image captured by the rear side view camera 12 that captures the rear of the left side and the rear of the right side of the vehicle 1, respectively.

[0175] In the prior art, even when the driver needs to confirm the image captured by the rearview camera, the display of the electronic rearview mirror 18 is sometimes switched to show only the image from the side-view camera. Furthermore, in the prior art, even when the driver needs to confirm the image captured by the side-view camera, sometimes only the image from the rearview camera is displayed on the electronic rearview mirror 18. Therefore, in the prior art, the visibility of the vehicle's surroundings is sometimes reduced.

[0176] On the other hand, in the information processing device 10 of this embodiment, when the first condition of a two-wheeled vehicle 2A1 that is not captured in the rear image VB but is captured in the rear side image VS is met, the display control unit 22C displays a second display image V2 obtained by superimposing the rear side image VS on a part of the rear image VB on the electronic rearview mirror 18 (display device). The rear side image VS is an image captured by the rear side shooting unit 12 that captures the rear of the left side and the rear of the right side of the vehicle 1 respectively.

[0177] Therefore, in the information processing device 10 of this embodiment, the rear image VB can be continuously displayed on the electronic rearview mirror 18, and the image V of the two-wheeled vehicle 2A1 located on the rear side of the vehicle 1 can be superimposed on the rear image VB at appropriate time intervals.

[0178] Therefore, the information processing device 10 of this embodiment can improve the visibility of the surrounding conditions of the vehicle 1 for the driver.

[0179] Therefore, the information processing device 10 of this embodiment can improve the visibility of the surrounding conditions of the vehicle 1.

[0180] Furthermore, in this embodiment, the information processing device 10 is described as an example of being mounted on the vehicle 1. However, the information processing device 10 may also be an external structure mounted on the vehicle 1. The information processing device 10 can be communicatively connected to various electronic devices provided on the vehicle 1, such as the rear side camera unit 12, rear camera unit 13, sensor 14, UI unit 15, driving control unit 16, electronic rearview mirror 18, and storage unit 19. Therefore, the information processing device 10 may also be an externally mounted information processing device on the vehicle 1. In this case, the information processing device equipped with the information processing device 10 and the aforementioned electronic devices can be configured to communicate via a network or the like. In addition, in this case, the vehicle that is communicatively connected to the information processing device 10 is equivalent to the vehicle 1.

[0181] Furthermore, at least some of the functions of the display control unit 22C in the information processing device 10 can also be executed by the SoC (System on a Chip) provided in the information processing device 10. For example, in the information processing device 10, the rear image VB captured by the rear camera unit 13 can be displayed on the electronic rearview mirror 18. If it is determined that the first condition mentioned above is met, the SoC generates a second display image V2 and displays it on the electronic rearview mirror 18. In addition, the SoC can also perform both the display of the rear image VB on the electronic rearview mirror 18 and the generation and display of the second display image V2 on the electronic rearview mirror 18.

[0182] In addition, this technology can also adopt the following structure. (1)

[0184] An information processing apparatus includes a display control unit, which performs the following processing:

[0185] The rear view captured by the rear camera unit that captures images of the rear of the vehicle is displayed on a display device installed in the vehicle; and

[0186] If a first condition is met, that a two-wheeled vehicle is not captured in the rear image but is captured in the rear side image, a second display image obtained by superimposing the rear side image on a portion of the rear image is displayed on the display device, wherein the rear side image is an image captured by a rear side camera that captures the rear of the left side and the rear of the right side of the vehicle respectively. (2)

[0188] According to the information processing apparatus described in (1), wherein,

[0189] If both the first and second conditions are met, the display control unit displays the second display image on the display device.

[0190] The second condition is at least one of the following: the road width of the vehicle's travel path is greater than or equal to a first width; the vehicle's speed is less than a first speed; the distance between the vehicle and other vehicles following the vehicle is less than a first distance; the total number of other vehicles captured in the rear image is greater than or equal to a first predetermined number; and the adjacent area adjacent to the vehicle's travel lane is not a roadway, and the vehicle's speed is less than a second speed. (3)

[0192] According to the information processing device described in (2), wherein,

[0193] If the first and second conditions are met, the faster the relative speed of the two-wheeled vehicle to the main vehicle, the shorter the time from when the first and second conditions are determined to be met, the faster the display control unit switches the rear image to the second display image and displays it on the display device. (4)

[0195] The information processing apparatus according to any one of (1) to (3), wherein,

[0196] If the third condition is met, the display control unit switches the second display image to the rear image and displays it on the display device, wherein the third condition is at least one of the following: the distance between the vehicle and the two-wheeled vehicle captured in the rear side image changes from a predetermined distance or more to less than the predetermined distance, and the two-wheeled vehicle is no longer captured in the rear side image; the road width of the vehicle's driving road is less than the first width; the vehicle speed is greater than or equal to the first speed; the distance between the vehicle and other vehicles following the vehicle is greater than or equal to the first distance; the other vehicles captured in the rear image are pre-defined vehicles to be watched; the vehicles to be watched captured in the rear image exist in the superimposed area of ​​the rear image included in the second display image, which is superimposed by the rear side image; and the vehicle cannot change lanes or turn left or right. (5)

[0198] The information processing apparatus according to any one of (1) to (4), wherein,

[0199] The display control unit displays the second display image, obtained by superimposing the rear side image of the two-wheeled vehicle onto a portion of the rear image, on the display device. (6)

[0201] The information processing apparatus according to any one of (1) to (5), wherein,

[0202] The display control unit displays the second display image obtained by superimposing the area in the rear image corresponding to the detection direction of the two-wheeled vehicle relative to the vehicle with the rear side image corresponding to the direction of the detection of the two-wheeled vehicle on the display device. (7)

[0204] The information processing apparatus according to any one of (1) to (6), wherein,

[0205] The display control unit displays the second display image obtained by superimposing the rear side image on the display device onto the area in the rear image that corresponds to the detection direction of the two-wheeled vehicle relative to the vehicle, excluding a specified area within a specified viewing angle range. (8)

[0207] According to the information processing apparatus described in (7), wherein,

[0208] The display control unit positions the center of the rear image at a position offset from the center of the display area of ​​the display device, and displays the second display image obtained by superimposing the rear side image on the area of ​​the rear image other than the designated area on the display device. (9)

[0210] According to the information processing apparatus described in (7) or (8), wherein,

[0211] The display control unit displays a second display image obtained by cropping the area of ​​the two-wheeled vehicle captured in the rear side image and superimposing it onto the area of ​​the rear image other than the designated area on the display device. (10)

[0213] An information processing method is an information processing method executed by an information processing device, wherein,

[0214] The rear view captured by the rear camera unit that photographs the rear of the vehicle is displayed on the display device installed in the vehicle.

[0215] If a first condition is met, that a two-wheeled vehicle is not captured in the rear image but is captured in the rear side image, a second display image obtained by superimposing the rear side image on a portion of the rear image is displayed on the display device, wherein the rear side image is an image captured by a rear side camera that captures the rear of the left side and the rear of the right side of the vehicle respectively. (11)

[0217] An information processing program is an information processing program for causing a computer to execute, the information processing program comprising the following steps:

[0218] The rear view captured by the rear camera unit that captures images of the rear of the vehicle is displayed on a display device installed in the vehicle; and

[0219] If a first condition is met, that a two-wheeled vehicle is not captured in the rear image but is captured in the rear side image, a second display image obtained by superimposing the rear side image on a portion of the rear image is displayed on the display device, wherein the rear side image is an image captured by a rear side camera that captures the rear of the left side and the rear of the right side of the vehicle respectively.

[0220] Furthermore, while embodiments have been described above, they are presented as examples and are not intended to limit the scope of the invention. The novel embodiments described above can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments described above are included within the scope or spirit of the invention, and are included within the scope of the invention as described in the claims and its equivalents.

Claims

1. An information processing apparatus comprising a display control unit, wherein the display control unit performs the following processing: The rear view captured by the rear camera unit that captures images of the rear of the vehicle is displayed on a display device installed in the vehicle; and If the first condition is met—that a two-wheeled vehicle exists that is not captured in the rear image but is captured in the rear-side image—a second display image, obtained by superimposing the rear-side image onto a portion of the rear image, is displayed on the display device. The rear-side images are images captured by rear-side cameras that capture images from the rear of the left side and the rear of the right side of the vehicle, respectively.

2. The information processing apparatus according to claim 1, wherein, If both the first and second conditions are met, the display control unit displays the second display image on the display device. The second condition is at least one of the following: the road width of the vehicle's travel path is greater than or equal to a first width; the vehicle's speed is less than a first speed; the distance between the vehicle and other vehicles following the vehicle is less than a first distance; the total number of other vehicles captured in the rear image is greater than or equal to a first predetermined number; and the adjacent area adjacent to the vehicle's travel lane is not a roadway, and the vehicle's speed is less than a second speed.

3. The information processing apparatus according to claim 2, wherein, If the first and second conditions are met, the faster the relative speed of the two-wheeled vehicle to the main vehicle, the shorter the time from when the first and second conditions are determined to be met, the faster the display control unit switches the rear image to the second display image and displays it on the display device.

4. The information processing apparatus according to claim 2, wherein, If the third condition is met, the display control unit switches the second display image to the rear image and displays it on the display device, wherein the third condition is at least one of the following: the distance between the vehicle and the two-wheeled vehicle captured in the rear side image changes from a predetermined distance or more to less than the predetermined distance, and the two-wheeled vehicle is no longer captured in the rear side image; the road width of the vehicle's driving road is less than the first width; the vehicle speed is greater than or equal to the first speed; the distance between the vehicle and other vehicles following the vehicle is greater than or equal to the first distance; the other vehicles captured in the rear image are pre-defined vehicles to be watched; the vehicles to be watched captured in the rear image exist in the superimposed area of ​​the rear image included in the second display image, which is superimposed by the rear side image; and the vehicle cannot change lanes or turn left or right.

5. The information processing apparatus according to claim 1, wherein, The display control unit displays the second display image, obtained by superimposing the rear side image of the two-wheeled vehicle onto a portion of the rear image, on the display device.

6. The information processing apparatus according to claim 1, wherein, The display control unit displays the second display image obtained by superimposing the area in the rear image corresponding to the detection direction of the two-wheeled vehicle relative to the vehicle with the rear side image corresponding to the direction of the detection of the two-wheeled vehicle on the display device.

7. The information processing apparatus according to claim 1, wherein, The display control unit displays the second display image obtained by superimposing the rear side image on the display device onto the area in the rear image that corresponds to the detection direction of the two-wheeled vehicle relative to the vehicle, excluding a specified area within a specified viewing angle range.

8. The information processing apparatus according to claim 7, wherein, The display control unit positions the center of the rear image at a position offset from the center of the display area of ​​the display device, and displays the second display image obtained by superimposing the rear side image on the area of ​​the rear image other than the designated area on the display device.

9. The information processing apparatus according to claim 7, wherein, The display control unit displays a second display image obtained by cropping the area of ​​the two-wheeled vehicle captured in the rear side image and superimposing it onto the area of ​​the rear image other than the designated area on the display device.

10. The information processing apparatus according to claim 1, wherein, The display control unit displays the rear image and the second display image as images for use in an electronic rearview mirror.

11. The information processing apparatus according to claim 4, wherein, The vehicles to be monitored include emergency vehicles and vehicles driven aggressively.

12. The information processing apparatus according to claim 4, wherein, It also has a situation guidance unit that, when the road in which the vehicle is traveling is a one-way single lane, guides the vehicle to indicate that it cannot change lanes or turn left or right.

13. The information processing apparatus according to claim 1, wherein, The display control unit displays the rear-side image of the vehicle's left side rear to the left end of the second display image, and displays the rear-side image of the vehicle's right side rear to the right end of the second display image.

14. An information processing method, which is an information processing method executed by an information processing device, wherein, The rear view captured by the rear camera unit that photographs the rear of the vehicle is displayed on the display device installed in the vehicle. If a first condition is met, that a two-wheeled vehicle is not captured in the rear image but is captured in the rear side image, a second display image obtained by superimposing the rear side image on a portion of the rear image is displayed on the display device, wherein the rear side image is an image captured by a rear side camera that captures the rear of the left side and the rear of the right side of the vehicle respectively.

15. A computer program product comprising an information processing program for causing a computer to execute, the information processing program comprising the following steps: The rear view captured by the rear camera unit that photographs the rear of the vehicle is displayed on the display device installed in the vehicle. as well as If a first condition is met, that a two-wheeled vehicle is not captured in the rear image but is captured in the rear side image, a second display image obtained by superimposing the rear side image on a portion of the rear image is displayed on the display device, wherein the rear side image is an image captured by a rear side camera that captures the rear of the left side and the rear of the right side of the vehicle respectively.

Images