IMAGE PROCESSING APPARATUS, DISPLAY SYSTEM, IMAGE PROCESSING METHOD, AND RECORDING MEDIUM
The image processing device addresses car sickness by optimizing screen transitions and orientations based on vehicle travel direction, enhancing passenger comfort through reduced motion sickness during vehicle travel.
Patent Information
- Application Number
- JP2022503737
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-02-28
- Filing Date
- 2021-02-26
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-02-26
AI Technical Summary
Conventional technologies for in-vehicle screen display may cause passenger discomfort, particularly leading to car sickness due to screen operations during vehicle travel.
An image processing device that switches screen displays in directions parallel or perpendicular to the vehicle's travel direction based on the frequency of screen transitions, reducing screen size and brightness during transitions to minimize motion sickness.
The solution effectively reduces the likelihood of passenger car sickness by optimizing screen transitions and display orientations, ensuring a more comfortable viewing experience.
Smart Images

Figure 0007586165000001 
Figure 0007586165000002 
Figure 0007586165000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an image processing device, a display system, an image processing method, and a recording medium. [Background technology]
[0002] There is known a technology for displaying content on a screen such as an organic electroluminescence (EL) sheet installed on the ceiling inside a vehicle. For example, there is known a technology for displaying a sky map as seen from a current location on a screen. There is also known a technology for displaying a single road showing a route to a destination and a position on the single road on a screen. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2002-328624 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, the above-mentioned conventional technology has a problem in that it may not be possible to provide a screen display that is comfortable for passengers. For example, it is known that operating a screen while in a car can easily cause car sickness.
[0005] Therefore, the present disclosure proposes an image processing device, a display system, an image processing method, and a recording medium that are capable of providing a screen display that is comfortable for passengers. [Means for solving the problem]
[0006] In order to solve the above problems, an image processing device of one embodiment according to the present disclosure includes a creation unit that creates an application screen that displays an image belonging to any of a plurality of modes on a screen provided in an automobile, and a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screen before and after the switch in a first direction parallel to or perpendicular to the traveling direction of the automobile, and, when an operation is performed to switch a displayed screen to a screen of a different mode, moves the screen before and after the switch in a second direction perpendicular to the first direction. [Brief description of the drawings]
[0007] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing device according to a first embodiment. [Diagram 2] FIG. 1 shows an example of a screen. [Diagram 3] FIG. 2 shows an example of a screen. [Figure 4] FIG. 2 is a diagram illustrating modes and panels. [Diagram 5] FIG. [Figure 6] FIG. 1 explains the screen transition animation. [Figure 7] FIG. 2 explains the screen transition animation. [Figure 8] This is a diagram (3) explaining the screen transition animation. [Figure 9] FIG. 13 is a diagram illustrating an animation of a screen transition between modes. [Figure 10] FIG. 13 is a diagram illustrating an animation of a screen transition within a mode. [Figure 11] 4 is a flowchart showing a process flow of the image processing device according to the first embodiment. [Figure 12] FIG. 13 is a diagram showing an example of a screen displaying a route. [Figure 13] FIG. 13 is a diagram showing an example of a screen displaying icons of destinations and passengers. [Figure 14]13 is a flowchart showing the flow of a process for adding a destination. [Figure 15] FIG. 13 is a diagram showing an example of a screen displaying a playlist. [Figure 16] FIG. 13 is a diagram showing an example of a screen on which a video to be played is displayed. [Figure 17] FIG. 1 is a diagram illustrating movement of a screen between screens. [Figure 18] FIG. 13 is a diagram showing an example of a screen after movement. [Figure 19] 1 is a block diagram showing a schematic configuration example of an automobile control system; [Figure 20] FIG. 1 is a diagram showing an example of the installation position of the imaging unit. [Figure 21] FIG. 2 shows an example of the installation position of the imaging unit. [Figure 22] FIG. 1 illustrates an example of the configuration of a network system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.
[0009] The explanation will be given in the following order: 1. First embodiment 2. Application examples for mobile objects
[0010] [1. First embodiment] Fig. 1 is a diagram showing an example of the configuration of an image processing device according to the first embodiment. The image processing device 10 shown in Fig. 1 is a device for performing processing related to displaying content on a screen provided in an automobile equipped with an automatic driving function.
[0011] The input unit 11 is an input device that receives inputs of the driving conditions of the vehicle, the conditions of the passengers, and operations by the passengers. The input unit 11 includes various sensors such as an image sensor, a depth sensor, and a touch sensor. The image sensor is a sensor that obtains two-dimensional images, such as a visible light camera and an infrared camera. The depth sensor is a sensor that obtains three-dimensional information including depth, such as a stereo camera, or a sensor that can implement a time of flight method, a structured light method, or the like. The input unit 11 receives operations by the passengers, such as operations by a touch display, operations by voice, and gesture operations using a skeletal field.
[0012] The communication unit 12 is an interface for performing data communication with other devices, and is realized by, for example, a network interface card (NIC).
[0013] The information processing unit 13 executes various processes related to content display. The information processing unit 13 is realized by, for example, a computer equipped with a CPU (Central Processing Unit). The information processing unit 13 performs a process for displaying an image included in the content based on information received from the input unit 11. The information processing unit 13 performs drawing control of multi-content such as a window that displays an application, and event delivery such as a touch for each content. The information processing unit 13 performs a process equivalent to a control layer of a general OS.
[0014] The image display device 40 is a device that displays an image included in the content. The image display device 40 may be a projector and a projector screen. The image display device 40 may also be a display such as a liquid crystal display. Hereinafter, the surface on which an image is actually displayed, such as a projector screen or a liquid crystal display, will simply be referred to as a screen.
[0015] FIG. 2 is a diagram (1) showing an example of a screen. As shown in FIG. 2, the image display device 40 may be a projector 20 and a projector screen 30. As shown in FIG. 2, the projector 20 projects an image onto each screen from the rear of the car. The projector 20 may be capable of changing the direction and height at which the image is projected in accordance with the shape of the screen. The projector 20 may be fixed behind the rear seats, or may be attached to the ceiling behind the rear seats. The projector 20 may also be provided in the headrest of the driver's seat or passenger seat.
[0016] Fig. 3 is a diagram showing an example of the screen (2). As shown in Fig. 3, the image display device 40 may be a liquid crystal display 31 provided on the ceiling of the car.
[0017] The audio output device 50 is a device that outputs audio included in the content. For example, the audio output device 50 is a speaker.
[0018] The display output control unit 14 creates images to be displayed and audio to be output. The display output control unit 14 is realized by, for example, a computer equipped with a CPU. As shown in FIG. 1, the display output control unit 14 has a content acquisition unit 141, an image and audio creation unit 142, and an input information storage unit 143.
[0019] The content acquisition unit 141 acquires content. The content acquisition unit 141 may acquire content from a predetermined storage device, or may acquire content from an external device or another vehicle via a network.
[0020] The image / audio creating unit 142 creates an application screen that displays an image belonging to one of a plurality of modes on a screen provided in the vehicle. When an operation is performed on the application to switch a displayed screen to a screen of the same mode, the image / audio creating unit 142 moves the screen before and after the switching in a first direction parallel or perpendicular to the traveling direction of the vehicle, and when an operation is performed to switch a displayed screen to a screen of a different mode, the image / audio creating unit 142 moves the screen before and after the switching in a second direction perpendicular to the first direction.
[0021] The input information storage unit 143 is a storage device that stores information input by passengers. For example, the input information storage unit 143 stores information on a playlist, which will be described later.
[0022] A screen belonging to each mode is called a panel. The modes and panels will be explained with reference to FIG. 4. FIG. 4 is a diagram for explaining the modes and panels. As shown in FIG. 4, for example, the application has an open car mode and a theater mode.
[0023] In the open-top car mode, an image of the sky above the car is displayed as the background, and the images of each content are displayed on top of the background. For example, the exterior scenery panel in the open-top car mode displays only an image of the sky above the car. Also, for example, the car navigation panel in the open-top car mode displays an image of the car navigation system on top of an image of the sky above the car.
[0024] In theater mode, images of content that is primarily for viewing are displayed. For example, a Hawaii panel in theater mode displays Hawaiian scenery. Or, for example, a fireworks panel in theater mode displays images of fireworks. In addition, a panel in theater mode may display movies, television programs, images taken in other cars, etc.
[0025] The traveling direction will be explained using Figure 5. Figure 5 is a diagram explaining the traveling direction. The traveling direction is the direction in which the car moves forward. However, if the screen is placed along the windshield, the direction indicated by the arrow in Figure 5 is called the traveling direction.
[0026] FIG. 6 is a diagram (1) explaining the screen transition animation. The screen in FIG. 6 is assumed to be displayed on a screen on the ceiling, for example. In this case, the top of the screen in FIG. 6 is located behind the bottom of the vehicle (the bottom of the screen is located in front of the top of the vehicle). In this case, the screen moving in a direction parallel to the driving direction means that the screen scrolls and moves in the direction of the arrow in FIG. 6.
[0027] Here, when an operation is performed on the application to switch the currently displayed screen to a screen of the same mode, the image / audio creation unit 142 moves the screen before and after the switch in a first direction perpendicular to the direction of travel of the vehicle, and when an operation is performed on the application to switch the currently displayed screen to a screen of a different mode, the image / audio creation unit 142 moves the screen before and after the switch in a second direction parallel to the direction of travel of the vehicle.
[0028] Fig. 7 is a diagram (2) explaining the screen transition animation. The direction indicated by the arrow in Fig. 7 is perpendicular to the direction in which the car is traveling. Fig. 8 is a diagram (3) explaining the screen transition animation. The direction indicated by the arrow in Fig. 8 is parallel to the direction in which the car is traveling.
[0029] Here, when switching between modes, the passenger switches screens to find the mode they want to view, so the number of screen transitions is likely to be small. On the other hand, when switching between screens within a mode, i.e., switching between panels of the same mode, a large number of screen transitions may occur, like zapping through television channels.
[0030] In addition, when the screen is moved in a direction perpendicular to the direction of travel, for example, the model in which the bottom of the screen is the ground and the top is the sky can be maintained, so it is thought that the passenger is less likely to get sick even if the screen is frequently switched. On the other hand, when the screen is moved in a direction parallel to the direction of travel, it is thought that the passenger is more likely to get sick.
[0031] Therefore, in this embodiment, the image processing device 10 moves the screen in a direction parallel to the driving direction when the screen transition is between modes where the screen switching frequency is considered to be low. Conversely, the image processing device 10 moves the screen in a direction perpendicular to the driving direction when the screen transition is within a mode where the screen switching frequency is considered to be high. This makes it possible to prevent the passenger from getting car sick.
[0032] FIG. 9 is a diagram illustrating an animation of the screen transition between modes. First, the passenger points to a black dot at the bottom of the panel with a gesture (101v). Then, the dot spreads horizontally and the meter fills up (102v). When the meter is fully filled, it is determined that the selection has been made (103v), the panel is reduced (104v), and each panel moves (scrolls) from bottom to top, as if the switched panel pushes the previous panel upward (105v). Then, panel switching between modes is completed (106v). The black dot used to input panel switching may be provided at the top of the panel. In that case, when the passenger points to a black dot at the top of the panel, the panel moves (scrolls) from top to bottom.
[0033] FIG. 10 is a diagram explaining the animation of screen transitions within a mode. First, the passenger points to the black dot on the left edge of the panel with a gesture (101p). Then, the dot spreads vertically and the meter fills up (102p). When the meter is full (103p), the panel shrinks (104p), and each panel moves (scrolls) from left to right, as if the switched panel is pushing the previous panel to the right (105p). Then, the switching of panels within the mode is completed (106p). Note that if the passenger points to the black dot on the right side of the panel with a gesture, the panel moves (scrolls) from right to left.
[0034] In 104v in Fig. 9 and 104p in Fig. 10, the image / audio generating unit 142 reduces and moves the screen before and after the switching. This reduces the overall display amount on the screen, and the passenger feels as if the field of vision has been widened, making the passenger less likely to get sick. In addition, the image / audio generating unit 142 reduces the brightness of the screen before and after the switching and moves it, making the passenger less likely to get sick.
[0035] Fig. 11 is a flowchart showing the flow of processing in the image processing device according to the first embodiment. As shown in Fig. 11, the image processing device 10 waits while no operation for switching panels is performed (step S101, No).
[0036] When an operation to switch the panels is performed (step S101, Yes), the image processing device 10 reduces the display of the panels (step S102).Then, the image processing device 10 decreases the brightness of the display of the panels (step S103).
[0037] Here, the image processing device 10 judges whether the panel switching is between modes (step S104). If the panel switching is between modes (step S104, Yes), the image processing device 10 scrolls the panels before and after the switching vertically (in a direction parallel to the running direction) (step S106). If the panel switching is not between modes (step S104, No), the image processing device 10 scrolls the panels before and after the switching horizontally (in a direction perpendicular to the running direction) (step S105).
[0038] The image processing method for each panel will now be described. The image and audio creation unit 142 creates a screen that displays the route from the current location to the destination, which is displayed on a screen attached to the ceiling of the car, with the destination displayed at the bottom and the current location displayed at the top.
[0039] In a typical car navigation system, the destination is displayed at the top of the screen and the current location is displayed at the bottom. On the other hand, in this embodiment, when the image processing device 10 displays the route on the ceiling screen, the image processing device 10 displays the current location at the top of the screen and the destination at the bottom, so that the driving direction and the traveling direction of the route are aligned, as shown in FIG.
[0040] FIG. 12 is a diagram showing an example of a screen displaying a route. As shown in FIG. 12, since the screen on the ceiling is large, the screen displayed there occupies most of the passenger's field of vision. Therefore, if the GUI on the screen moves significantly, the passenger is likely to get motion sickness. To avoid this, the image processing device 10 fixes the current position and the destination, and simplifies the route and displays it as a straight line. Furthermore, the image processing device 10 can recognize the current driving position at intervals by moving spots such as tourist spots and service areas between the current position and the destination. In the example of FIG. 12, the image processing device 10 erases the corresponding icon from the route after passing △△ Park.
[0041] A screen displayed on the ceiling screen can be operated by multiple passengers. For example, multiple passengers can each add a destination to the car navigation panel at their own discretion. In such a case, as shown in FIG. 13, the image / audio creation unit 142 creates a screen that displays the route from the current location to the destination, an icon indicating the destination category, and an icon indicating the passenger who set the destination. For each destination added to the car navigation panel, the image processing device 10 displays an icon of the passenger who added the destination. FIG. 13 is a diagram showing an example of a screen that displays icons of destinations and passengers.
[0042] The flow of the process of adding a destination by the image processing device 10 will be described. Fig. 14 is a flowchart showing the flow of the process of adding a destination. As shown in Fig. 14, first, the image processing device 10 waits until an operation to add a destination is performed (step S201, No).
[0043] When an operation to add a destination is performed (step S201, Yes), the image processing device 10 identifies the passenger who performed the operation (step S202). Then, the image processing device 10 identifies the passenger who performed the operation (step S202). Furthermore, the image processing device 10 determines the category of the added destination (step S203). Then, the image processing device 10 displays an icon of the destination next to the icon of the passenger who performed the operation.
[0044] Destinations are broadly classified into detours and final destinations. The detour category includes toilet breaks, meal breaks, etc. Destinations in the toilet break category include, for example, service areas and parking areas. Also, the meal break category includes, for example, restaurants and convenience stores.
[0045] Categories classified as final destinations include shopping, public facilities, and tourist attractions. Destinations in the shopping category include shopping malls and supermarkets. Destinations in the public facilities category include libraries, city halls, schools, train stations, roadside stations, and bus terminals. Destinations in the tourist attractions category include amusement parks, art museums, and zoos.
[0046] For example, if a passenger adds a convenience store as a destination, the passenger's icon and a meal break category icon are added to the screen. This allows the passenger to communicate to the driver, for example, that a child in the back seat needs a toilet break or a meal break. Also, for example, if multiple passengers are using a single car for ride sharing, each passenger can easily communicate their desired destination to the other passengers.
[0047] Furthermore, the image and sound creating unit 142 creates sound for playing music that is previously associated with the current location. Fig. 15 is a diagram showing an example of a screen displaying a playlist. As shown in Fig. 15, positions previously associated with a playlist may be shown on the route. When the car approaches a position associated with the playlist, the image processing device 10 outputs the created music.
[0048] Here, the automatic driving control device 60 is a device that controls the speed during automatic driving, etc. The automatic driving control device 60 is, for example, an ECU (Engine Control Unit). The route control device 70 is a device that determines the route during automatic driving. The automatic driving control device 60 and the route control device 70 can change the route to the destination and the traveling speed of the automobile according to the display time (playback time of the content) of the screen created by the image and sound creation unit 142.
[0049] Fig. 16 is a diagram showing an example of a screen displaying the video to be played. For example, as shown in Fig. 16, the automatic driving control device 60 and the route control device 70 control the route or the driving speed so that the destination is reached after playing back 60 minutes of video and 30 minutes of video.
[0050] The image processing device 10 can move the image between the screens in response to an operation by a passenger. For example, the image processing device 10 can display the route displayed on the car navigation system on the screen on the ceiling.
[0051] In this case, when an operation is performed to display a first screen, which is a screen displayed on the car's car navigation system and shows the route from the current location to the destination, with the destination displayed at the top and the current location displayed at the bottom, on a screen mounted on the ceiling of the car, the image and audio creation unit 142 creates a second screen, which shows the destination at the bottom and the current location at the top, as a screen to be displayed on the screen mounted on the ceiling.
[0052] FIG. 17 is a diagram for explaining the movement of the screen between the screens. The left side of FIG. 17 shows the route displayed on the car navigation system. The right side of FIG. 17 shows the route displayed on the screen on the ceiling. As shown in FIG. 17, for example, the vertical positional relationship of "Makuhari" and "Takeshi" is reversed between the car navigation system and the ceiling screen. FIG. 18 is a diagram showing an example of the screen after the movement. As shown in FIG. 18, the destination is displayed in front on the ceiling screen, so the passenger can intuitively understand the route.
[0053] (Effects of the First Embodiment) As described above, the image processing device includes a creation unit (in the embodiment, the image and audio creation unit 142) that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle, and a switching unit (in the embodiment, the image and audio creation unit 142) that moves the screen before and after switching in a first direction parallel or perpendicular to the traveling direction of the vehicle when an operation is performed on the application to switch the displayed screen to a screen of the same mode, and moves the screen before and after switching in a second direction perpendicular to the first direction when an operation is performed to switch the displayed screen to a screen of a different mode. For example, the image processing device can determine the direction of screen movement according to the frequency of screen switching between and within modes. Therefore, according to the first embodiment, a screen display that is comfortable for the passengers can be performed.
[0054] The switching unit moves the screens before and after the switch in a first direction perpendicular to the driving direction of the vehicle when an operation is performed on the application to switch the displayed screen to a screen of the same mode, and moves the screens before and after the switch in a second direction parallel to the driving direction of the vehicle when an operation is performed on the application to switch the displayed screen to a screen of a different mode.
[0055] The switching unit reduces and moves the screen before and after switching. Therefore, according to the first embodiment, it is possible to prevent a passenger who operates the screen from getting sick.
[0056] The switching unit moves the screen before and after switching by lowering the brightness. Therefore, according to the first embodiment, it is possible to prevent a passenger who operates the screen from getting sick.
[0057] The creation unit creates a screen that displays a route from a current location to a destination, which is displayed on a screen provided on the ceiling of the vehicle, and displays the destination at the bottom and the current location at the top. Therefore, according to the first embodiment, information can be displayed in a manner that is intuitively easy for passengers to understand.
[0058] When an operation is performed to display a first screen, which is a screen displayed on a car navigation system for a car and shows a route from a current location to a destination, with the destination displayed at the top and the current location displayed at the bottom, on a screen provided on the ceiling of the car, the creation unit creates a second screen, which shows the destination displayed at the bottom and the current location displayed at the top, as a screen to be displayed on the screen provided on the ceiling. Therefore, according to the first embodiment, information can be displayed in a manner that is easy for passengers to intuitively understand, in accordance with the screen.
[0059] The creation unit creates a screen that displays a route from a current location to the destination, an icon indicating a category of the destination, and an icon indicating a passenger who has set the destination. Therefore, according to the first embodiment, a passenger can easily communicate a desired purpose to other passengers, particularly in a ride-sharing service.
[0060] The creation unit further creates audio for playing music that is associated in advance with the current location. Therefore, according to the first embodiment, the passenger can listen to music that matches the location.
[0061] The vehicle further includes a driving control unit that changes the route to the destination and the driving speed of the vehicle according to the display time of the screen created by the creation unit. Therefore, according to the first embodiment, the passenger can watch the video to the end. For example, according to the first embodiment, the passenger can learn information about the destination in advance when traveling.
[0062] [2. Application to moving objects] The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.
[0063] FIG. 19 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.
[0064] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 19, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053 are illustrated.
[0065] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.
[0066] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.
[0067] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.
[0068] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.
[0069] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.
[0070] The microcomputer 12051 can calculate control target values for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.
[0071] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.
[0072] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.
[0073] The audio / image output unit 12052 transmits at least one output signal of audio and image to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 19, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.
[0074] FIG. 20 is a diagram (1) showing an example of the installation position of the imaging unit.
[0075] In FIG. 20, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, 12105, and 12106.
[0076] The imaging units 12101, 12102, 12103, 12104, 12105, and 12106 are provided, for example, at the front nose, side mirrors, rear bumper, back door, above the windshield in the vehicle interior, roof, and other positions of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided above the windshield in the vehicle interior mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided above the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like. An imaging unit 12106 provided on the roof mainly captures images of the sky.
[0077] 20 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.
[0078] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.
[0079] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the image capture ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the image capture units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.
[0080] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.
[0081] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.
[0082] Fig. 21 is a diagram (2) showing an example of the installation position of the imaging unit. As shown in Fig. 21, the imaging unit 12106 captures an image of the sky above the vehicle. The imaging unit 12106 may be a wide-angle camera or a 360-degree camera.
[0083] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the microcomputer 12051 in the configuration described above. Specifically, the processing of the display output control unit 14 of the image processing device 10 is realized by the microcomputer 12051. In addition, the image of the sky above the automobile described in FIG. 4 and the like may be captured by the imaging unit 12106.
[0084] The automobile equipped with the image processing device may be connected to a network. FIG. 22 is a diagram showing an example of the configuration of a network system. As shown in FIG. 22, an automobile 100V equipped with an image processing device may be connected to another automobile 110V so as to be able to communicate data. The automobile 100V may also be connected to a server 100S via a network N so as to be able to communicate with the server 100S. The network N is, for example, the Internet. In this case, the image processing device provided in the automobile 100V can acquire content from the server 100S or the automobile 110V.
[0085] Note that the system including the devices in FIG. 1 may be realized as one device. That is, as one embodiment, a device having functions similar to those of the image processing device 10, as well as functions similar to at least one of the image display device 40, the audio output device 50, the automatic driving control device 60, and the route control device 70 can be realized. For example, an image processing device having functions similar to those of all of the image processing device 10, the image display device 40, the audio output device 50, the automatic driving control device 60, and the route control device 70 can provide processing equivalent to that of the system including all of the devices in FIG. 1.
[0086] 1 may be provided in a server on a network outside the vehicle, so that the vehicle and the server can communicate with each other. In this case, the image display device 40, the audio output device 50, the automatic driving control device 60, and the route control device 70 may be controlled by the server.
[0087] The present technology can also be configured as follows. (1) a creation unit that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screens before and after the switching in a first direction parallel or perpendicular to a traveling direction of the vehicle, and, when an operation is performed to switch a displayed screen to a screen of a different mode, moves the screens before and after the switching in a second direction perpendicular to the first direction; An image processing device comprising: (2) The image processing device described in (1), wherein the switching unit, when an operation is performed on the application to switch the currently displayed screen to a screen of the same mode, moves the screen before and after the switch in a first direction perpendicular to the driving direction of the vehicle, and when an operation is performed on the application to switch the currently displayed screen to a screen of a different mode, moves the screen before and after the switch in a second direction parallel to the driving direction of the vehicle. (3) The image processing device according to any one of (1) to (2), wherein the switching unit reduces and displays the screens before and after the switching and moves them. (4) The image processing device according to any one of (1) to (3), wherein the switching unit moves the screen before and after switching by lowering the brightness. (5) The image processing device according to any one of (1) to (4), wherein the creation unit creates a screen that displays a route from a current location to a destination, to be displayed on a screen provided on the ceiling of the vehicle, with the destination displayed at the bottom and the current location displayed at the top. (6) The creation unit is an image processing device described in any of (1) to (4), which, when an operation is performed to display a first screen, which is a screen displayed on a car navigation system of the car and shows a route from a current location to a destination, with the destination displayed at the top and the current location displayed at the bottom, on a screen mounted on the ceiling of the car, creates a second screen, which shows the destination at the bottom and the current location at the top, as a screen to be displayed on the screen mounted on the ceiling. (7) The image processing device according to any one of (1) to (6), wherein the creation unit creates a screen displaying a route from a current location to a destination, an icon indicating a category of the destination, and an icon indicating a passenger who has set the destination. (8) The image processing device according to any one of (1) to (7), wherein the creation unit further creates audio for playing music that is associated in advance with the current location. (9) The image processing device according to any one of (1) to (8), further comprising a driving control unit that changes the route to the destination and the driving speed of the automobile depending on the display time of the screen created by the creation unit. (10) A display system including a screen provided in a vehicle, a projector for projecting an image onto the screen, and an image processing device, The image processing device includes: a creation unit that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screens before and after the switching in a first direction parallel or perpendicular to a traveling direction of the vehicle, and, when an operation is performed to switch a displayed screen to a screen of a different mode, moves the screens before and after the switching in a second direction perpendicular to the first direction; A display system comprising: (11) The computer creating an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; An image processing method in which, when an operation is performed on the application to switch the displayed screen to a screen of the same mode, the screens before and after the switch are moved in a first direction parallel to or perpendicular to the driving direction of the vehicle, and, when an operation is performed on the application to switch the displayed screen to a screen of a different mode, the screens before and after the switch are moved in a second direction perpendicular to the first direction. (12) Computer, a creation unit that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screens before and after the switching in a first direction parallel or perpendicular to a traveling direction of the vehicle, and, when an operation is performed to switch a displayed screen to a screen of a different mode, moves the screens before and after the switching in a second direction perpendicular to the first direction; A recording medium on which a program for functioning as a [Explanation of symbols]
[0088] 10 Image processing device 11 Input section 12 Communications Department 13 Information Processing Section 14 Display output control section 20 Projector 30 Projector Screen 31 Liquid crystal display 40 Image display device 50 Audio output device 60 Automatic Driving Control Device 70 Route control device 141 Content Acquisition Department 142 Image and audio production section 143 Input information storage unit
Claims
1. a creation unit that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screens before and after the switching in a first direction parallel or perpendicular to a traveling direction of the vehicle, and, when an operation is performed to switch the displayed screen to a screen of a different mode, moves the screens before and after the switching in a second direction perpendicular to the first direction; the creation unit creates, when an operation is performed to display, on a screen provided on the ceiling of the vehicle, a first screen which is a screen displayed on a car navigation system of the vehicle and displays a route from a current location to a destination, and on which the destination is displayed at an upper part and the current location is displayed at a lower part, a second screen which displays the destination at a lower part and the current location at an upper part as a screen to be displayed on the screen provided on the ceiling; Image processing device.
2. The image processing device of claim 1, wherein the switching unit moves the screen before and after the switching in a first direction perpendicular to the traveling direction of the vehicle when an operation is performed on the application to switch the currently displayed screen to a screen of the same mode, and moves the screen before and after the switching in a second direction parallel to the traveling direction of the vehicle when an operation is performed on the application to switch the currently displayed screen to a screen of a different mode.
3. The image processing device according to claim 1 , wherein the switching unit reduces and displays the screen before and after the switching and moves the screen.
4. The image processing device according to claim 1 , wherein the switching unit moves the screen before and after the switching with a lowered brightness.
5. The image processing device according to claim 1 , wherein the creation unit creates a screen displaying a route from a current location to a destination, an icon indicating a category of the destination, and an icon indicating a passenger who has set the destination.
6. The image processing device according to claim 1 , wherein the creating unit further creates a sound for playing music that is associated in advance with the current location.
7. 2. The image processing device according to claim 1, further comprising a driving control unit that changes a route to a destination and a driving speed of the automobile in accordance with a display time of the screen created by the creation unit.
8. A display system including a screen provided in a vehicle, a projector for projecting an image onto the screen, and an image processing device, The image processing device includes: a creation unit that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screens before and after the switching in a first direction parallel or perpendicular to a traveling direction of the vehicle, and, when an operation is performed to switch the displayed screen to a screen of a different mode, moves the screens before and after the switching in a second direction perpendicular to the first direction; the creation unit creates, when an operation is performed to display, on a screen provided on the ceiling of the vehicle, a first screen which is a screen displayed on a car navigation system of the vehicle and displays a route from a current location to a destination, and on which the destination is displayed at an upper part and the current location is displayed at a lower part, a second screen which displays the destination at a lower part and the current location at an upper part as a screen to be displayed on the screen provided on the ceiling; Display system.
9. The computer creating an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; when an operation is performed on the application to switch the displayed screen to a screen of the same mode, the screens before and after the switching are moved in a first direction parallel to or perpendicular to a traveling direction of the vehicle, and when an operation is performed to switch the displayed screen to a screen of a different mode, the screens before and after the switching are moved in a second direction perpendicular to the first direction; a first screen for displaying a route from a current location to a destination, the first screen being displayed on the car navigation system of the car, the destination being displayed on an upper portion and the current location being displayed on a lower portion, on a screen provided on the ceiling of the car; and a second screen for displaying the destination being displayed on a lower portion and the current location being displayed on an upper portion, when an operation is performed to display the first screen on the screen provided on the ceiling. Image processing methods.
10. Computer, a creation unit that creates an application screen that displays a screen belonging to any one of a plurality of modes on a screen provided in the vehicle; a switching unit that, when an operation is performed on the application to switch a displayed screen to a screen of the same mode, moves the screens before and after the switching in a first direction parallel or perpendicular to a traveling direction of the vehicle, and, when an operation is performed to switch a displayed screen to a screen of a different mode, moves the screens before and after the switching in a second direction perpendicular to the first direction; A recording medium having a program recorded thereon for causing the device to function as a the creation unit creates, when an operation is performed to display, on a screen provided on the ceiling of the vehicle, a first screen which is a screen displayed on a car navigation system of the vehicle and displays a route from a current location to a destination, and on which the destination is displayed at an upper part and the current location is displayed at a lower part, a second screen which displays the destination at a lower part and the current location at an upper part as a screen to be displayed on the screen provided on the ceiling; Recording medium.
Citation Information
Patent Citations
Display device for vehicle
JP2002328624A
AV system
JP2008001308A
Control device and program
JP2013222224A
Navigation device, head-up display, control method, program, and storage medium
JP2015105903A
On-vehicle system
JP2019158529A
