IMAGE PROCESSING APPARATUS, DISPLAY SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM

Abstract

The present invention makes it possible to optimize a mode of a screen to view content. An image processing device (10) according to the present disclosure includes a shape determination unit (152) for determining, on the basis of an automated driving state of an automobile and information about content to be displayed on a screen mounted in the automobile, a shape of the screen suitable for displaying the content, and an image creation unit (153) for creating an image of content to be displayed on the screen, in accordance with the shape determined to be suitable for displaying the content by the shape determination unit (152).

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, a display system, an image processing method, and a recording medium. [Background technology]

[0002] There is known a technology for displaying content on a screen such as an organic electroluminescence (EL) sheet installed on the ceiling inside a vehicle. For example, there is known a technology for displaying a sky map as seen from a current location on a screen. There is also known a technology for displaying a single road showing a route to a destination and a position on the single road on a screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-328624 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technology has a problem in that the screen configuration may not be suitable for viewing content. For example, it is difficult for a driver to comfortably view content displayed on an overhead screen while maintaining a good forward field of vision.

[0005] Therefore, the present disclosure proposes an image processing device, a display system, an image processing method, and a recording medium that can optimize the aspect of the screen for viewing content. [Means for solving the problem]

[0006] In order to solve the above problems, one form of image processing device according to the present disclosure includes a judgment unit that judges whether or not conditions associated with each of a plurality of shapes of the screen are satisfied based on information related to the autonomous driving state of the automobile and the content displayed on the screen provided in the automobile, and a screen shape output unit that outputs information identifying the shape associated with the condition judged to be satisfied by the judgment unit. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a screen. [Diagram 3] FIG. 2 shows the screen as seen from the rear of the car. [Figure 4] FIG. 1 is a diagram showing a screen shape pattern. [Diagram 5] FIG. 1 shows the display pattern of metadata. [Figure 6] FIG. 2 shows the metadata display pattern. [Figure 7] FIG. 13 is a diagram showing the relationship between the screen shape and the inclination of the seat backrest. [Figure 8] FIG. 1 shows an example of content displayed on a screen. [Figure 9] FIG. 1 shows the content display pattern. [Figure 10] FIG. 2 shows an example of content displayed on the screen. [Figure 11] FIG. 3 shows an example of content displayed on the screen. [Figure 12] FIG. 2 shows the content display pattern. [Figure 13] FIG. 4 shows an example of content displayed on the screen. [Figure 14] FIG. 3 shows the content display pattern. [Figure 15]FIG. 5 shows an example of content displayed on the screen. [Figure 16] FIG. 4 shows the content display pattern. [Figure 17] FIG. 6 shows an example of content displayed on the screen. [Figure 18] FIG. 5 shows the content display pattern. [Figure 19] FIG. 7 shows an example of content displayed on the screen. [Figure 20] FIG. 6 shows the content display pattern. [Figure 21] FIG. 8 shows an example of content displayed on the screen. [Figure 22] FIG. 7 shows the content display pattern. [Figure 23] 4 is a flowchart showing a process flow of the image processing device according to the first embodiment. [Figure 24] 10 is a flowchart showing a process for determining a screen shape. [Diagram 25] FIG. 11 is a diagram illustrating an example of the configuration of an image processing device according to a second embodiment. [Figure 26] FIG. 13 is a diagram showing the relationship between the screen shape and the inclination of the seat backrest. [Figure 27] 10 is a flowchart showing a process flow of an image processing apparatus according to a second embodiment. [Figure 28] FIG. 13 is a diagram illustrating an example of the configuration of an image processing device according to a third embodiment. [Figure 29] 13 is a flowchart showing a flow of processing in an image processing device according to a third embodiment. [Diagram 30] FIG. 9 shows an example of content displayed on the screen. [Diagram 31] 1 is a block diagram showing a schematic configuration example of an automobile control system; [Diagram 32] FIG. 1 is a diagram showing an example of the installation position of the imaging unit. [Diagram 33]FIG. 2 shows an example of the installation position of the imaging unit. [Diagram 34] FIG. 1 illustrates an example of the configuration of a network system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0009] The explanation will be given in the following order: 1. First embodiment 2. Second embodiment 3. Third embodiment 4. Application examples for mobile objects

[0010] [1. First embodiment] Fig. 1 is a diagram showing an example of the configuration of an image processing device according to the first embodiment. The image processing device 10 shown in Fig. 1 is a device for performing processing related to displaying content on a screen provided in an automobile equipped with an automatic driving function.

[0011] The input unit 11 is an input device that receives inputs such as the driving conditions of the vehicle, the conditions of the passengers, and operations by the passengers. The input unit 11 includes various sensors such as an image sensor, a depth sensor, and a touch sensor. The image sensor is a sensor that obtains two-dimensional images, such as a visible light camera and an infrared camera. The depth sensor is a sensor that obtains three-dimensional information including depth, such as a stereo camera, or a sensor that can implement a time-of-flight method, a structured light method, or the like.

[0012] Here, the input unit 11 can accept an autonomous driving state as the state of the automobile. The autonomous driving state includes not only whether the automobile is in autonomous driving or not, but also the autonomous driving level. The input unit 11 also accepts operations by the passenger, such as operations by a touch display, operations by voice, and gesture operations using a skeletal field.

[0013] The communication unit 12 is an interface for performing data communication with other devices, and is realized by, for example, a network interface card (NIC).

[0014] The information processing unit 13 executes various processes related to content display. The information processing unit 13 is realized by, for example, a computer equipped with a CPU (Central Processing Unit). The information processing unit 13 performs a process for displaying an image included in the content based on information received from the input unit 11. The information processing unit 13 performs drawing control of multi-content such as a window that displays an application, and event delivery such as a touch for each content. The information processing unit 13 performs a process equivalent to a control layer of a general OS.

[0015] The image display device 40 displays an image included in the content. The image display device 40 may be a projector and a projector screen. The image display device 40 may also be a display such as a liquid crystal display. Hereinafter, a surface on which an image is actually displayed, such as a projector screen or a liquid crystal display, will simply be referred to as a screen.

[0016] The shape output unit 14 outputs the screen shape. Specifically, the shape output unit 14 changes the screen shape. The screen shape and a method for changing the screen shape will be described later.

[0017] The display control unit 15 determines the screen shape and creates an image to be displayed. The display control unit 15 is realized by, for example, a computer equipped with a CPU. As shown in FIG. 1, the display control unit 15 includes a content acquisition unit 151, a shape determination unit 152, an image creation unit 153, and a shape storage unit 154.

[0018] The content acquisition unit 151 acquires content. The content acquisition unit 151 may acquire content from a predetermined storage device, or may acquire content from an external device or another vehicle via a network.

[0019] Shape determination unit 152 determines the shape of the screen suitable for displaying the content, based on the automatic driving state of the car and information related to the content to be displayed on the screen provided in the car.

[0020] The image creation unit 153 creates an image of the content to be displayed on the screen in accordance with the screen shape determined by the shape determination unit 152 to be suitable for displaying the content.

[0021] Here, the screens included in the image display device 40 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the screens. As shown in Fig. 2, the automobile is provided with a lower front screen 31, an upper front screen 32, and a look-up screen 33. Each screen may be a projector screen that displays an image projected from a projector, or may be a liquid crystal display.

[0022] Fig. 2 shows an example of a case where a projector projects an image. As shown in Fig. 2, projector 20 projects an image onto each screen from the rear of a car. Projector 20 may be capable of changing the direction and height at which the image is projected in accordance with the shape of the screen. Projector 20 may also be provided on the headrest of the driver's seat.

[0023] The lower front screen 31 and the upper front screen 32 are disposed inside the windshield. The upper front screen 32 is disposed closer to the ceiling than the lower front screen 31. In addition, the look-up screen 33 is disposed on the ceiling.

[0024] 2, there are two front screens, a lower front screen 31 and an upper front screen 32, but the number of front screens may be one, or three or more. Also, the front screen may be provided on the sun visor.

[0025] Figure 3 shows the screens as seen from the rear of the car. As shown in Figure 3, the lower windscreen 31 and the upper windscreen 32 obscure part of the driver's view through the windshield.

[0026] Each screen is movable. For example, the shape output unit 14 can move each screen along the ceiling and the windshield using an actuator such as a motor as power. In particular, the lower front screen 31 and the upper front screen 32 move between a position visible to the passenger and a position invisible to the passenger. The state in which each screen is in a position visible to the passenger is called the extended state. On the other hand, the state in which each screen is in a position invisible to the passenger is called the stored state.

[0027] In the following description, the person in the car is called a passenger. Furthermore, the person in the driver's seat is called a driver, regardless of whether or not he or she is actually operating the car. For example, when the car is running using the automatic driving function, the passenger in the driver's seat is called a driver, even if he or she is not operating the car at all.

[0028] The arrangement of the lower front screen 31, the upper front screen 32, and the look-up screen 33 is called the screen shape. There are several patterns of screen shapes depending on the screen to be used. Fig. 4 is a diagram showing the patterns of the screen shape. The patterns of the screen shape are stored in the shape storage unit 154.

[0029] First, as shown in Fig. 4, in a pattern in which only the look-up screen 33 is used, both the lower front screen 31 and the upper front screen 32 are in a stored state. In the example of Fig. 4, both the lower front screen 31 and the upper front screen 32 move into the gap between the look-up screen 33 and the ceiling.

[0030] Also, as shown in Fig. 4, in a pattern in which part of the front screen is used, the lower front screen 31 is in a stored state and the upper front screen 32 is in a deployed state. In the example of Fig. 4, the lower front screen 31 moves to the gap between the look-up screen 33 and the ceiling. Meanwhile, the upper front screen 32 moves to a predetermined position inside the windshield.

[0031] Furthermore, as shown in Fig. 4, in a pattern in which all screens are used, both the lower front screen 31 and the upper front screen 32 are in an extended state. In the example of Fig. 4, both the lower front screen 31 and the upper front screen 32 move to a predetermined position inside the windshield.

[0032] The shape determination unit 152 determines one of the three patterns shown in Fig. 4 as an optimal pattern based on a predetermined condition. In the first embodiment, the shape determination unit 152 performs the determination based on at least one of an automatic driving state, content information, and seat setting information.

[0033] The autonomous driving state is whether or not the car is autonomous and the autonomous driving level. That is, the shape determination unit 152 determines the shape of the screen based on the autonomous driving state including whether or not the car is autonomous and the autonomous driving level.

[0034] There are various levels of autonomous driving, including level 3, where the system performs all driving tasks under limited conditions and the driver is expected to respond to requests for intervention by the system, and level 5, where the system performs all driving tasks. Note that the driving levels listed here are just examples.

[0035] When the automatic driving state is manual driving, the shape determination unit 152 determines that a shape in which the screen is not arranged on the windshield of the vehicle is suitable for displaying the content. When the automatic driving state is manual driving, it means that the automatic driving function is turned off and the driver is manually operating the vehicle. In this case, it is considered that the driver will secure a forward field of vision for safety. Therefore, when the automatic driving state is manual driving, the shape determination unit 152 determines that a shape in which both the lower front screen 31 and the upper front screen 32 are in a stored state is suitable.

[0036] When the autonomous driving state is during autonomous driving, the shape determination unit 152 determines that a shape in which at least one of the multiple screens is arranged on the windshield of the vehicle is suitable for displaying content.

[0037] Here, it is assumed that the higher the autonomous driving level, the less frequently the driver is required to perform manual operations. For example, when the autonomous driving level is 3, if the system detects an obstacle that is difficult to automatically avoid, the driver is required to manually operate the steering wheel and brakes. On the other hand, when the autonomous driving level is 5, the driver is not required to perform any operations at all.

[0038] For example, when the autonomous driving level is 3, the shape determination unit 152 determines that a shape in which the lower front screen 31 is in a stored state and the upper front screen 32 is in an extended state (corresponding to the front screen partially used in FIG. 4) is suitable. In this case, the driver can see the front from part of the windshield, and can perform operations if requested.

[0039] Also, for example, when the autonomous driving level is 5, the shape determination unit 152 determines that a shape in which both the lower front screen 31 and the upper front screen 32 are extended (using the entire screen in FIG. 4) is appropriate. In this case, the driver does not need to perform any operation, and can watch the content using the entire screen.

[0040] Furthermore, the shape determination unit 152 determines the shape based on information related to the content. For example, if the size of the screen on which each content is displayed is determined in advance, the shape determination unit 152 determines the shape according to that size.

[0041] Here, the driving of the driver can be assisted by displaying images captured by a car navigation system, a rear-mirror camera, a rear seat, and the like on the upper front screen 32. For this reason, when content for assisting the driver is displayed, the shape determination unit 152 may determine that a shape in which the upper front screen 32 is in an extended state (partial use of the front screen in FIG. 4) is appropriate even in manual driving.

[0042] Furthermore, the sense of immersion can be improved by displaying content with all screens, including the lower front screen 31, in the extended state. For this reason, when content that requires a sense of immersion, such as watching a sports event or in a planetarium, is displayed, the shape determination unit 152 determines that a shape in which both the lower front screen 31 and the upper front screen 32 are extended (using all screens in FIG. 4) is appropriate.

[0043] The shape determination unit 152 may perform the determination based on the information related to the content only when the vehicle is in an automatic driving mode. Also, when a screen shape is specified by a passenger, the shape determination unit 152 may give priority to the specified screen shape.

[0044] Some content has main content and information related to the main content (hereinafter referred to as content metadata). For example, if the main content is a video of a soccer game, the current scores of both teams may be the metadata. The image creation unit 153 can determine the display position of the metadata according to the shape of the screen.

[0045] FIG. 5 is a diagram (1) showing a display pattern of metadata. The area surrounded by a solid line indicates the screen that is used (the image is displayed). Moreover, the area surrounded by a dotted line indicates the screen that is not used (the image is not displayed). As shown in FIG. 5, in a pattern in which only the lookup screen is used, the image creation unit 153 displays the metadata in a part of the image of the lookup screen 33. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. This is because, in a pattern in which only the lookup screen is used, it is sufficient for only the passengers in the rear seats to be able to view the content.

[0046] Furthermore, in a pattern in which part of the front screen is used, the image creation unit 153 displays metadata on the upper front screen 32. In a pattern in which part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used.

[0047] Furthermore, in a pattern in which all screens are used, the image creation unit 153 displays metadata on the lower front screen 31. In a pattern in which all screens are used, the lower front screen 31, the upper front screen 32 and the lookup screen 33 are used.

[0048] As described above, driving assistance information may be displayed on the screen. The image creation unit 153 can determine the display position of the driving assistance information according to the shape of the screen, similar to the metadata. However, depending on the autonomous driving state, the display of the driving assistance information may not be necessary.

[0049] FIG. 6 is a diagram (2) showing a display pattern of metadata. The area surrounded by a solid line indicates the screen that is used (the image is displayed). Moreover, the area surrounded by a dotted line indicates the screen that is not used (the image is not displayed). As shown in FIG. 6, in a pattern in which only the lookup screen is used, the image creation unit 153 can prevent the display of driving assistance information. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. This is because it is difficult for a driver who is manually driving to see the lookup screen 33.

[0050] Furthermore, in a pattern in which part of the front screen is used, the image creation unit 153 displays driving assistance information on the upper front screen 32. In this way, the image creation unit 153 creates an image to be displayed on a screen arranged on the windshield of the vehicle, among the multiple screens, and which shows information to assist driving. In a pattern in which part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the look-up screen 33 are used. This allows the driver during automatic driving to check information on the upper front screen 32 and to perform manual driving if an operation request is made.

[0051] In addition, in a pattern in which the entire screen is used, the image creation unit 153 does not display driving assistance information. In a pattern in which the entire screen is used, the lower front screen 31, the upper front screen 32, and the look-up screen 33 are used. This corresponds to the case in which the autonomous driving level is 5, as described above, and the driver does not perform manual driving.

[0052] The shape determination unit 152 determines the shape of the screen suitable for displaying the content based on the autonomous driving state, information about the content, and the inclination of the backrest of the car seat. For example, when the car seat is inclined to a degree that is unsuitable for driving, the shape determination unit 152 determines that a shape in which a part of the screen is placed on the windshield of the car is suitable for displaying the content. It is considered that the driver will tilt the backrest of the seat to look up at the look-up screen.

[0053] A predetermined angle may be set in advance as the degree of inclination unsuitable for driving. For example, when the seat back is inclined by 30 degrees or more, the shape determination unit 152 may determine that the seat back is inclined to an extent unsuitable for driving. In addition, the angle of the seat back during manual driving is recorded, and when the inclination of the seat back exceeds a predetermined range calculated from the recorded angle, the shape determination unit 152 determines that the seat back is inclined to an extent unsuitable for driving. The inclination of the seat back can be known from the state of the backrest inclination adjustment mechanism. In addition, the inclination of the backrest may be measured by a tilt measurement sensor provided on the backrest.

[0054] FIG. 7 is a diagram showing the relationship between the screen shape and the inclination of the seat backrest. State (a) is the initial state, in which the backrest of the front seat (driver or passenger seat) is not tilted. When the front seat backrest is tilted, a transition occurs from state (a) to state (b). State (b) is the state in which the shape of the screen is not changed after the front seat backrest is tilted.

[0055] At this time, the shape determination unit 152 determines that a shape in which one or both of the lower front screen 31 and the upper front screen 32 are in the extended state is suitable for viewing content. As a result, as the passenger tilts the seat back, a transition occurs from state (b) to state (c-1) or state (c-2). In other words, one or both of the lower front screen 31 and the upper front screen 32 move and enter the extended state.

[0056] In states (c-1) and (c-2), the shape of the screen has already been deformed after the front seat backs have been tilted. In state (c-1), both the lower front screen 31 and the upper front screen 32 are in the extended state. In state (c-2), only the upper front screen 32 is in the extended state.

[0057] Hereinafter, the display mode of the content for each screen shape will be described with examples. FIG. 8 is a diagram (1) showing an example of the content displayed on the screen. As shown in FIG. 8, the image creation unit 153 creates an image to be displayed on the look-up screen 33 arranged on the ceiling of the car among the multiple screens, which is an image taken of the sky above the car, and also creates an image to be displayed on the upper front screen 32 arranged on the windshield of the car, which is an image taken of the rear of the car. In this way, by displaying the surrounding image in real time, passengers in a car that is not an open-top car can experience the sense of freedom of an open-top car.

[0058] FIG. 9 is a diagram (1) showing a display pattern of contents. The area surrounded by a solid line indicates the screen that is used (the image is displayed). Moreover, the area surrounded by a dotted line indicates the screen that is not used (the image is not displayed). As shown in FIG. 9, when only the lookup screen is used and when all the screens are used, the image taken of the rear of the car is not displayed on any screen. This is because, as described above, driving assistance information is not required. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In a pattern in which a part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In a pattern in which all the screens are used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0059] Fig. 10 is a diagram (2) showing an example of the content displayed on the screen. As shown in Fig. 10, the image creation unit 153 creates an image of the back seat, which is to be displayed on the upper front screen 32, which is one of the multiple screens and is arranged on the windshield of the automobile. This allows the passengers in the front seats to talk to the passengers in the back seats while checking their expressions.

[0060] Fig. 11 is a diagram (3) showing an example of content displayed on the screen. In Fig. 9, an image of the sky above the car is displayed on the look-up screen 33, whereas in Fig. 11, an image different from the image of the sky above the car is displayed on the look-up screen 33. For example, by displaying a spectacular view or a view of a foreign tourist spot on the look-up screen 33, the passenger can enjoy the view of a place that is difficult to reach.

[0061] FIG. 12 is a diagram (2) showing a display pattern of contents. The area surrounded by a solid line indicates the screen that is used (the image is displayed). Moreover, the area surrounded by a dotted line indicates the screen that is not used (the image is not displayed). As in the example of FIG. 9, in the example of FIG. 12, when only the lookup screen is used and when all the screens are used, the image taken behind the car is not displayed on any screen. In the pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In the pattern in which part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In the pattern in which all the screens are used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0062] The image creation unit 153 creates a first image for display on a look-up screen 33 arranged on the roof of the vehicle among the multiple screens, and a second image for display on an upper front screen 32 arranged on the windshield of the vehicle, the second image displaying an explanatory text for the first image.

[0063] FIG. 13 is a diagram (4) showing an example of content displayed on the screen. In the example of FIG. 13, an image of a polar bear published in an illustrated book is displayed as a first image on the lookup screen 33. Also, an explanatory text published in the illustrated book is displayed on the upper front screen 32. Also, the image creation unit 153 may create an image according to the destination of the car. For example, if the car is heading to an aquarium, the image creation unit 153 creates an image of a fish illustrated book and an image of the explanatory text.

[0064] FIG. 14 is a diagram (3) showing a display pattern of contents. The area surrounded by a solid line indicates the screen to be used (the image is displayed). The area surrounded by a dotted line indicates the screen not to be used (the image is not displayed). In the example of FIG. 14, metadata is displayed according to the display pattern described with reference to FIG. 5. In this case, the metadata is an explanatory text of an illustrated book. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In a pattern in which a part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In a pattern in which all the screens are used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0065] Fig. 15 is a diagram (5) showing an example of the content displayed on the screen. In the example of Fig. 15, a video of a soccer game is displayed as the first image on the look-up screen 33. Also, the upper front screen 32 displays the scores of both teams.

[0066] FIG. 16 is a diagram (4) showing a display pattern of the content. The area surrounded by a solid line indicates the screen that is used (the image is displayed). The area surrounded by a dotted line indicates the screen that is not used (the image is not displayed). In the example of FIG. 14, metadata is displayed according to the display pattern described with reference to FIG. 5. In this case, the metadata is the scores of both teams. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In a pattern in which a part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In a pattern in which all the screens are used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0067] Fig. 17 is a diagram (6) showing an example of the content displayed on the screen. In the example of Fig. 17, an image of the route from the current location to the destination is displayed as the first image on the look-up screen 33. Also, the upper front screen 32 displays the distance to the destination.

[0068] FIG. 18 is a diagram (5) showing a display pattern of contents. The area surrounded by a solid line indicates the screen to be used (the image is displayed). The area surrounded by a dotted line indicates the screen not to be used (the image is not displayed). As in the example of FIG. 9, in the example of FIG. 18, when only the lookup screen is used and when all the screens are used, the image taken of the rear of the car is not displayed on any screen. This allows the amount of information displayed on the screen to be increased when manual driving is not performed. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In a pattern in which a part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In a pattern in which all the screens are used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0069] Fig. 19 is a diagram (7) showing an example of the content displayed on the screen. In the example of Fig. 19, images corresponding to answer choices of the quiz are displayed as the first images on the lookup screen 33. Also, the question text of the quiz is displayed on the upper front screen 32.

[0070] FIG. 20 is a diagram (6) showing a display pattern of content. The area surrounded by a solid line indicates a screen that is used (an image is displayed). The area surrounded by a dotted line indicates a screen that is not used (an image is not displayed). In the example of FIG. 20, metadata is displayed according to the display pattern described with reference to FIG. 5. In this case, the metadata is a question text for a quiz. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In a pattern in which a part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In a pattern in which all the screens are used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0071] Fig. 21 is a diagram (8) showing an example of the content displayed on the screen. In the example of Fig. 21, an image of the running application is displayed on the lookup screen 33. Also, an image of the application launcher is displayed on the upper front screen 32.

[0072] In this manner, the image creation unit 153 creates an image of the running application for display on a first screen located on the roof of the vehicle, and an image of the application's launcher for display on a second screen located on the windshield of the vehicle.

[0073] FIG. 22 is a diagram (7) showing a display pattern of contents. The area surrounded by a solid line indicates a screen that is used (an image is displayed). The area surrounded by a dotted line indicates a screen that is not used (an image is not displayed). In the example of FIG. 22, metadata is displayed according to the display pattern described with reference to FIG. 5. In this case, the metadata is a list of icons of executable applications or an image of an application launcher. In a pattern in which only the lookup screen is used, the lower front screen 31 and the upper front screen 32 are not used, and the lookup screen 33 is used. In a pattern in which a part of the front screen is used, the lower front screen 31 is not used, and the upper front screen 32 and the lookup screen 33 are used. In a pattern in which the entire screen is used, the lower front screen 31, the upper front screen 32, and the lookup screen 33 are used.

[0074] Fig. 23 is a flowchart showing a process flow of the image processing device according to the first embodiment. As shown in Fig. 23, it is determined whether or not the passenger views the content (step S10). For example, when the passenger performs an operation to instruct the display of the content by voice or gesture, the image processing device 10 determines that the passenger views the content.

[0075] When it is determined that the passenger does not view the content (step S10, No), the image processing device 10 executes a stop process. For example, the image processing device 10 executes the stop process by not displaying an image and by putting the front screen in a retracted state (step S20). Alternatively, the image processing device 10 may execute the stop process by displaying a black screen.

[0076] When the image processing device 10 determines that the passenger will view the content (step S10, Yes), it acquires the automatic driving status, content information, and seat setting information (step S30). Then, the image processing device 10 determines the screen shape (step S40). Furthermore, the image processing device 10 outputs the screen shape (step S50), creates an image corresponding to the screen shape, and displays the image (step S70).

[0077] Fig. 24 is a flowchart showing the flow of the process of determining the screen shape (step S40 in Fig. 23). As shown in Fig. 24, first, the image processing device 10 determines whether or not the vehicle is in an autonomous driving mode (step S401). If the vehicle is in an autonomous driving mode (step S401, Yes), the image processing device 10 determines whether or not the condition for using the entire screen is met (step S402). If the vehicle is not in an autonomous driving mode (step S401, No) or if the condition for using the entire screen is not met (step S402), the image processing device 10 determines whether or not the condition for using a portion of the front screen is met (step S404).

[0078] If the condition for using the entire screen is met (step S402, Yes), the image processing device 10 determines that the optimal screen shape pattern is for using the entire screen (step S403). If the condition for using only the front screen is met (step S404, Yes), the image processing device 10 determines that the optimal screen shape pattern is for using only the front screen (step S405). If the condition for using only the front screen is not met (step S404, No), the image processing device 10 determines that the optimal screen shape pattern is for using only the lookup screen (step S406).

[0079] (Effects of the First Embodiment) As described above, the image processing device includes a determination unit (shape determination unit 152 in the embodiment) that determines the shape of the screen suitable for displaying the content based on the autonomous driving state of the automobile and information about the content to be displayed on the screen provided in the automobile, and a creation unit (image creation unit 153 in the embodiment) that creates an image of the content to be displayed on the screen according to the screen shape determined by the determination unit to be suitable for displaying the content. For example, the image processing device can determine how much field of vision the driver should secure according to the autonomous driving state. Also, the image processing device can determine the shape of the screen suitable for the content. Therefore, according to the first embodiment, the aspect of the screen can be optimized for viewing the content.

[0080] The determination unit determines the shape of the screen based on whether the vehicle is in autonomous driving mode and the autonomous driving state including the autonomous driving level. Therefore, according to the first embodiment, the shape of the screen can be finely optimized according to not only whether the vehicle is in autonomous driving mode but also the autonomous driving level.

[0081] When the autonomous driving state is autonomous driving, the determination unit determines that a shape in which at least one of the multiple screens is arranged on the windshield of the vehicle is suitable for displaying content. Therefore, according to the first embodiment, when the vehicle is autonomous driving, the screen can be optimized to a shape that prioritizes viewing of content.

[0082] When the autonomous driving state is manual driving, the determination unit determines that a shape in which the screen is not placed on the windshield of the vehicle is suitable for displaying the content. Therefore, according to the first embodiment, when the vehicle is manually driven, the screen can be optimized to a shape that prioritizes safety.

[0083] The determination unit determines the shape of the screen suitable for displaying the content based on the autonomous driving state, information about the content, and the inclination of the backrest of the seat in the vehicle. Therefore, according to the first embodiment, passengers can comfortably watch the content on a screen with a shape optimized for the inclination of the backrest of the seat.

[0084] When the seat back of the car is tilted to an extent that makes it unsuitable for driving, the determination unit determines that a shape in which a part of the screen is placed on the windshield of the car is suitable for displaying the content. Therefore, according to the first embodiment, the driver can watch the content more comfortably when not driving.

[0085] The creation unit creates an image showing information for assisting driving, which is to be displayed on a screen arranged on the windshield of the vehicle among the multiple screens. Therefore, according to the first embodiment, the information for assisting driving can be displayed at a position that is easy for the driver to see while driving.

[0086] The creation unit creates an image to be displayed on a first screen, which is disposed on the ceiling of the vehicle, of the multiple screens, and an image to be displayed on a second screen, which is disposed on the windshield of the vehicle, and an image to be displayed on the rear of the vehicle. Therefore, according to the first embodiment, an image that allows the driver to check the rear view can be displayed at a position that is easy for the driver to see while driving.

[0087] The creation unit creates an image of the back seat, which is to be displayed on a screen arranged on the windshield of the vehicle among the multiple screens. Therefore, according to the first embodiment, the passenger in the front seat can talk to the passenger in the back seat while checking the facial expression of the passenger in the front seat.

[0088] The creation unit creates a first image to be displayed on a first screen arranged on the ceiling of the vehicle among the multiple screens, and a second image to be displayed on a second screen arranged on the windshield of the vehicle, the second image displaying an explanatory text for the first image. Therefore, according to the first embodiment, a passenger can view the explanatory text and enjoy the content more deeply.

[0089] The creation unit creates an image of the running application to be displayed on a first screen disposed on the ceiling of the vehicle, and an image of a launcher of the application to be displayed on a second screen disposed on the windshield of the vehicle. Therefore, according to the first embodiment, a passenger can start and run an application by an intuitive operation.

[0090] [2. Second embodiment] Fig. 25 is a diagram showing an example of the configuration of an image processing device according to the second embodiment. As shown in Fig. 25, the image processing device 10a of the second embodiment includes a seat setting device 50, unlike the image processing device 10 of the first embodiment. In the second embodiment, the image processing device 10 determines the optimal inclination of the backrest of the seat based on the autonomous driving situation and content information, and actually sets it according to the determination result.

[0091] The shape determination unit 152 further determines the inclination of the backrest of the car seat suitable for displaying the content. Then, the seat setting device 50 sets the inclination of the backrest of the seat according to the inclination of the backrest of the seat determined to be suitable for displaying the content by the shape determination unit 152. At this time, the backrest of the seat automatically inclines according to the set inclination.

[0092] Fig. 26 is a diagram showing the relationship between the screen shape and the inclination of the seat backrest. State (d) is the initial state, in which the backrest of the front seat (driver or passenger seat) is not inclined and the shape of the screen is not deformed. The undeformed state in state (d) corresponds to the state in Fig. 5 where only the look-up screen is used. That is, the undeformed state in state (d) is the state where both the lower front screen 31 and the upper front screen 32 are stored and only the look-up screen 33 is used (images are displayed).

[0093] Here, the shape determination unit 152 determines the shape of the screen and the inclination of the seat back suitable for the content. At this time, a transition occurs from state (d) to state (e). That is, the upper front screen 32 moves and enters the extended state. Meanwhile, the inclination of the seat back does not change. State (e) is a state in which the shape of the screen has been transformed and the inclination of the seat back has not been changed. The transformed state in state (e) corresponds to the state in which the front screen is partially used in FIG. 5. That is, the transformed state in state (e) is a state in which the lower front screen 31 is in a stored state, the upper front screen 32 is in a extended state, and the upper front screen 32 and the look-up screen 33 are used (images are displayed).

[0094] Furthermore, when another content that uses the entire screen is displayed, for example, the shape determination unit 152 further determines the shape of the screen and the inclination of the seat backrest. At this time, a transition occurs from state (e) to state (f). That is, the lower front screen 31 moves and becomes a protruding state. This results in a state in which the entire screen is used. Furthermore, the inclination of the front seat backrest is changed. State (f) is a state in which the shape of the screen has been transformed and the inclination of the seat backrest has been changed. The transformed state in state (f) corresponds to the use of the entire screen in FIG. 5. That is, the transformed state in state (f) is a state in which both the lower front screen 31 and the upper front screen 32 are in a protruding state, and the lower front screen 31, the upper front screen 32, and the look-up screen 33 are used (images are displayed). Here, since the entire screen including the look-up screen 33 is used, the front seat backrest is inclined at an angle greater than that suitable for driving.

[0095] Fig. 27 is a flowchart showing a process flow of the image processing device according to the second embodiment. As shown in Fig. 27, it is determined whether or not the passenger views the content (step S10). When it is determined that the passenger does not view the content (step S10, No), the image processing device 10 executes a stop process.

[0096] When the image processing device 10 determines that the passenger will view the content (step S10, Yes), it acquires the automatic driving status and content information (step S31). Then, the image processing device 10 determines the screen shape (step S40). Furthermore, the image processing device 10 outputs the screen shape (step S50), sets the inclination of the seat backrest (step S60), creates an image corresponding to the screen shape, and displays the image (step S70).

[0097] The determination unit (shape determination unit 152 in this embodiment) further determines the inclination of the backrest of the automobile seat suitable for displaying the content. The seat setting unit (seat setting device 50 in this embodiment) sets the inclination of the seat according to the inclination of the backrest of the seat determined by the determination unit to be suitable for displaying the content. Therefore, according to the second embodiment, the passenger can watch the content in a comfortable position without manually changing the inclination of the backrest of the seat.

[0098] [3. Third embodiment] Fig. 28 is a diagram showing an example of the configuration of an image processing device according to the third embodiment. As shown in Fig. 28, the image processing device 10b of the second embodiment receives input of shape information, unlike the image processing device 10a of the second embodiment. In the third embodiment, the image processing device 10 determines the optimal seat back inclination based on shape information specified by the passenger in addition to the automatic driving situation and content information, and actually sets it according to the determination result.

[0099] The shape information is a screen shape designated by the passenger. For example, the passenger designates whether to use only the lookup screen, use a part of the front screen, or use the entire screen. The image processing device 10b determines the screen shape by prioritizing the designation of the passenger. However, when it is impossible to change to the screen shape designated by the passenger, the image processing device 10b may not prioritize the designation of the passenger. For example, when the passenger designates the use of the entire screen during manual driving, the image processing device 10b does not determine that the optimal screen shape is the use of the entire screen.

[0100] Fig. 29 is a flowchart showing a process flow of the image processing device according to the third embodiment. As shown in Fig. 29, it is determined whether or not the passenger views the content (step S10). When it is determined that the passenger does not view the content (step S10, No), the image processing device 10 executes a stop process.

[0101] When the image processing device 10 determines that the passenger will view the content (step S10, Yes), it acquires the automatic driving situation, content information, and shape information (step S31). Then, the image processing device 10 determines the screen shape (step S40). Furthermore, the image processing device 10 outputs the screen shape (step S50), sets the inclination of the seat backrest (step S60), creates an image corresponding to the screen shape, and displays the image (step S70). An example of the inclination of the seat backrest is the example of FIG. 26 described in the second embodiment. In the example of FIG. 26, the inclination of the front seat backrest may be set to either an angle suitable for driving or an angle larger than the angle suitable for driving.

[0102] (Other embodiments) In the embodiments described so far, the main content and metadata corresponding to the main content are displayed simultaneously, as shown in Fig. 14 etc. However, unrelated main content and metadata, or different main contents, may be displayed simultaneously on the screen, as shown in Fig. 30.

[0103] Fig. 30 is a diagram (9) showing an example of content displayed on the screen. In the example of Fig. 30, a video of a soccer game is displayed on the look-up screen 33. An application execution screen is displayed on the upper front screen 32. An application launcher screen is displayed on the lower front screen 31. This allows one passenger to view content on the full screen while another passenger views different content on the front screen.

[0104] [4. Application to moving objects] The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.

[0105] FIG. 31 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.

[0106] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 31, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053 are illustrated.

[0107] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0108] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0109] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.

[0110] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0111] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.

[0112] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0113] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0114] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0115] The audio / video output unit 12052 transmits at least one output signal of audio and image to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 31, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

[0116] FIG. 32 is a diagram (1) showing an example of the installation position of the imaging unit.

[0117] In FIG. 32, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, 12105, and 12106.

[0118] The imaging units 12101, 12102, 12103, 12104, 12105, and 12106 are provided, for example, at the front nose, side mirrors, rear bumper, back door, upper part of the windshield in the vehicle interior, roof, etc. of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield in the vehicle interior mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the upper part of the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, etc. An imaging unit 12106 provided on the roof mainly captures images of the sky.

[0119] 32 shows an example of the imaging ranges of the imaging units 12101 to 12104. An imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and an imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.

[0120] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.

[0121] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the imaging ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.

[0122] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0123] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

[0124] Fig. 33 is a diagram (2) showing an example of the installation position of the imaging unit. As shown in Fig. 33, the imaging unit 12106 captures an image of the sky above the vehicle. The imaging unit 12106 may be a wide-angle camera or a 360° camera.

[0125] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the microcomputer 12051 in the configuration described above. Specifically, the processing of the display control unit 15 of the image processing device 10 is realized by the microcomputer 12051. In addition, the image of the sky above the automobile described in FIG. 8 and the like may be captured by the imaging unit 12106.

[0126] The automobile equipped with the image processing device may be connected to a network. FIG. 34 is a diagram showing an example of the configuration of a network system. As shown in FIG. 34, an automobile 100V equipped with an image processing device may be connected to another automobile 110V so as to be able to communicate data. The automobile 100V may also be connected to a server 100S via a network N so as to be able to communicate with the server 100S. The network N is, for example, the Internet. In this case, the image processing device provided in the automobile 100V can acquire content from the server 100S or the automobile 110V.

[0127] 1, 25, and 28 may be realized as one device. That is, as one embodiment, a device having functions similar to at least one of the image display device 40 and the seat setting device 50 in addition to functions similar to those of the image processing device 10 can be realized. For example, an image processing device having functions similar to those of the image processing device 10a and the seat setting device 50 in FIG. 25 can provide processing equivalent to that of the system including all the devices in FIG. 25.

[0128] 1, 25, and 28 may be provided in a server on a network outside the vehicle, so that the vehicle and the server can communicate with each other. In this case, the image display device 40 and the seat setting device 50 may be controlled by the server.

[0129] The present technology can also be configured as follows. (1) a determination unit that determines a shape of the screen suitable for displaying the content based on information regarding an automatic driving state of the vehicle and information regarding the content to be displayed on a screen provided in the vehicle; a creation unit that creates an image of the content to be displayed on the screen in accordance with the shape determined by the determination unit to be suitable for displaying the content; An image processing device comprising: (2) The image processing device according to (1), wherein the determination unit determines the shape of the screen based on an autonomous driving state including whether the automobile is autonomously driving or not and an autonomous driving level. (3) The image processing device described in any of (1) to (2), wherein the determination unit determines that, when the autonomous driving state is autonomous driving, a shape in which at least one of a plurality of screens is arranged on the windshield of the vehicle is suitable for displaying the content. (4) The image processing device described in any of (1) to (3), wherein the determination unit determines that a shape in which the screen is not placed on the windshield of the vehicle is suitable for displaying the content when the autonomous driving state is during manual driving. (5) The image processing device according to any one of (1) to (4), wherein the determination unit determines the shape of the screen suitable for displaying the content based on the autonomous driving state, information about the content, and the inclination of the seat back of the vehicle. (6) The image processing device described in any of (1) to (5), wherein the determination unit determines that a shape in which a portion of the screen is placed on the windshield of the vehicle is suitable for displaying the content when the backrest of the vehicle's seat is tilted to a degree unsuitable for driving. (7) the determining unit further determines an inclination of a backrest of the seat of the vehicle suitable for displaying the content; The image processing device according to any one of (1) to (4), further comprising a seat setting unit that sets the inclination of the backrest of the seat according to the inclination of the backrest of the seat that is determined by the determination unit to be suitable for displaying the content. (8) The image processing device according to any one of (1) to (7), wherein the creation unit creates an image to be displayed on a screen among a plurality of screens arranged on the windshield of the vehicle, the image indicating information for assisting driving. (9) The image processing device according to any one of (1) to (8), wherein the creation unit creates an image to be displayed on a first screen among a plurality of screens, the first screen being arranged on the ceiling of the vehicle, the image being taken of the sky above the vehicle, and an image to be displayed on a second screen arranged on the windshield of the vehicle, the image being taken of the rear of the vehicle. (10) The image processing device according to any one of (1) to (9), wherein the creation unit creates an image of a rear seat to be displayed on one of a plurality of screens, the screen being arranged on the windshield of the automobile. (11) The image processing device according to any one of (1) to (10), wherein the creation unit creates a first image to be displayed on a first screen among a plurality of screens, the first screen being arranged on the ceiling of the vehicle, and a second image to be displayed on a second screen arranged on the windshield of the vehicle, the second image displaying an explanatory text for the first image. (12) The image processing device according to any one of (1) to (11), wherein the creation unit creates an image of a running application for display on a first screen arranged on the ceiling of the vehicle, and an image of an application launcher for display on a second screen arranged on the windshield of the vehicle. (13) A display system having a movable screen mounted on a vehicle, a projector, and an image processing device, The image processing device includes: a determination unit that determines a shape of the screen suitable for displaying the content based on information about an automatic driving state of the vehicle and information about the content to be displayed on the screen; a creation unit that creates an image of the content to be displayed on the screen in accordance with the shape determined by the determination unit to be suitable for displaying the content; A display control device comprising: A display system having: (14) the determining unit further determines an inclination of a backrest of the seat of the vehicle suitable for displaying the content; The display system described in (13) further includes a seat setting device that sets the inclination of the seat backrest according to the inclination of the seat backrest determined by the determination unit to be suitable for displaying the content. (15) The computer determining a shape of the screen suitable for displaying the content based on information about an autonomous driving state of the vehicle and information about content to be displayed on a screen provided in the vehicle; Creating an image of the content to be displayed on the screen according to the shape determined to be suitable for displaying the content. Image processing methods. (16) Computer, a determination unit that determines a shape of the screen suitable for displaying the content based on information regarding an automatic driving state of the vehicle and information regarding the content to be displayed on a screen provided in the vehicle; a creation unit that creates an image of the content to be displayed on the screen in accordance with the shape determined by the determination unit to be suitable for displaying the content; A recording medium on which a program for functioning as a [Explanation of symbols]

[0130] 10, 10a, 10b Image processing device 11 Input section 12 Communications Department 13 Information Processing Section 14 Shape output section 15 Display control section 20 Projector 31 Lower front screen 32 Upper front screen 33 Lookup Screen 40 Image display device 50 Seat setting device 151 Content Acquisition Department 152 Shape determination section 153 Image Creation Department 154 Shape storage unit

Claims

1. A determination unit that determines a shape of the screen suitable for displaying content based on information regarding an autonomous driving state of the vehicle, including whether the vehicle is autonomous and the level of autonomous driving, and information regarding content to be displayed on a screen provided in the vehicle; and a creation unit that creates an image of the content to be displayed on the screen in accordance with the shape determined by the determination unit to be suitable for displaying the content; a seat setting unit that outputs information indicating the inclination of the backrest of the seat of the vehicle, The determination unit further determines an inclination of the backrest of the seat suitable for displaying the content, the seat setting unit outputs information indicating a tilt of the backrest of the seat according to the tilt of the backrest of the seat determined by the determination unit to be suitable for displaying the content. Image processing device.

2. The image processing device according to claim 1 , wherein the determination unit determines, when the autonomous driving state is autonomous driving, that a shape in which at least one of the multiple screens is arranged on the windshield of the vehicle is suitable for displaying the content.

3. The image processing device according to claim 1 or 2, wherein the determination unit determines that a shape in which the screen is not placed on a windshield of the vehicle is suitable for displaying the content when the autonomous driving state is a manual driving state.

4. The image processing device according to any one of claims 1 to 3, wherein the determination unit determines the shape of the screen suitable for displaying the content based on the autonomous driving state, information about the content, and the inclination of the backrest of the seat of the vehicle.

5. The image processing device according to any one of claims 1 to 4, wherein the creation unit creates an image to be displayed on a screen among a plurality of screens arranged on the windshield of the automobile, the image indicating information for assisting driving.

6. The image processing device according to any one of claims 1 to 5, wherein the creation unit creates an image for display on a first screen among a plurality of screens, the first screen being arranged on the ceiling of the vehicle, the image being taken of the sky above the vehicle, and an image for display on a second screen arranged on the windshield of the vehicle, the image being taken of the rear of the vehicle.

7. The image processing device according to any one of claims 1 to 6, wherein the creation unit creates an image of a rear seat to be displayed on one of a plurality of screens, the screen being arranged on the windshield of the automobile.

8. The image processing device according to any one of claims 1 to 7, wherein the creation unit creates a first image for display on a first screen among a plurality of screens, the first screen being arranged on the ceiling of the vehicle, and a second image for display on a second screen arranged on the windshield of the vehicle, the second image displaying an explanatory text of the first image.

9. The image processing device according to any one of claims 1 to 8, wherein the creation unit creates an image of a running application for display on a first screen positioned on the ceiling of the vehicle, and an image of an application launcher for display on a second screen positioned on the windshield of the vehicle.

10. A display system for an interior of an automobile, comprising: a determination unit that determines a shape of the screen suitable for displaying the content based on information regarding an autonomous driving state of the vehicle, including whether the vehicle is autonomous and an autonomous driving level, and information regarding the content to be displayed on the screen; a creation unit that creates an image of the content to be displayed on the screen in accordance with the shape determined by the determination unit to be suitable for displaying the content; a seat setting unit that outputs information indicating the inclination of the backrest of the seat of the vehicle, the determining unit further determines an inclination of a backrest of the seat of the vehicle suitable for displaying the content; the seat setting unit outputs information indicating a tilt of the backrest of the seat according to the tilt of the backrest of the seat determined by the determination unit to be suitable for displaying the content. Display system.

11. The computer determining a shape of the screen suitable for displaying the content based on information about an autonomous driving state of the vehicle, including whether the vehicle is autonomous and the autonomous driving level, and information about content to be displayed on a screen provided in the vehicle; creating an image of the content to be displayed on the screen in accordance with the shape determined to be suitable for displaying the content; determining a suitable inclination of the seat back of the vehicle for displaying the content; outputting information indicating a tilt of the backrest of the seat according to the tilt of the backrest of the seat determined to be suitable for displaying the content; Image processing methods.

12. Computer, a determination unit that determines a shape of the screen suitable for displaying the content based on an autonomous driving state of the vehicle, including whether the vehicle is autonomous and the autonomous driving level, and information regarding the content to be displayed on a screen provided in the vehicle; and a creation unit that creates an image of the content to be displayed on the screen in accordance with the shape determined by the determination unit to be suitable for displaying the content; a seat setting unit that outputs information indicating the inclination of the backrest of the seat of the vehicle; The determination unit further determines an inclination of the backrest of the seat suitable for displaying the content, the seat setting unit outputs information indicating a tilt of the backrest of the seat according to the tilt of the backrest of the seat determined by the determination unit to be suitable for displaying the content. program.

Images