Projector control device, projector control method, and program product

Abstract

An apparatus and method are provided for analyzing the state of vehicle occupants and, based on the analysis results, performing illumination and image output control of a projector on a per-vehicle roof screen partition area. A data processing unit is provided, configured to perform occupant state analysis and, based on the occupant analysis results, perform projector output control. The data processing unit performs projector control by outputting illumination light with a brightness determined according to the occupant analysis results, on a per-partition area of ​​the vehicle roof screen where projector output light is emitted. The data processing unit analyzes the occupant state based on images captured by a camera and outputs illumination light with a brightness determined according to the occupant state, on a per-partition area of ​​the screen above each occupant.

Description

Technical Field

[0001] This disclosure relates to a projector control device, projector control method, and procedure. More specifically, this disclosure relates to a projector control device, projector control method, and procedure for controlling light emitted from a projector to the roof of a vehicle. Background Technology

[0002] In recent years, the development of autonomous vehicles has accelerated, and it is predicted that fully autonomous vehicles that do not require human operation will be realized in the future.

[0003] In the case of conventional vehicles that require driver operation, brightening the interior space would interfere with the driver's driving environment and cause trouble for oncoming vehicles, so it is not permissible to brighten the interior lighting.

[0004] However, in autonomous vehicles, it is predicted that these limitations will be eliminated. When the interior space can be made bright and freely usable, users as occupants will want to use it as if it were their own room.

[0005] Note that, as a conventional technology that discloses the configuration of lighting devices that can be used in vehicles, there is, for example, Patent Document 1 (Japanese Patent Application Publication No. 2008-132807).

[0006] Patent document 1 discloses a combined lighting device that can be used in a vehicle.

[0007] For example, when there are multiple occupants in an autonomous vehicle, each occupant is expected to take different actions according to their preferences, such as sleeping, reading, playing games, or watching movies.

[0008] However, no configuration for lighting control and video projection suitable for each occupant has yet been proposed.

[0009] Citation List

[0010] Patent documents

[0011] Patent Document 1: Japanese Patent Application Publication No. 2008-132807

[0012] Patent Document 2: International Publication No. 2017 / 104447 Summary of the Invention

[0013] The problem to be solved by the present invention

[0014] For example, this disclosure arose in view of the above-mentioned problems, and its object is to provide a projector control device, projector control method and procedure capable of performing optimal lighting control and video projection in accordance with the actions or states of each occupant in the vehicle.

[0015] Solution to the problem

[0016] The first aspect of this disclosure is a projector control device, comprising:

[0017] The data processing unit is configured to perform occupant status analysis of the vehicle and execute output control of the projector according to the occupant analysis results.

[0018] The data processing unit controls the projector to output illumination light with brightness determined according to the occupant analysis results, using the screen of the vehicle roof on which the projector outputs light as a unit, in sections.

[0019] Furthermore, a second aspect of this disclosure is a projector control method executed in a projector control device, the projector control method comprising, via a data processing unit:

[0020] The occupant status analysis steps for performing occupant status analysis in a vehicle; and

[0021] The projector control steps, which execute the projector output control based on the occupant analysis results, include...

[0022] In the projector control steps,

[0023] Projector control is performed on a partitioned area of ​​the screen on the vehicle roof where the projector outputs light, and the projector outputs illumination light with a brightness determined according to the occupant analysis results.

[0024] Furthermore, a third aspect of this disclosure is a program that causes a projector control device to perform projector control, the program causing a data processing unit to execute:

[0025] The occupant state analysis steps for controlling the execution of occupant state analysis in the vehicle; and

[0026] The projector control steps, which execute projector output control based on occupant analysis results, include...

[0027] In the projector control steps,

[0028] Projector control is performed on a partitioned area of ​​the screen on the vehicle roof where the projector outputs light, and the projector outputs illumination light with a brightness determined according to the occupant analysis results.

[0029] Note that the programs disclosed herein can be provided, for example, through a storage medium or communication medium that provides various program codes in a computer-readable format to an information processing apparatus or computer system capable of executing the program codes. By providing such programs in a computer-readable format, processing corresponding to the programs is implemented on the information processing apparatus or computer system.

[0030] Other objects, features, and advantages of this disclosure will become apparent from the more detailed description based on the embodiments and accompanying drawings described below. Note that the systems described in this specification are logical collections of multiple devices and are not limited to systems in which each type of device is housed within the same housing.

[0031] One embodiment of this disclosure comprises an apparatus and method for analyzing the state of vehicle occupants and, based on the analysis results, performing illumination and image output control of a projector on a per-vehicle roof screen partition basis.

[0032] Specifically, for example, a data processing unit is provided, configured to perform occupant status analysis of a vehicle and execute projector output control according to the occupant analysis results. The data processing unit performs projector control by outputting illumination light with brightness determined according to the occupant analysis results, on a unit of partitioned areas of a screen on the vehicle roof onto which the projector outputs light. The data processing unit analyzes the occupant status based on images captured by a camera and outputs illumination light with brightness determined according to the occupant status, on a unit of screen partitioned areas above each occupant.

[0033] This configuration enables the analysis of the vehicle occupants' status and, based on the analysis results, the control of the projector's lighting and image output in units of screen partitions on the vehicle roof.

[0034] Note that the effects described in this instruction manual are merely examples and are not limited; other effects may exist. Attached Figure Description

[0035] Figure 1 This is an illustration of an example of a vehicle equipped with the projector control device disclosed herein.

[0036] Figure 2 This is an illustration of an example of a vehicle equipped with the projector control device disclosed herein.

[0037] Figure 3 These are illustrations showing multiple examples of different states of vehicle occupants.

[0038] Figure 4 These are illustrations showing multiple examples of different states of vehicle occupants.

[0039] Figure 5 It is a diagram illustrating the acquisition of occupant observation information using cameras or distance sensors.

[0040] Figure 6 It is a diagram illustrating the acquisition of occupant observation information using cameras or distance sensors.

[0041] Figure 7 This is a diagram illustrating a specific example of projector and screen control based on the analysis results of the state or actions of each vehicle occupant.

[0042] Figure 8 This is a diagram illustrating a specific example of projector and screen control based on the analysis results of the state or actions of each vehicle occupant.

[0043] Figure 9 This is a diagram illustrating a specific example of projector and screen control based on the analysis results of the state or actions of each vehicle occupant.

[0044] Figure 10 This is a diagram illustrating a specific example of projector and screen control based on the analysis results of the state or actions of each vehicle occupant.

[0045] Figure 11 This is a diagram illustrating a specific example of projector and screen control based on the analysis results of the state or actions of each vehicle occupant.

[0046] Figure 12 This is a diagram illustrating an example of the configuration of the projector control device of this disclosure.

[0047] Figure 13 This is a diagram illustrating specific examples of past log data.

[0048] Figure 14 This is a flowchart illustrating the processing sequence executed by the projector control device of this disclosure.

[0049] Figure 15 This is a diagram illustrating an example of an algorithm for determining occupant status based on images captured by a camera.

[0050] Figure 16 This is a diagram illustrating an example of a specific control mode based on the occupant status.

[0051] Figure 17 This is a diagram illustrating an example of a specific control mode based on the occupant status.

[0052] Figure 18 This is a flowchart illustrating the processing sequence executed by the projector control device of this disclosure.

[0053] Figure 19 This is a flowchart illustrating the processing sequence executed by the projector control device of this disclosure.

[0054] Figure 20 This is a diagram illustrating an example of the hardware configuration of the projector control device of this disclosure. Detailed Implementation

[0055] The details of the projector control device, projector control method, and procedure of this disclosure will be described below with reference to the accompanying drawings. Note that the description will be carried out according to the following items.

[0056] 1. Overview of the projector control device disclosed herein

[0057] 2. Analysis and processing of the actions and states of occupants inside the vehicle.

[0058] 3. Specific examples of occupant analysis and projector control

[0059] 4. Examples of the configuration of the projector control device disclosed herein

[0060] 5. Processing sequence executed by the projector control device of this disclosure

[0061] 6. Examples of the hardware configuration of the projector control device disclosed herein

[0062] 7. Summary of the composition of this disclosure

[0063] [1. Overview of the projector control device disclosed herein]

[0064] First, an overview of the projector control device according to this disclosure will be given.

[0065] As mentioned above, in autonomous vehicles, it is predicted that the interior space will be brighter, and users as occupants will want to use the interior space as if it were their own room.

[0066] However, in a vehicle with multiple occupants, each occupant is expected to want to do different things according to their preferences, such as sleeping, reading, playing games, or watching movies.

[0067] The projector control device disclosed herein is capable of lighting control and image (still image and moving image) projection control suitable for such actions of each occupant.

[0068] The projector control device according to this disclosure uses the vehicle ceiling as a screen, outputs emitted light from the projector to the ceiling (screen), and uses reflected light from the ceiling (screen) for illumination. Furthermore, it projects images onto the ceiling (screen).

[0069] Furthermore, the projector control device disclosed herein allocates the actions or states of each occupant in the vehicle, and provides the optimal lighting environment and images (still images or moving images) for each occupant based on the analysis results.

[0070] That is, using the divided areas of the car's interior ceiling (screen) as units, different lighting with different brightness and different image outputs are implemented.

[0071] Note that the captured images from the camera and the position detection information from the distance sensor, etc., are used for the analysis and processing of the actions or states of each occupant inside the vehicle. Furthermore, the projector control device according to this disclosure not only acquires and analyzes temporary information such as facial information and position, but also acquires and analyzes temporal motion information.

[0072] Figure 1 This is an illustration illustrating an example of a vehicle equipped with the projector control device of this disclosure.

[0073] Figure 1 The diagrams illustrate the vehicle's (a) top view and (b) side view. Note that both diagrams are illustrated as sectional views.

[0074] Figure 1 The example shown in the diagram is an example of a projector 11 placed behind the rear seat of a vehicle.

[0075] Projector 11 projects light toward the roof of the vehicle. The roof is configured as screen 12.

[0076] Note that the projector 11 can output simple illumination light (white light, etc.) or it can output images (still images or moving images). When the projector 11 outputs illumination light, the reflected light from the screen 12 is used for illumination.

[0077] In addition, the projector 11 outputs illumination light and images on a per-view basis (in units of screen partitions) for each occupant of the vehicle. Specific examples of these will be described later.

[0078] Figure 1 The illustration shows an example using a single projector 11, but it is also possible to use multiple projectors.

[0079] Figure 2 This diagram illustrates an example of using two projectors.

[0080] also, Figure 2 The configuration shown in the diagram is such that one projector 11a is placed behind the rear seat of the vehicle, and the other projector 11b is placed behind the front seat of the vehicle.

[0081] The projector 11a on the back of the rear seat mainly outputs illumination and images to the rear area of ​​the vehicle on the screen 12 on the vehicle ceiling, which is the observation area for the occupants sitting in the rear seat.

[0082] On the other hand, the projector 11b on the back of the front seat mainly outputs illumination and images to the front area of ​​the vehicle on the screen 12 on the vehicle roof, which is the observation area of ​​the occupant sitting in the front seat.

[0083] Note that in autonomous vehicles, not only are forward-facing seats as in conventional vehicles envisioned, but also seat configurations facing each other are also envisioned.

[0084] As the installation location, one or more projectors 11 are appropriately installed in the optimal position according to the seating configuration.

[0085] In addition, the output angle of the output light from the projector 11 is also set to the optimal angle according to the installation position of the projector.

[0086] Note that when an image is projected from an oblique angle, the image on the screen becomes distorted. A correction process to eliminate this image distortion is performed inside the projector 11 before the image is projected. This correction process enables the display of a distortion-free image.

[0087] Note that such image distortion correction processing is described, for example, in Patent Document 2 (International Publication No. 2017 / 104447).

[0088] The projector 11 according to this disclosure also projects images, for example, by performing such existing correction processing internally. This correction processing enables the display of images without distortion.

[0089] For reference Figure 1 and Figure 2 As mentioned above, it is possible to configure the system with only one projector 11 or to configure the system with multiple projectors 11.

[0090] As screen 12, a general screen can also be used. In addition, for example, a static functional screen that utilizes the optical properties of Fresnel lenses, reflective beads, etc., and has specific reflectivity characteristics can be used.

[0091] In addition, dynamic functional screens that can adjust the characteristics of the screen through electrical control can be used, such as light-controlled thin film (LCF) or polymer-dispersed liquid crystal (PDLC).

[0092] When using this dynamic functional screen, the characteristics of the control screen 12 are processed according to the output light of the projector 11.

[0093] Specifically, for example, by using a camera to identify the user's condition, control can be performed to enhance the lighting function by controlling the characteristics of the screen according to the user's movement, and control can be performed to obtain an image that is easily visible from the user's position.

[0094] [2. Analysis and processing of the actions and states of the occupants inside the vehicle]

[0095] Next, we will explain the analysis and processing of the actions and states of the occupants inside the vehicle.

[0096] As described above, the projector control device according to this disclosure analyzes the actions or states of each occupant in the vehicle and, based on the analysis results, provides the optimal lighting environment for each occupant and provides image (still image or moving image) processing.

[0097] That is, by dividing the ceiling (screen) inside the vehicle into zones (regions), it performs the output of lighting with different brightness and different images (still images and moving images).

[0098] In order to produce lighting or image outputs that correspond to the state or actions of the occupants, it is necessary to analyze the actions or states of the occupants inside the vehicle.

[0099] The actions and states of occupants inside a vehicle are diverse. When there are multiple occupants in a vehicle, each occupant is in a different state, such as engaging in different activities, like sleeping, reading, playing games, or watching movies.

[0100] Figure 3 and Figure 4 The illustrations show several examples of different states of vehicle occupants.

[0101] Figure 3 (1) The diagram illustrates that there are a total of 4 occupants, 2 in the front seat and 2 in the back seat, and all occupants are looking ahead.

[0102] Figure 3 (2) The diagram illustrates the state of two occupants in the front seat of the vehicle, with both occupants looking ahead.

[0103] Figure 3 (3) The diagram illustrates a situation where there is one occupant in the right front seat of the vehicle and the occupant is looking forward.

[0104] exist Figure 4 In (4), there are a total of 4 occupants: 2 in the front seats and 2 in the back seats. The occupant in the right front seat is looking ahead. The occupant in the left front seat is dozing off. The two occupants in the back seats are talking.

[0105] exist Figure 4In (5), one passenger is in the front right seat of the vehicle, and two passengers are in the rear seats. The passenger in the front right seat is looking ahead. The two passengers in the rear seats are looking at the screen (ceiling).

[0106] exist Figure 4 In (6), there are 2 occupants in the front seat of the vehicle and 1 occupant in the right rear seat. The occupant in the right front seat is reading a book. The occupant in the left front seat and the occupant in the right rear seat are talking.

[0107] As mentioned above, the actions and states of the occupants inside the vehicle are diverse.

[0108] In order to analyze these actions or states of the individual occupants of the vehicle, the projector control device of this disclosure acquires the occupant observation information by using a camera and a distance sensor.

[0109] refer to Figure 5 The accompanying figures will illustrate the composition of the observational information of the occupants obtained through the use of cameras and distance sensors.

[0110] Figure 5 The illustration shows an example of the installation of the camera 21 and the distance sensor 22 in the projector control device of this disclosure.

[0111] Figure 5 The configuration shown in the diagram is a configuration in which multiple devices, including a camera 21 and a distance sensor 22, are installed in front of each occupant's seat.

[0112] That is, the device, which integrates a camera 21 and a distance sensor 22, is installed in four locations: in front of the right front seat, in front of the left front seat, in front of the right rear seat, and in front of the left rear seat.

[0113] The image and distance information acquired through these four devices are input into the data processing unit of the projector control device.

[0114] The data processing unit analyzes whether there are occupants in each seat. In addition, the data processing unit analyzes the temporal images (motion images) of the camera 21 and the temporal detection information of the distance sensor 22 in order to perform action analysis and processing for each occupant, such as "looking forward", "sleeping", "reading", "playing games" or "looking at the screen (ceiling) (watching movies)", etc.

[0115] The projector control unit executes output control of the projector based on the analysis results of each occupant in the data processing unit. Furthermore, in the case of a dynamic functional screen whose characteristics can be adjusted via electrical control, such as a light-controlled thin film (LCF) or polymer-dispersed liquid crystal (PDLC), screen control is also executed.

[0116] Note that the arrangement of camera 21 and distance sensor 22 is not limited to... Figure 5 The structure shown in the diagram, for example, is as follows: Figure 6 As shown in the diagram, it is also possible to install cameras diagonally on each seat.

[0117] Furthermore, the number of devices does not have to be set to correspond to the number of individual seats. For example, a device consisting of two devices can be used, including: a device that includes a camera and a distance sensor to measure the images and distances of both front seats; and a device that includes a camera and a distance sensor to measure the images and distances of the entire rear seat.

[0118] Alternatively, a configuration could be adopted in which an omnidirectional camera and a distance sensor are mounted on the roof of the vehicle.

[0119] [3. Specific examples of occupant analysis and projector control]

[0120] Next, specific examples of occupant analysis and projector control will be explained.

[0121] As described above, the projector control device of this disclosure analyzes the status and actions of each vehicle occupant based on images captured by a camera and information acquired by a distance sensor.

[0122] Furthermore, based on the analysis results, output control of the projector is executed, and screen control is also executed if the screen is a dynamic functional screen with adjustable characteristics.

[0123] The following will illustrate several specific examples (control examples 1-5) of controlling projectors and screens based on the analysis results of the states and actions of each vehicle occupant.

[0124] (Control Example 1)

[0125] First, refer to Figure 7 Example 1 illustrates the control mechanism.

[0126] Figure 7 The diagrams illustrate the following figures.

[0127] (a) Interior condition of the vehicle

[0128] (b) Processing sequence (processing performed by the data processing unit of the projector control device)

[0129] (c) Screen (Ceiling) (= Projector output status)

[0130] Figure 7 (a) The vehicle interior status indicates that there are occupants in the left and right front seats and the left and right rear seats, and that the faces of each occupant are facing each other.

[0131] Note that the status of each occupant can be analyzed using images captured by camera 21 and detection information from distance sensor 22.

[0132] Figure 7 (b) The processing sequence diagram illustrates the processing sequence executed by the data processing unit of the projector control device.

[0133] The process for each step will be explained.

[0134] (Step S11)

[0135] First, the data processing unit of the projector control device performs occupant analysis processing in step S11.

[0136] This processing is performed by inputting the captured image from camera 21 and the detection information from distance sensor 22.

[0137] The data processing unit of the projector control device generates data through the occupant analysis process in step S11. Figure 7 The “Crew Analysis Result 51” is illustrated in the (b) processing sequence.

[0138] "Occupant Analysis Result 51" includes the following data.

[0139] Occupants = 4 people in all seats

[0140] Status = 4 crew members are talking

[0141] Note that "Occupant Analysis Result 51" also includes the position data of each occupant based on the detection information from distance sensor 22.

[0142] (Step S12)

[0143] Next, in step S12, the data processing unit of the projector control device generates control data for the projector 11 based on the "occupant analysis result 51" generated in step S11. Note that if the screen (ceiling) 12 is a dynamic functional screen whose characteristics can be adjusted by electrical control, such as a light-controlled thin film (LCF) or polymer-dispersed liquid crystal (PDLC), screen control data is also generated.

[0144] Based on the "occupant analysis result 51" generated as a result of the occupant analysis processing in step S11, the data processing unit of the projector control device generates "projector (& screen) control data 52" which is optimal for setting the lighting environment or outputting images for the current state of the occupants.

[0145] exist Figure 7In the example shown in the diagram, the data processing unit of the projector control device generates the following "Projector (& Screen) Control Data 52".

[0146] Projector (& screen) control data 52 = High brightness illumination output across the entire screen area.

[0147] The data processing unit of the projector control device generates "projector (& screen) control data 52" as described above, and outputs the generated control data to the projector control unit and the screen control unit.

[0148] Based on the control data generated by the data processing unit, specifically, based on the following control data

[0149] Projector (& screen) control data 52 = High brightness illumination output across the entire screen area.

[0150] The projector control unit and the screen control unit control the projector 11 and the screen 12.

[0151] As a result of this control, the screen (ceiling) 12 is set as follows: Figure 7 The state shown in the diagram in (c).

[0152] Specifically, the projector 11 outputs illumination light (white light, etc.), which, through the illumination light (white light, etc.) reflected by the screen (ceiling) 12, brightly illuminates the entire interior of the vehicle.

[0153] Notice, Figure 7 The illumination output range of the projector 11 of the screen 12 shown in the diagram (c) is adjusted according to the position of each occupant analyzed based on the detection information of the distance sensor 22.

[0154] (Control Example 2)

[0155] Next, we will refer to Figure 8 Example 2 illustrates the control mechanism.

[0156] Similar to Figure 7 , Figure 8 The diagrams illustrate the following figures.

[0157] (a) Interior condition of the vehicle

[0158] (b) Processing sequence (processing performed by the data processing unit of the projector control device)

[0159] (c) Screen (Ceiling) (= Projector output status)

[0160] Figure 8 (a) The interior of the vehicle is such that there are occupants in the left and right front seats and the left and right rear seats, with the two occupants in the front seats having their eyes closed and the two occupants in the rear seats facing each other.

[0161] In these occupant states, the state of each occupant can be analyzed by using the captured images from camera 21 and the detection information from distance sensor 22.

[0162] Figure 8 (b) Processing sequence diagram illustrates the processing sequence executed by the data processing unit of the projector control device.

[0163] The process for each step will be explained.

[0164] (Step S11)

[0165] In step S11, the data processing unit of the projector control device inputs the captured image from the camera 21 and the detection information from the distance sensor 22, and performs occupant analysis processing.

[0166] The data processing unit generates data through occupant analysis. Figure 8 (b) Process the “Crew Analysis Result 51” shown in the diagram in the sequence.

[0167] "Occupant Analysis Result 51" includes the following data.

[0168] Occupants = 4 people in all seats

[0169] Situation: The two people in the front row are dozing off, and the two people in the back row are talking.

[0170] (Step S12)

[0171] Next, in step S12, the data processing unit of the projector control device generates control data for the projector 11 based on the "occupant analysis result 51" generated in step S11. Note that if the screen (ceiling) 12 is a dynamic functional screen, screen control data is also generated.

[0172] exist Figure 8 In the example shown in the diagram, the data processing unit of the projector control device generates the following "Projector (& Screen) Control Data 52".

[0173] Projector (& screen) control data 52 = Low brightness lighting output for the front row and high brightness lighting output for the rear row.

[0174] The data processing unit of the projector control device generates "projector (& screen) control data 52" as described above, and outputs the generated control data to the projector control unit and the screen control unit.

[0175] Based on the control data generated by the data processing unit, specifically, based on the following control data

[0176] Projector (& screen) control data 52 = Low brightness lighting output for the front row and high brightness lighting output for the rear row.

[0177] The projector control unit and the screen control unit control the projector 11 and the screen 12.

[0178] As a result of this control, the screen (ceiling) 12 is set as follows: Figure 8 The state shown in the diagram in (c).

[0179] That is, the front area of ​​the screen 12, which is located above the front seats of the vehicle, is set to a dim low-brightness lighting (white light, etc.) output state, while the rear area of ​​the screen 12, which is located above the rear seats of the vehicle, is set to a bright high-brightness lighting (white light, etc.) output state.

[0180] This process sets the space for occupants dozing in the front seat to be dimly lit, while the space for occupants talking in the back seat to be brightly lit.

[0181] Notice, Figure 8 The partitioned areas of screen 12 shown in diagram (c) are adjusted according to the positions of each occupant analyzed based on the detection information from distance sensor 22. This is similarly applied to the output range of the output light from projector 11.

[0182] (Control Example 3)

[0183] Next, we will refer to Figure 9 Example 3 illustrates the control mechanism.

[0184] Similar to Figure 7 , Figure 9 The diagrams illustrate the following figures.

[0185] (a) Interior condition of the vehicle

[0186] (b) Processing sequence (processing performed by the data processing unit of the projector control device)

[0187] (c) Screen (Ceiling) (= Projector output status)

[0188] exist Figure 9 (a) Inside the vehicle, there are occupants in the left and right front seats, and the occupant in the right front seat is reading. In addition, the occupant in the left front seat has his eyes closed.

[0189] Explanation Figure 9 (b) illustrates the processing sequence executed by the data processing unit of the projector control device.

[0190] The process for each step will be explained.

[0191] (Step S11)

[0192] In step S11, the data processing unit of the projector control device inputs the captured image from the camera 21 and the detection information from the distance sensor 22, and performs occupant analysis processing.

[0193] The data processing unit generates data through occupant analysis. Figure 9 (b) Process the “Crew Analysis Result 51” shown in the diagram in the sequence.

[0194] "Occupant Analysis Result 51" includes the following data.

[0195] Passengers = 2 people in the front seat

[0196] Situation = The passenger in the right front seat is reading, while the passenger in the left front seat is dozing off.

[0197] (Step S12)

[0198] Next, in step S12, the data processing unit of the projector control device generates control data for the projector 11 based on the "occupant analysis result 51" generated in step S11. Note that if the screen (ceiling) 12 is a dynamic functional screen, screen control data is also generated.

[0199] exist Figure 9 In the example shown in the diagram, the data processing unit of the projector control device generates the following "Projector (& Screen) Control Data 52".

[0200] Projector (& screen) control data 52 = High brightness in the front right area, and low brightness lighting output in the front left area and rear area.

[0201] The data processing unit of the projector control device generates "projector (& screen) control data 52" as described above, and outputs the generated control data to the projector control unit and the screen control unit.

[0202] Based on the control data generated by the data processing unit, specifically, based on the following control data

[0203] Projector (& screen) control data 52 = High brightness in the front right area, and low brightness lighting output in the front left area and rear area.

[0204] The projector control unit and the screen control unit control the projector 11 and the screen 12.

[0205] As a result of this control, the screen (ceiling) 12 is set as follows: Figure 9 The state shown in the diagram in (c).

[0206] That is, the front right side area of ​​the screen 12, which is located above the front seats of the vehicle, is set to a bright high-brightness illumination light (white light, etc.) output state, while the front left side area of ​​the vehicle and the rear area of ​​the screen 12, which is located above the rear seats, are set to a dim low-brightness illumination light (white light, etc.) output state.

[0207] This process sets the space in the right front seat where the passenger is reading to be bright, while the space in the left front seat where the passenger is dozing off and the rear seats where there are no passengers are set to be dark.

[0208] Note, for example, that the projector 11 and screen 12 can be configured so that light is concentrated on the area of ​​the book in the space of the passenger reading in the right front seat of the vehicle.

[0209] Note that when the screen (canopy) 12 is a dynamic functional screen whose characteristics can be adjusted by electrical control, such as a light-controlled thin film (LCF) or polymer-dispersed liquid crystal (PDLC), it is also possible to control the screen to set the light to illuminate only a specific selected area.

[0210] Notice, Figure 9 The partitioned areas of screen 12 shown in diagram (c) are adjusted according to the positions of each occupant analyzed based on the detection information from distance sensor 22. This is similarly applied to the output range of the output light from projector 11.

[0211] (Control Example 4)

[0212] Next, we will refer to Figure 10 Example 4 illustrates the control mechanism.

[0213] Similar to Figure 7 , Figure 10 The diagrams illustrate the following figures.

[0214] (a) Interior condition of the vehicle

[0215] (b) Processing sequence (processing performed by the data processing unit of the projector control device)

[0216] (c) Screen (Ceiling) (= Projector output status)

[0217] exist Figure 10 (a) In the vehicle interior, there are four occupants in the left and right front seats and the left and right rear seats, and the occupant in the right front seat is reading. In addition, the occupant in the left front seat has his eyes closed, and the two occupants in the rear left and right seats are looking at the screen (ceiling) 12.

[0218] This will be explained in Figure 10 (b) illustrates the processing sequence executed by the data processing unit of the projector control device.

[0219] The process for each step will be explained.

[0220] (Step S11)

[0221] In step S11, the data processing unit of the projector control device inputs the captured image from the camera 21 and the detection information from the distance sensor 22, and performs occupant analysis processing.

[0222] The data processing unit generates data through the occupant analysis process. Figure 10 (b) Process the “Crew Analysis Result 51” shown in the diagram in the sequence.

[0223] "Occupant Analysis Result 51" includes the following data.

[0224] Occupants = 4 people in all seats

[0225] Situation = The occupant in the right front seat is reading, the occupant in the left front seat is dozing off, and the occupant assigned to the rear seat is looking at the screen.

[0226] (Step S12)

[0227] Next, in step S12, the data processing unit of the projector control device generates control data for the projector 11 based on the "occupant analysis result 51" generated in step S11. Note that if the screen (ceiling) 12 is a dynamic functional screen, screen control data is also generated.

[0228] exist Figure 9 In the example shown in the diagram, the data processing unit of the projector control device generates the following "Projector (& Screen) Control Data 52".

[0229] Projector (& screen) control data 52 = High-brightness lighting for the front right side area, low-brightness lighting for the front left side area, and image output for the rear area.

[0230] The data processing unit of the projector control device generates "projector (& screen) control data 52" as described above, and outputs the generated control data to the projector control unit and the screen control unit.

[0231] Based on the control data generated by the data processing unit, specifically, based on the following control data

[0232] Projector (& screen) control data 52 = High-brightness lighting for the front right side area, low-brightness lighting for the front left side area, and image output for the rear area.

[0233] The projector control unit and the screen control unit control the projector 11 and the screen 12.

[0234] As a result of this control, the screen (ceiling) 12 is set as follows: Figure 10 The state shown in the diagram in (c).

[0235] That is, the right front area of ​​the screen 12, which is located above the front seats of the vehicle, is set to a bright high-brightness illumination light (white light, etc.) output state, the left front area of ​​the vehicle is set to a dim low-brightness illumination light (white light, etc.) output state, and images, such as motion picture content, are output to the rear area of ​​the screen 12, which is located above the rear seats.

[0236] Note that the motion graphics content to be output to the rear area can be freely selected through user input to the projector control device.

[0237] Through this process, the space in the right front seat where the occupant is reading is set to be bright, the space in the left front seat where the occupant is dozing off is set to be dark, and images, such as motion graphics, are displayed on the rear seat side where the occupant is looking at screen 12.

[0238] Notice, Figure 10 The partitioned areas of screen 12 shown in diagram (c) are adjusted according to the positions of each occupant analyzed based on the detection information from distance sensor 22. This similarly applies to the output range of the image and illumination light of projector 11.

[0239] (Control Example 5)

[0240] Next, we will refer to Figure 11 Example 5 illustrates the control mechanism.

[0241] Similar to Figure 7 , Figure 11 The diagrams illustrate the following figures.

[0242] (a) Interior condition of the vehicle

[0243] (b) Processing sequence (processing performed by the data processing unit of the projector control device)

[0244] (c) Screen (Ceiling) (= Projector output status)

[0245] exist Figure 11 (a) In the vehicle interior, there are four occupants in the left and right front seats and the left and right rear seats, and the occupant in the right front seat is reading. In addition, the occupant in the left front seat and the two occupants in the rear left and right seats are looking at the screen (ceiling) 12.

[0246] This will be explained in Figure 11 (b) illustrates the processing sequence executed by the data processing unit of the projector control device.

[0247] The process for each step will be explained.

[0248] (Step S11)

[0249] In step S11, the data processing unit of the projector control device inputs the captured image from the camera 21 and the detection information from the distance sensor 22, and performs occupant analysis processing.

[0250] The data processing unit generates data through the occupant analysis process. Figure 11 (b) Process the “Crew Analysis Result 51” shown in the diagram in the sequence.

[0251] "Occupant Analysis Result 51" includes the following data.

[0252] Occupants = 4 people in all seats

[0253] Status: The occupant in the right front seat is reading, while the occupant in the left front seat and the occupant assigned to the rear seats are looking at the screen.

[0254] (Step S12)

[0255] Next, in step S12, the data processing unit of the projector control device generates control data for the projector 11 based on the "occupant analysis result 51" generated in step S11. Note that if the screen (ceiling) 12 is a dynamic functional screen, screen control data is also generated.

[0256] exist Figure 9 In the example shown in the diagram, the data processing unit of the projector control device generates the following "Projector (& Screen) Control Data 52".

[0257] Projector (& screen) control data 52 = High-brightness lighting for the front right side area, and image output for the front left side area and rear area.

[0258] The data processing unit of the projector control device generates "projector (& screen) control data 52" as described above, and outputs the generated control data to the projector control unit and the screen control unit.

[0259] Based on the control data generated by the data processing unit, specifically, based on the following control data

[0260] Projector (& screen) control data 52 = High-brightness lighting for the front right side area, and image output for the front left side area and rear area.

[0261] The projector control unit and the screen control unit control the projector 11 and the screen 12.

[0262] As a result of this control, the screen (ceiling) 12 is, for example, set as follows: Figure 11 The state shown in the diagram in (c).

[0263] That is, the front right side area of ​​the screen 12, which is located above the front seats of the vehicle, is set to a bright high-brightness illumination light (white light, etc.) output state, while images, such as motion picture content, are output to the front left side area of ​​the vehicle and the rear area of ​​the screen 12, which are located above the rear seats.

[0264] Note that in Figure 11 In the example shown in the diagram, the content displayed in the front left area of ​​screen 12 is different from the content displayed in the back area.

[0265] The displayed content can be set in units of partitioned areas of screen 12. That is, it can be freely set through user input to the projector control device.

[0266] Through this process, the space of the occupant reading in the right front seat of the vehicle is set to be bright, the occupant looking at screen 12 in the left front seat can view the moving image content A displayed in the front left area of ​​screen 12, and can also view the content B displayed in the rear area of ​​screen 12 on the rear seat side where the occupant is looking at screen 12.

[0267] Notice, Figure 11 The partitioned areas of screen 12 shown in diagram (c) are adjusted according to the positions of each occupant analyzed based on the detection information from distance sensor 22. This similarly applies to the output range of the image and illumination light of projector 11.

[0268] The above describes several specific examples of control performed by the projector control device of this disclosure.

[0269] These specific examples demonstrate that the projector control device of this disclosure analyzes the actions or states of the occupants of the vehicle and, based on the analysis results, performs lighting and image display on the screen 12 in units of partitioned areas of the screen 12.

[0270] Through these controls, occupants can set the lighting space or view content according to their individual actions or states.

[0271] [4. Example of the configuration of the projector control device disclosed herein]

[0272] Next, an example of the configuration of the projector control device disclosed herein will be described.

[0273] Figure 12 This is a diagram illustrating an example of the configuration of the projector control device 100 of this disclosure.

[0274] like Figure 12 As shown in the diagram, the projector control device 100 of this disclosure includes a camera 101, a distance sensor 102, a GPS 103, a data processing unit 104, a storage device 105, an input unit (operation unit) 106, a communication unit 107, a projector 108, and a screen 109.

[0275] The data processing unit 104 includes an occupant analysis unit 121, a position analysis unit 122, a control information generation unit 123, a projector control unit 124, and a screen control unit 125.

[0276] The communication unit 107 communicates with the external server 150 via a network.

[0277] Camera 101 captures images of the occupants inside the vehicle. It captures motion images as temporal image data of the occupants.

[0278] Distance sensor 102 continuously measures the distance to the occupants inside the vehicle to acquire distance observation data for analyzing the occupants' movements.

[0279] Note that the above reference is as follows. Figure 5 and Figure 6 The camera 101 and the distance sensor 102 are installed at a location that allows them to observe the movement and status of all occupants inside the vehicle.

[0280] This can be achieved by installing multiple cameras and distance sensors inside the vehicle, corresponding to the number of seats for each occupant, or by using a single camera to capture images of multiple occupants.

[0281] Note that the occupant's position and motion information detected by the distance sensor 102 is used not only for analyzing the occupant's actions or state, but also for determining the projection range, lighting range, or partitioning mode of the image. These determination processes are performed in the data processing unit 104.

[0282] GPS 103 communicates with GPS satellites to obtain the vehicle's current location. The acquired location information is input into the data processing unit 104.

[0283] Although not illustrated in the specific examples above, the projector control device 100 of this disclosure can also control the projector by using the current position of the vehicle.

[0284] This process will be described later. Specifically, for example, projector setting information from past logs for when the same location was previously accessed is retrieved, and the projector is controlled based on the retrieved past logs.

[0285] As illustrated in the figure, the data processing unit 104 includes an occupant analysis unit 121, a position analysis unit 122, and a control information generation unit 123.

[0286] The occupant analysis unit 121 analyzes the temporal images (video images) from the camera 101 and the temporal detection information from the distance sensor 102 in order to perform action analysis processing for each occupant based on their state and actions, such as "looking ahead", "sleeping", "reading", "playing games" or "looking at the screen (ceiling) (watching movies)", etc.

[0287] That is, execute the above reference. Figures 7-12 (Control Examples 1-5) describes the processing of step S11 of the processing sequence to generate occupant analysis results, which include data on the presence or absence of occupants in each seat and the actions or states of each occupant.

[0288] Furthermore, the position of each occupant is analyzed by analyzing the detection information from the distance sensor 102. This occupant position information is used to set the output range of the projector's output light and to set the partitioned areas of the screen.

[0289] Based on the occupant analysis results generated by the occupant analysis unit 121, the control information generation unit 123 generates control information for the projector 108 and the screen 109.

[0290] That is, execute the above reference. Figures 7-12 The processing of step S12 of the processing sequence illustrated in (control examples 1-5) is performed.

[0291] Based on the presence or absence of occupants in each seat and the data on the actions or states of each occupant, control information is generated for optimal lighting and image output in units of partitioned areas of screen 109.

[0292] The control information includes control information for the projector 108, and in the case where the screen 109 is a dynamic functional screen whose characteristics can be adjusted by electrical control, such as a light-controlled thin film (LCF) or a polymer-dispersed liquid crystal (PDLC), it also includes screen control information for adjusting the screen characteristics determined according to the occupant analysis results.

[0293] Note that, as described above, the control information generation unit 123 sets the output range of the projector 108 and the partitioned areas of the screen 109 according to the positions of each occupant analyzed by the occupant analysis unit 121 based on the detection information of the distance sensor 102.

[0294] The projector control information generated by the control information generation unit 123 is output to the projector control unit 124.

[0295] In addition, the screen control information generated by the control information generation unit 123 is output to the screen control unit 108.

[0296] The projector control unit 124 controls the projector 108 according to the control information input from the control information generation unit 123.

[0297] The screen control unit 125 controls the screen 109 according to the control information input from the control information generation unit 123.

[0298] The projector control unit 124 and the screen control unit 125 perform control processing for optimal lighting output and image output in units of partitioned areas of the screen 109 generated by the control information generation unit 123.

[0299] The projector control unit 124 controls the projector 108 to output images and illumination light with illuminance determined according to occupant analysis results, in units of partitioned areas of the screen 109 on the vehicle roof where the output light of the projector 108 is output.

[0300] The screen control unit 125 performs screen control to adjust the screen characteristics determined according to the occupant analysis results, on a unit basis, the screen 109 of the vehicle roof that outputs the output light of the projector 108 thereon.

[0301] The location analysis unit 122 in the data processing unit 104 analyzes the current location of the vehicle by using information acquired by the GPS 103.

[0302] The analyzed location data is input to the control information generation unit 123. For example, the control information generation unit 123 obtains the setting information of the projector and screen when the projector and screen passed the same position in the past from the past log, and generates control information for the projector and screen based on the obtained past log.

[0303] This process will be explained later.

[0304] The storage unit 105 records the program (application) to be processed in the data processing unit 104, map data, past log data, content such as motion images to be output by the projector 108, and user information such as occupant profiles and facial images.

[0305] The data processing unit 104 retrieves necessary data from the storage unit 105 as needed and processes it. For example, it can also perform comparison processing between the facial image contained in the user information stored in the storage unit 105 and the facial image contained in the captured image of the camera 101 to identify who the occupant is and to perform user-specific control corresponding to the designated occupant.

[0306] Note that the historical log data stored in storage unit 105 records past user settings, such as each user's preferred lighting and content. In addition, it also records operation information associated with map data.

[0307] Figure 13 The diagram illustrates a specific example of past log data stored in storage unit 105.

[0308] like Figure 13 As shown in the diagram, for example, the following data are recorded in association with the past log data stored in storage unit 105.

[0309] (A) Location (latitude, longitude)

[0310] (B) Time

[0311] (C) Crew Status

[0312] (D) Projector Settings & Screen Settings

[0313] (E) Output content

[0314] Based on past log data, when the data processing unit 104 passes through the same area as in the past, it can control the projector and screen with settings similar to those in the past.

[0315] Note that, not limited to storage unit 105, past log information can be stored, for example, on an external server 150.

[0316] The historical log information recorded in storage unit 105 or external server 150 is data obtained by accumulating user information in chronological order according to the vehicle's movement location using map and GPS information. It records the usage status of video and lighting functions, movement time, in-vehicle activities, location information, etc. Based on this log information, user preferences and the frequency of function usage can be understood.

[0317] For example, historical log information can indicate that there is a high level of conversation while looking at the scenery in a certain location (urban area). In this case, controls such as "automatically increasing the intensity of the lighting" can be implemented. Furthermore, in a certain area where passengers are frequently sleeping, optimization controls linked to map information, such as "pre-lowering the intensity of the lighting," can be implemented.

[0318] The input unit (operation unit) 106 is a user-operable input unit, and can, for example, set the lighting, select and specify the output content.

[0319] The communication unit 107 communicates with the external server 150 via a network such as the Internet.

[0320] Server 150 includes, for example, content provision servers, map information periodic servers, historical log accumulation servers, user profile registration servers, etc.

[0321] The projector control device 100 can obtain this data by using its own storage unit 105 or an external server 150.

[0322] [5. Processing sequence executed by the projector control device of this disclosure]

[0323] Next, the processing sequence executed by the projector control device 100 of this disclosure will be described.

[0324] refer to Figure 14 The flowcharts in the middle and later sections will illustrate the processing sequence executed by the projector control device 100 of this disclosure.

[0325] Note that, according to Figure 14 The processing of the intermediate and subsequent processes is performed, for example, according to the program stored in the storage unit of the projector control device 100, under the control of a control unit such as a CPU with program execution function.

[0326] The following will explain in turn the following steps. Figure 14 The diagram illustrates the processing steps of each step in the workflow.

[0327] (Step S101)

[0328] First, in step S101, the projector control device 100 acquires images from the camera and sensors, as well as sensor detection information.

[0329] (Step S102)

[0330] Next, in step S102, the projector control device 100 analyzes the images and sensor detection information acquired in step S101 to determine the status of each occupant.

[0331] The above has been referenced. Figures 7-11 This section provides a specific example of occupant status determination and processing.

[0332] Note that the occupant status determination process is in Figure 12 It is executed in the occupant analysis unit 121 of the data processing unit 104 shown in the diagram.

[0333] Figure 12 The occupant analysis unit 121 of the data processing unit 104 shown in the diagram analyzes the images and sensor detection information according to a pre-set algorithm to determine the status of each occupant.

[0334] Reference Figure 15 Provide examples of specific decision-making algorithms.

[0335] Figure 15 The example shown in the diagram illustrates an algorithm for determining occupant status based on images captured by a camera.

[0336] For example, based on images captured by a camera, in situations where one observes states such as "several people are moving their mouths, their faces facing each other," Figure 12 The occupant analysis unit 121 of the data processing unit 104 shown in the diagram generates an analysis result indicating "occupants are talking" as the occupant analysis result.

[0337] Furthermore, based on images captured by the camera, when observing states such as "the occupant is looking out the window for a period of time and remains still," the occupant analysis unit 121 generates an analysis result indicating "the occupant is looking out the window," which is then presented as the occupant analysis result.

[0338] Furthermore, based on images captured by a camera, in situations where one is observed to be "still for a period of time, with eyes closed and the face facing the same direction," Figure 12 The occupant analysis unit 121 of the data processing unit 104 shown in the diagram generates an analysis result indicating "occupant is asleep, drowsy, or about to fall asleep", which is used as the occupant analysis result.

[0339] As described above, in step S102, Figure 12 The occupant analysis unit 121 of the data processing unit 104 shown in the diagram is configured as follows: Figure 15 The diagram shows a pre-set judgment algorithm used to determine the status of each crew member.

[0340] Notice, Figure 12 The occupant analysis unit 121 of the data processing unit 104 shown in the diagram also analyzes the position of each occupant based on the detection information of the distance sensor 102.

[0341] This analysis information is used to set the output range of the projector 108 and to set the partitioned areas of the screen 109.

[0342] (Step S103)

[0343] Finally, in step S103, the projector control device 100 controls the projector 108 and the screen 109 based on the occupant analysis results indicating the occupant status determined in step S102.

[0344] Specifically, as referenced above Figures 7-11The control processing of lighting and content output is carried out in units of partitioned areas of screen 109, corresponding to the status of the occupants.

[0345] Note that the output range of the projector 108 and the partitioning of the screen 109 are also adjusted according to the position of each occupant based on the detection information analyzed by the distance sensor 102.

[0346] Note that this control mode can be changed via input processing by the user (occupant) through input unit 106.

[0347] For example, the content output from projector 108 can be individually set by each occupant. See also the above reference. Figure 11 The settings described above allow for the output of different content above each passenger's seat.

[0348] Figure 16 The diagram illustrates an example of a specific control mode corresponding to the occupant's state.

[0349] The projector control device 100 of this disclosure, for example, performs operations such as... Figure 16 The diagram shows the control processing corresponding to the occupant status.

[0350] Figure 16 The control example shown in the diagram is as follows.

[0351] (1) Crew = During conversation: Lighting setting = High brightness lighting, Content output = Stop

[0352] (2) Occupant = In sleeping mode: Lighting setting = Low brightness lighting, Content output = Stop

[0353] (3) Occupant = In reading mode: Lighting setting = High brightness lighting, Content output = Stop

[0354] (4) Crew = When playing games: Lighting settings = Medium brightness lighting, Content output = Stop

[0355] (5) Occupant = When looking straight ahead: Lighting setting = Medium brightness lighting, Content output = Stop

[0356] (6) Crew = While watching the screen: Lighting settings = Stop, Content output = Execute

[0357] Note that these control modes can be changed by the user via input unit 106.

[0358] Figure 17 This illustration shows an example of control changes made through user input.

[0359] Figure 17 The control example shown in the diagram is as follows.

[0360] (1) Crew = In conversation mode: Lighting settings = Stop, Content output = Execute

[0361] (2) Occupant = In sleeping mode: Lighting setting = Stop, Content output = Stop

[0362] (3) Occupant = In reading mode: Lighting setting = High brightness lighting, Content output = Stop

[0363] (4) Crew = When playing games: Lighting settings = Low brightness lighting, Content output = Stop

[0364] (5) Occupant = When looking straight ahead: Lighting setting = Stop, Content output = Stop

[0365] (6) Crew = While watching the screen: Lighting settings = Stop, Content output = Execute

[0366] As described above, the control mode executed according to the occupant status can be changed to various settings by the user via input unit 106.

[0367] Next, refer to Figure 18 The flowchart shown in the diagram illustrates the process according to the above reference. Figure 12 The GPS 103 acquires the vehicle's location information and processes it according to a sequence of control steps based on past log information stored in storage unit 105 or server 150.

[0368] That is, the projector setting information recorded in the past log information with the same or similar location information as the vehicle location information obtained by GPS 103 is obtained from the storage unit 105 or the server 150, and the projector is controlled based on the obtained past log.

[0369] This will be explained in Figure 18 The diagram illustrates the processing of each step in the workflow.

[0370] (Step S121)

[0371] First, in step S121, the projector control device 100 acquires location information and analyzes the current vehicle position.

[0372] This processing is handled by Figure 12The GPS 103 and the position analysis unit 122 of the data processing unit 104 of the projector control device 100 shown in the diagram are executed.

[0373] (Step S122)

[0374] Next, in step S122, the projector control device 100 acquires past log information corresponding to the current vehicle location.

[0375] This processing is handled by Figure 12 The control information generation unit 123 of the data processing unit 104 of the projector control device 100 shown in the diagram executes.

[0376] The control information generation unit 123 obtains past log information from the storage unit 105 or the server 150, which contains location information that is the same as or similar to the current vehicle location analyzed by the location analysis unit 122.

[0377] Note that the above reference is as follows. Figure 13 For example, the following data are recorded in relation to each other in past log information.

[0378] (A) Location (latitude, longitude)

[0379] (B) Time

[0380] (C) Crew Status

[0381] (D) Projector Settings & Screen Settings

[0382] (E) Output content

[0383] (Step S123)

[0384] Next, in step S123, based on the projector setting information and screen setting information recorded in the past log information corresponding to the current vehicle position obtained in step S122, the projector control device 100 performs control to implement the same settings as these setting information.

[0385] As described above, the past log information recorded in storage unit 105 or external server 150 is data obtained by accumulating user information in a time-series manner according to the vehicle's movement location using map information and GPS information. It records the usage status of video and lighting functions, movement time, in-vehicle activities, location information, etc. Based on this log information, user preferences and the frequency of function usage can be understood.

[0386] As per Figure 18The diagram illustrates a specific example of the control process. For instance, there is an example where information is obtained from past logs indicating that a person is actively talking while looking at the scenery outside in a certain location (urban area), and based on this past log information, the control function "automatically increases the intensity of the lighting function" when the vehicle passes through the urban area.

[0387] In addition, if a log indicates that there are many instances of passengers sleeping in a certain area, controls such as "pre-reducing the intensity of the lighting function" can be implemented.

[0388] Note that when using projectors and screen controls that utilize past log information, a configuration can be adopted that further considers the degree of matching between the current time and the time recorded in the past log information.

[0389] Furthermore, a configuration can be adopted in which the faces of occupants in the captured images contained in the camera 101 are compared with the facial images of users registered in the storage unit 105 or the server 150 to identify the occupants (users) currently in the vehicle, and only if the identified user is the user corresponding to the user ID recorded in the past log information, is the same control mode as that recorded in the past log information performed.

[0390] By implementing such controls, user-specific controls can be made according to the preferences of the designated user.

[0391] Past log information is applied to Figure 18 The control processing shown in the diagram above and the reference above Figure 14 The control processes described can be performed together based on the actions or states of the occupants.

[0392] Reference Figure 19 The flowchart shown in the diagram illustrates this processing sequence.

[0393] This will be explained in Figure 19 The diagram illustrates the processing of each step in the workflow.

[0394] (Step S141)

[0395] First, in step S141, the projector control device 100 acquires location information and analyzes the current vehicle position.

[0396] This processing is handled by Figure 12 The position analysis unit 122 of the projector control device 100 shown in the diagram executes the GPS 103 and the data unit 104.

[0397] (Steps S142 and S143)

[0398] Next, in steps S142 and S143, the projector control device 100 searches the storage unit 105 or the server 150 and determines whether the past log information corresponding to the current vehicle location is registered in the storage unit 105 or the server 150.

[0399] This processing is handled by Figure 12 The control information generation unit 123 of the data processing unit 104 of the projector control device 100 shown in the diagram executes.

[0400] If it is determined that the past log information corresponding to the current vehicle location is registered in storage unit 105 or server 150, the process proceeds to step S144.

[0401] On the other hand, if it is determined that the past log information corresponding to the current vehicle location is not registered in the storage unit 105 or the server 150, the process proceeds to step S145.

[0402] (Step S144)

[0403] Based on the historical log information corresponding to the current vehicle location, the following processing steps are performed.

[0404] In step S144, based on the projector setting information and screen setting information recorded in the past log information corresponding to the current vehicle position obtained in step S142, the projector control device 100 performs control to implement the same settings as these setting information.

[0405] (Step S145)

[0406] On the other hand, if it is determined that the past log information corresponding to the current vehicle location is not registered in the storage unit 105 or the server 150, the process proceeds to step S145 and performs the following processing.

[0407] In step S145, the projector control device 100 acquires images from the camera and sensors, as well as sensor detection information.

[0408] (Step S146)

[0409] Next, in step S146, the projector control device 100 analyzes the images and sensor detection information acquired in step S145 to determine the status of each occupant.

[0410] Specific examples of occupant status determination and processing have been referenced above. Figures 7-11 An explanation was provided.

[0411] Note that the occupant status determination process is in Figure 12 This is performed in the occupant analysis unit 121 of the data processing unit 104 shown in the diagram.

[0412] Figure 12 The occupant analysis unit 121 of the data processing unit 104 shown in the diagram analyzes the images and sensor detection information according to a pre-set algorithm to determine the status of each occupant.

[0413] (Step S147)

[0414] Finally, in step S147, the projector control device 100 controls the projector 108 and the screen 109 based on the occupant analysis results indicating the occupant status determined in step S146.

[0415] Specifically, as referenced above Figures 7-11 The control processing of lighting and content output is carried out in units of partitioned areas of screen 109, corresponding to the status of the occupants.

[0416] Note that the output range of the projector 108 and the partitioning of the screen 109 are also adjusted according to the position of each occupant based on the detection information analyzed by the distance sensor 102.

[0417] Note that this control mode can be changed via input processing by the user (occupant) through input unit 106.

[0418] For example, the content output from projector 108 can be individually set by each occupant. See also the above reference. Figure 11 The settings described above allow for the output of different content above each passenger's seat.

[0419] Furthermore, in the projector and screen control using past log information executed in step S144, the processing may be configured in which the degree of matching between the current time and the time recorded in the past log information is further taken into account as described above, or in which user-specific control processing is performed using user identification results.

[0420] Figure 19 The process illustrated in the diagram is one of the steps where past log information is applied. Figure 18 The control processing shown in the diagram above and the reference above Figure 14 The description outlines the sequence of control processes based on crew actions or states, but control processes applied to past log information are performed first.

[0421] Not limited to such a processing sequence, a configuration in which processing is performed sequentially can be adopted, in which, for example, the first step is to execute... Figure 19 The processing steps S145 to S147 shown in the diagram are as follows, i.e., refer to Figure 14The description covers control processing based on occupant actions or states, and the application of past log information when the control mode based on occupant actions or states cannot be determined. Figure 18 The control process is illustrated in the diagram.

[0422] [6. Example of the hardware configuration of the projector control device disclosed herein]

[0423] Next, specific hardware configuration examples of the projector control device disclosed herein will be described.

[0424] Figure 20 This is a diagram illustrating the specific hardware configuration of the projector control device disclosed herein.

[0425] This will be explained in Figure 20 The hardware components are shown in the diagram.

[0426] The central processing unit (CPU) 301 functions as a data processing unit, configured to perform various processes according to a program stored in the read-only memory (ROM) 302 or storage unit 308. For example, it performs processes corresponding to the sequences described in the above embodiments. The random access memory (RAM) 303 stores the program executed by the CPU 301, as well as the data. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304.

[0427] CPU 301 is connected to input / output interface 305 via bus 304. Input / output interface 305 is connected to input unit 306 and output unit 307. Input unit 306 includes various switches, keyboard, touch panel, mouse, microphone, camera, etc., and output unit 307 includes display, speaker, etc.

[0428] Note that the output unit 307 includes a projector and a screen.

[0429] The CPU 301 takes in commands, status data, etc. from the input unit 306, performs various processes, and outputs the processing results to, for example, the output unit 307.

[0430] The storage unit 308, connected to the input / output interface 305, includes, for example, a hard disk, and stores programs executed by the CPU 301 and various data. The communication unit 309 functions as a data transmission / reception unit for data communication via a network such as the Internet or a local area network, and communicates with external devices.

[0431] The driver 310 connected to the input / output interface 305 drives removable media 311 such as disks, optical disks, magneto-optical disks, or semiconductor memories (such as memory cards) and performs data recording and reading.

[0432] [7. Summary of the composition of this disclosure]

[0433] Embodiments of this disclosure have been described in detail with reference to specific examples. However, it will be apparent to those skilled in the art that various modifications and substitutions can be made to the embodiments without departing from the scope of this disclosure. In other words, this invention is disclosed by way of example and should not be construed as restrictive. To determine the scope of this disclosure, reference should be made to the claims.

[0434] Note that the technology disclosed in this specification may have the following configuration.

[0435] (1) A projector control device, comprising:

[0436] The data processing unit is configured to perform occupant status analysis of the vehicle and execute output control of the projector according to the occupant analysis results.

[0437] The data processing unit controls the projector to output illumination light with brightness determined according to the occupant analysis results, using the screen of the vehicle roof on which the projector outputs light as a unit, in sections.

[0438] (2) The projector control device according to (1), wherein

[0439] The data processing unit outputs illumination light with brightness determined according to the occupant's status, on a unit basis, in the screen partition area above each occupant.

[0440] (3) The projector control device according to (1) or (2), wherein

[0441] The data processing unit analyzes the occupants' status based on images captured by the occupants' cameras.

[0442] (4) The projector control device according to any one of (1) to (3), wherein

[0443] The data processing unit sets the output range of the projector's output light and the partitioned areas of the screen according to the occupant's position measured by the distance sensor.

[0444] (5) The projector control device according to any one of (1) to (4), wherein

[0445] When an occupant is asleep, the data processing unit controls the projector to set the screen partition area above the sleeping occupant to low brightness lighting output or to stop the lighting.

[0446] (6) The projector control device according to any one of (1) to (5), wherein

[0447] The data processing unit controls the projector to set the screen partition above the sleeping occupant to high-brightness lighting output when the occupant is not asleep.

[0448] (7) The projector control device according to any one of (1) to (6), wherein

[0449] The data processing unit outputs lighting or images with brightness determined according to the occupant's status, on a unit of screen partition area above the occupant.

[0450] (8) The projector control device according to any one of (1) to (7), wherein

[0451] The data processing unit outputs images to the screen partition area above the occupant while the occupant is looking at the screen above.

[0452] (9) The projector control device according to any one of (1) to (8), wherein

[0453] The data processing unit performs screen control, on a unit basis, on the partitioned areas of the vehicle roof screen on which the projector outputs light, to adjust the screen characteristics according to the occupant analysis results.

[0454] (10) The projector control device according to (9), wherein the screen is a dynamic functional screen capable of adjusting screen characteristics.

[0455] (11) The projector control device according to any one of (1) to (10), wherein

[0456] The data processing unit refers to past log information that records projector setting information associated with location information, and obtains projector setting information recorded in past log information that has location information that is the same as or close to the vehicle's current location.

[0457] The projector control was adjusted to match the settings obtained from past log information.

[0458] (12) The projector control device according to (11), wherein

[0459] The historical log information is recorded in the storage unit of the projector control device or in an external server.

[0460] (13) The projector control device according to any one of (1) to (12), wherein

[0461] The data processing unit performs occupant user identification based on camera-captured images of the occupants.

[0462] From the historical log information recording projector settings for each user, the projector settings associated with the identified user can be retrieved, and

[0463] The projector control was adjusted to match the settings obtained from past log information and corresponding to the user's projector settings.

[0464] (14) A projector control method executed in a projector control device, the projector control method comprising, via a data processing unit:

[0465] The occupant status analysis steps for performing occupant status analysis in a vehicle; and

[0466] The projector control steps, which execute the projector output control based on the occupant analysis results, include...

[0467] In the projector control steps,

[0468] Projector control is performed on a partitioned area of ​​the screen on the vehicle roof where the projector outputs light, and the projector outputs illumination light with a brightness determined according to the occupant analysis results.

[0469] (15) A program that causes a projector control device to perform projector control, the program causing a data processing unit to execute:

[0470] The occupant state analysis steps are used to perform occupant state analysis in order to control the vehicle; and

[0471] The projector control steps involve performing projector output control based on occupant analysis results, wherein...

[0472] In the projector control steps,

[0473] Projector control is performed on a partitioned area of ​​the screen on the vehicle roof where the projector outputs light, and the projector outputs illumination light with a brightness determined according to the occupant analysis results.

[0474] Furthermore, the series of processes described in the specification can be executed by hardware, software, or a combination of both. In the case of software-executed processing, the program recording the processing sequence can be installed in the memory of a computer integrated into dedicated hardware and executed, or the program can be installed in a general-purpose computer capable of performing various processes and executed. For example, the program can be pre-recorded on a recording medium. The program can be installed from the recording medium onto the computer, or it can be received via a network such as a local area network (LAN) or the Internet and installed on a recording medium such as an internal hard drive.

[0475] Note that the various processes described in this specification can be executed not only in chronological order as described, but also in parallel or individually, depending on the processing capacity of the device executing the process or as needed. Furthermore, the system described in this specification is a logical collection of multiple devices, and is not limited to a system where each type of device is housed in the same housing.

[0476] Industrial applicability

[0477] As described above, one embodiment of this disclosure provides an apparatus and method for analyzing the state of vehicle occupants and, based on the analysis results, performing illumination and image output control of a projector on a per-vehicle roof screen partition basis.

[0478] Specifically, for example, a data processing unit is provided, configured to perform occupant status analysis of a vehicle and execute projector output control according to the occupant analysis results. The data processing unit performs projector control by outputting illumination light with brightness determined according to the occupant analysis results, on a partitioned area of ​​the screen on the vehicle roof where the projector outputs light. The data processing unit analyzes the occupant status based on images captured by a camera and outputs illumination light with brightness determined according to the occupant status, on a partitioned area of ​​the screen above each occupant.

[0479] This configuration enables an apparatus and method for analyzing the state of vehicle occupants and, based on the analysis results, controlling the lighting and image output of a projector in units of screen partitions on the vehicle roof.

[0480] List of reference numerals

[0481] 11 Projectors

[0482] 12 screens (ceiling)

[0483] 21 cameras

[0484] 22 Distance Sensor

[0485] 100 Projector Control Device

[0486] 101 Camera

[0487] 102 Distance Sensor

[0488] 103GPS

[0489] 104 Data Processing Unit

[0490] 105 storage units

[0491] 106 Input Units (Operation Units)

[0492] 107 Communication Unit

[0493] 108 Projector

[0494] 109 screen

[0495] 121 Crew Analysis Unit

[0496] 122 Position Analysis Unit

[0497] 123 Control Information Generation Unit

[0498] 124 Projector Control Unit

[0499] 125 screen control unit

[0500] 150 servers

[0501] 301CPU

[0502] 302ROM

[0503] 303RAM

[0504] 304 bus

[0505] 305 Input / Output Interface

[0506] 306 Input Unit

[0507] 307 Output Unit

[0508] 308 memory cells

[0509] 309 Communication Unit

[0510] 310 drive

[0511] 311 Removable Media

Claims

1. A projector control device, comprising: Data processing unit, the data processing unit being configured to: Perform occupant status analysis in the vehicle; Refer to the past log information that records the projector settings associated with the vehicle's location information; Retrieve projector setting information recorded in past log information that has the same or similar location information as the vehicle's current location; as well as The projector's output control is executed based on the occupant analysis results and the acquired projector setting information. The data processing unit performs output control of the projector for outputting illumination light with brightness determined according to the occupant analysis results, on a unit basis, using partitioned areas of the screen on the vehicle roof where the projector outputs light. The data processing unit performs output control of the projector to adjust the settings to be the same as those obtained from past log information.

2. The projector control device according to claim 1, wherein... The data processing unit outputs illumination light with brightness determined according to the occupant's status, on a unit basis, in the screen partition area above each occupant.

3. The projector control device according to claim 1, wherein... The data processing unit analyzes the occupants' status based on images captured by the occupants' cameras.

4. The projector control device according to claim 1, wherein... The data processing unit sets the output range of the projector's output light and the partitioned areas of the screen according to the occupant's position measured by the distance sensor.

5. The projector control device according to claim 1, wherein... When an occupant is asleep, the data processing unit controls the projector to set the screen area above the sleeping occupant to low brightness lighting output or to stop the lighting.

6. The projector control device according to claim 1, wherein... The data processing unit controls the projector to set the screen partition above the sleeping occupant to high-brightness lighting output when the occupant is not asleep.

7. The projector control device according to claim 1, wherein... The data processing unit outputs lighting or images with brightness determined according to the occupant's status, on a unit of screen partition area above the occupant.

8. The projector control device according to claim 1, wherein... The data processing unit outputs images to the screen partition area above the occupant while the occupant is looking at the screen above.

9. The projector control device according to claim 1, wherein... The data processing unit performs screen control, on a unit basis, on the partitioned areas of the vehicle roof screen on which the projector outputs light, to adjust the screen characteristics determined according to the occupant analysis results.

10. The projector control device according to claim 9, wherein the screen includes a dynamic functional screen capable of adjusting screen characteristics.

11. The projector control device according to claim 1, wherein... The past log information includes information recorded in the storage unit of the projector control device or recorded on an external server.

12. The projector control device according to claim 1, wherein... The data processing unit User identification of occupants is performed based on camera images captured by the occupants. From the historical log information recording projector settings for each user, obtain the projector settings associated with the identified user, and The projector control was adjusted to match the settings obtained from past log information and corresponding to the user's projector settings.

13. A projector control method executed in a projector control device, the projector control method comprising, Through the data processing unit: Perform occupant status analysis in the vehicle; Refer to the past log information that records the projector settings associated with the vehicle's location information; Retrieve projector setting information recorded in past log information that has the same or similar location information as the vehicle's current location; as well as The projector's output control is executed based on the occupant analysis results, whereby... Output control of the projector, which outputs illumination light with a brightness determined according to the occupant analysis results, is performed on a partitioned basis, using the screen of the vehicle roof on which the projector outputs light. Perform output control of the projector to adjust the settings to be the same as those obtained from past log information.

14. A computer program product comprising a program that causes a projector control device to perform projector control, the program causing a data processing unit to perform a method, the method comprising: Perform occupant status analysis in the vehicle; Refer to the past log information that records the projector settings associated with the vehicle's location information; Retrieve projector setting information recorded in past log information that has the same or similar location information as the vehicle's current location; as well as The projector's output control is executed based on the occupant analysis results, whereby... Output control of the projector, which outputs illumination light with a brightness determined according to the occupant analysis results, is performed on a partitioned basis, using the screen of the vehicle roof on which the projector outputs light. Perform output control of the projector to adjust the settings to be the same as those obtained from past log information.

Images