In-vehicle information processing system

Abstract

This invention provides control technology for a vehicle air conditioner that allows air to be directed towards a desired part of the vehicle occupant via gesture. [Solution] The in-vehicle information processing device includes a control unit that performs the following actions based on an image of the vehicle taken by a camera: detecting an action by a vehicle occupant to identify any part of the occupant's body; setting the part as the target of airflow from the air conditioner; tracking the positional changes of the part in the image and determining the position of the part in real space from the image; and controlling the airflow direction of the air conditioner to blow air toward the position of the part in real space.

Description

Technical Field

[0001] The present invention relates to a technology for controlling an air conditioner mounted on a vehicle.

Background Art

[0002] Vehicles are equipped with vehicle air conditioners so that passengers can comfortably spend time in the vehicle interior. In a vehicle, the operation unit of the air conditioner is often provided near the dashboard, making it difficult for the passengers in the rear seats of the vehicle to operate the air conditioner. Also, when the driver operates the air conditioner, the line of sight is diverted from the front, resulting in a decrease in the concentration on the driving operation.

[0003] Regarding this, there is a known technology for operating the set temperature, air volume, and air direction of an air conditioner by gestures (Patent Document 1). According to the technology as described in Patent Document 1, since the air conditioner is operated by gestures, the passengers in the rear seats can also operate the air conditioner, and the driver can also operate the air conditioner without diverting the line of sight from the front.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, vehicle passengers have needs such as wanting to cool only a part of the body, wanting to quickly warm cold hands, etc., that is, wanting to direct the air of the air conditioner only to specific parts of the body such as the face, neck muscles, and palm. However, with the conventional technology as described in Patent Document 1, it is difficult to continuously blow air to a specific body part of a passenger with a changing body position.

[0006] One aspect of the technology of this disclosure aims to provide a control technology for a vehicle air conditioner that can direct airflow to a desired part of a vehicle occupant by gesture. [Means for solving the problem]

[0007] One aspect of the technology of this disclosure is exemplified by the following in-vehicle information processing device. Specifically, the control unit of the in-vehicle information processing device detects an action that identifies a part of an occupant's body based on an image of the vehicle interior captured by a camera, and directs the airflow of the air conditioner to that part. The control unit also tracks the positional changes of the part in the image and determines the position of the part in real space from the image, and controls the airflow direction of the air conditioner to direct the airflow towards the position of the part in real space. [Effects of the Invention]

[0008] According to the technology of this disclosure, a vehicle occupant can continuously receive airflow from an air conditioner on a desired part of their body simply by making a gesture that identifies the part of their body. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing an example of a vehicle according to this embodiment. [Figure 2] Figure 2 is a schematic diagram showing the area near the dashboard of the vehicle according to this embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the hardware configuration of an in-vehicle information processing device according to the present invention. [Figure 4] Figure 4 shows an example of the hardware configuration of an air conditioner according to the embodiment. [Figure 5] Figure 5 is a functional block diagram showing an example of a functional unit of an in-vehicle information processing device according to an embodiment. [Figure 6]Figure 6A is an explanatory diagram showing an example of gesture operation for identifying body parts. Figure 6B is an explanatory diagram showing an example of gesture operation for changing airflow. Figure 6C is an explanatory diagram showing an example of gesture operation for changing air conditioning temperature. [Figure 7] Figure 7 shows an example of a screen displayed on the display when controlling the air conditioner according to the embodiment. [Figure 8] Figure 8 is a flowchart showing an example of the processing flow of an in-vehicle information processing device according to the present invention. [Figure 9] Figure 9 is a functional block diagram showing an example of the functional section of an in-vehicle information processing device according to the first modified example. [Figure 10] Figure 10 is a functional block diagram showing an example of the functional section of an in-vehicle information processing device according to the second modified example. [Figure 11] Figure 11 shows an example of the screen displayed on the display before and after air conditioner control according to the second modified example. [Modes for carrying out the invention]

[0010] <Embodiment> (Hardware configuration) Embodiments of the present invention will be described below with reference to the drawings. Figures 1 and 2 show an example of a vehicle 1 according to an embodiment. Figure 1 is a view of the vehicle 1 from above, and the ceiling is omitted from the illustration in order to explain the interior of the vehicle 1. Figure 2 is a schematic diagram showing the general outline of the front part of the interior of the vehicle 1, and shows the general outline of the vicinity of the windshield 16 and the dashboard 10.

[0011] Vehicle 1 according to this embodiment is a so-called four-door sedan type passenger car with a driver's seat and a passenger seat positioned at the front and a three-seater seat at the rear. A dashboard 10 is positioned in front of the driver's seat and passenger seat. A display 12, a microphone 13, and an operation panel 14 are positioned near the center of the dashboard 10. A steering wheel 11 is positioned directly in front of the driver's seat.

[0012] In addition, cameras 151 and 152 are respectively arranged near the ceiling in the vicinity of the windshield 16 on the driver's side and the passenger's side of the vehicle 1. The cameras 151 and 152 are arranged at positions where they can photograph the passengers inside the vehicle. The cameras 151 and 152 may have different shooting fields in the same shooting method, or may be configured to shoot substantially the same field of view in different shooting methods.

[0013] (In-vehicle information processing device) An IVI (In-Vehicle Infotainment) system is introduced in the vehicle 1, and an in-vehicle information processing device 20 as its control terminal is installed in front of the vehicle. FIG. 3 is a block diagram showing the hardware configuration of the in-vehicle information processing device 20. As shown in FIG. 3, the in-vehicle information processing device 20 includes components such as a processor 21, a memory 22, an input interface (IF) 23, an output IF 24, and a communication IF 25, which are interconnected by a connection bus 29.

[0014] As the processor 21, for example, any arithmetic processing device such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor) can be adopted. At least a part of the processes executed by the processor 21 may also be executed by an integrated circuit (IC) or other digital circuits. In addition, at least a part of the processor 21 may include an analog circuit.

[0015] The memory 22 includes a main storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), and an auxiliary storage device such as an SSD (Solid State Drive), an EPROM (Erasable Programmable ROM), an HDD (Hard Disk Drive), a USB memory, and an SD (Secure Digital) memory card.

[0016] The memory 22 stores information such as programs executed by the processor 21, data processed by the processor 21, operation setting information, and various tables. When the program stored in the memory 22 is executed by the processor 21, each component of the in-vehicle information processing device 20 and the hardware in the vehicle 1 communicatively connected thereto is controlled. Thereby, functional units as described later can be realized. That is, the processor 21 in the present embodiment is an example of the control unit according to the present invention.

[0017] The input IF 23 is an IF connected to various input devices such as the microphone 13, the operation panel 14, and the cameras 151 and 152. For example, the processor 21 acquires the image data of the passengers photographed by the cameras 151 and 152 via the input IF 23. Note that the microphone 13, the operation panel 14, and the cameras 151 and 152 can each be configured using a desired known technique.

[0018] Further, the output IF 24 is an IF connected to output devices such as the display 12 and a speaker (not shown). For example, the processor 21 causes the display 12 to display information related to the control state of the air conditioner 30 via the output IF 24. Note that the display 12 can also display various other information including route guidance information and entertainment information. The display 12 is, for example, a liquid crystal display, and a touch panel display can also be adopted as a configuration that also serves as an input device.

[0019] The communication IF 25 is an IF for making a communication connection with other in-vehicle devices such as the air conditioner 30 and an external network, and an appropriate configuration can be adopted according to the connection method (difference between wired and wireless, communication standard, etc.).

[0020] (Air conditioner) Vehicle 1 is equipped with an air conditioner 30. Figure 4 is a block diagram illustrating the schematic configuration of the air conditioner 30. The air conditioner 30 includes air outlets 31a, 31b, 31c, 31d, 31e, 31f, a louver drive mechanism 32, and an air supply mechanism 33. These components are controlled by the in-vehicle information processing device 20, thereby enabling the air conditioning system of vehicle 1 to function.

[0021] As shown in Figures 1 and 2, air vents 31a and 31b are located on the driver's side, air vents 31c and 31d are located on the passenger side, and air vents 31e and 31f are located on the rear seat side. Each of the air vents 31a, 31b, 31c, 31d, 31e, and 31f is provided with a louver. In the following, unless there is a need to distinguish between the air vents 31a, 31b, 31c, 31d, 31e, and 31f, they will simply be referred to as air vent 31.

[0022] The louver drive mechanism 32 includes an actuator, such as a motor, and transmits the displacement of the actuator to the louvers of each outlet. By changing the orientation of the louvers with the louver drive mechanism 32, the direction of the air blown out from each outlet 31 is changed.

[0023] The air supply mechanism 33 cools or heats the air taken in from inside or outside the vehicle and supplies it to the interior of the vehicle through the outlet 31. The specific configuration and mechanism for supplying cool and warm air by the air supply mechanism 33 differ between vehicles equipped with internal combustion engines and EVs (Electric Vehicles), but in either case, publicly known technologies can be adopted as appropriate, so a detailed explanation is omitted. In the following, when cool air and warm air are not distinguished, they will simply be referred to as "wind."

[0024] (Functional block) Next, the functional blocks of the in-vehicle information processing device 20 related to the control of the air conditioner 30 will be described based on Figure 5. As shown in Figure 5, the in-vehicle information processing device 20 includes the following functional units related to the control of the air conditioner 30: a gesture detection unit 201, an airflow area identification unit 202, an identification area tracking unit 203, an airflow direction control unit 204, an airflow volume control unit 205, a temperature control unit 206, and a display unit 207. The in-vehicle information processing device 20 realizes each of the above functional units by having the processor 21 execute a program loaded into the memory 22 (main memory).

[0025] The gesture detection unit 201 analyzes the image data acquired by cameras 151 and 152 in real time. The gesture detection unit 201 then detects the occupant's actions related to controlling the air conditioner 30 (hereinafter also referred to as gesture operations) from the image data. The airflow area identification unit 202 determines whether the occupant's gesture operation identifies a part of the body. If the airflow area identification unit 202 determines that the occupant's gesture operation identifies a part of the body, it sets that part as the area to be air-conditioned by the air conditioner 30.

[0026] Furthermore, to detect specific regions or gesture-related actions from image data, pre-trained models that have been trained based on previously acquired image data can be used. For example, known techniques such as R-CNN, YOLO (You Only Look Once), SSD (Single Shot MultiBox), DCN (Deformed Convolutional Networks), and DETR (End-to-End Object Detection with Transformers) may be employed as methods for extracting specific regions from images.

[0027] Actions that occupants can use to identify body parts might include, for example, placing their hand on the body part (or tapping it several times). Such actions are easy for anyone and can be performed intuitively, even while driving.

[0028] However, the gestures used by the crew to identify body parts are not limited to these. For example, the action of bringing the body part closer to the air vent 31 can also be considered a gesture used by the crew to identify a body part. This means that actions performed unconsciously by the crew, such as wanting to warm numb hands or cool a flushed face, can also be considered gestures.

[0029] In addition, actions such as pointing the body part to be identified towards cameras 151 and 152 (showing it to the camera) can also be used as gestures for the occupant to identify a body part. This allows for identifying a "hand" without significantly changing posture. Furthermore, gestures for identifying a body part can be performed even when both hands are occupied.

[0030] The specific area tracking unit 203 tracks the area set by the airflow area identification unit 202 in real time using image data acquired by cameras 151 and 152, and determines its actual position inside the vehicle. Furthermore, the process of tracking the area set by the airflow area identification unit 202 within the image can employ any desired tracking process, such as using a Cowman filter. There are also no particular limitations on the method for determining the position of the area tracked in the image in real space. For example, it may be determined using a table that associates the XY coordinates in the field of view captured by cameras 151 and 152 with the three-dimensional coordinates in real space.

[0031] The airflow direction control unit 204 controls the louvers of the air outlet 31 so that air from the air conditioner 30 is blown to the location determined by the specific location tracking unit 203. Specifically, the airflow direction control unit 204 transmits a control command signal to the louver drive mechanism 32 via the communication IF 25, thereby moving the louvers to blow air to the specified location in real time.

[0032] Furthermore, the airflow control unit 204 can individually control each of the louvers at the air outlets 31a, 31b, 31c, 31d, 31e, and 31f, thereby enabling individual airflow control for the driver's seat occupant, the passenger seat occupant, and the rear seat occupant.

[0033] The airflow control unit 205 controls the air supply mechanism 33 to blow air at a set airflow rate. The temperature control unit 206 controls the air supply mechanism 33 to blow cool air or warm air based on the cooling or heating control target. In this embodiment, the airflow rate and temperature can be set and changed via the operation panel 14, but can also be done by gesture control.

[0034] When setting or changing the airflow or temperature using gesture controls, the airflow control unit 205 and the temperature control unit 206 perform the following controls. After the area to be airflow controlled is set by the airflow area identification unit 202, the gesture detection unit 201 determines whether a predetermined gesture operation to adjust the airflow or temperature has been performed based on the image data. If it is determined that a gesture operation has been performed, the airflow control unit 205 controls the airflow and the temperature control unit 206 controls the air supply mechanism 33 to change the airflow and temperature, respectively, according to the content of the gesture operation.

[0035] Figures 6A and 6C are schematic diagrams illustrating examples of gesture operations. Figure 6A shows an example of a gesture operation to determine the area to which air is to be blown, where the user places their hand on the area they want to specify. Figure 6B shows an example of a gesture operation to adjust the airflow, where the user sweeps their arm horizontally up and down. For example, sweeping the arm from top to bottom is a gesture operation to decrease the airflow, and sweeping the arm from bottom to top is a gesture operation to increase the airflow. Figure 6C shows an example of a gesture operation to adjust the temperature, where the user rotates their forearm. For example, rotating the forearm counterclockwise towards the elbow is a gesture operation to increase the temperature, and rotating it clockwise is a gesture operation to decrease the temperature.

[0036] The display unit 207 is a functional unit that displays various information on the display 12. Figure 7 shows an example of a display screen that shows the control status when the air conditioner 30 is operating in a mode that tracks and blows air to a specific area. In the example screen in Figure 7, a human-shaped silhouette is displayed to show which part of the body (in this embodiment, the area slightly to the left of the neck) is being blown to, a display indicating that the airflow is being adjusted, and a display of the current set temperature (21°C) are shown.

[0037] (Processing flow) Figure 8 is a flowchart showing an example of a processing flow for control in which the in-vehicle information processing device 20 according to the embodiment detects the occupant's gesture operation and blows air to a specific area. The following describes an example of the processing flow of the in-vehicle information processing device 20 with reference to Figure 8.

[0038] First, as a premise of the processing flow, the operation of the control panel 14, etc., in the interior of vehicle 1 A series of processing flows are initiated when the air conditioning is started. In step S1, the in-vehicle information processing device 20 acquires image data of the vehicle interior from cameras 151 and 152 (S1). Subsequently, the in-vehicle information processing device 20 (gesture detection unit 201) determines, based on the image data, whether or not the occupant is performing a gesture operation to identify a part of their body (S2).

[0039] If it is determined in step S2 that the occupant is performing a gesture operation to identify a body part (i.e., an action to identify a body part is detected), the process proceeds to step S3. On the other hand, if it is not determined in step S2 that the occupant is performing a gesture operation to identify a body part, the process proceeds to step S7.

[0040] In step S3, the in-vehicle information processing device 20 (air blower area identification unit 202) identifies the area to be blown according to the gesture operation detected in step S2 (S3). Subsequently, in step S4, the in-vehicle information processing device 20 (identified area tracking unit 203) performs a process to track the identified area (including a process to determine the position in real space from the position of the image data) (S4). Note that the method for extracting and tracking the specific area from the image data has already been explained, so a further explanation will be omitted.

[0041] Then, the in-vehicle information processing device 20 (wind direction control unit 204) sends a command signal to the louver drive mechanism 32 to drive the louvers of each outlet 31a, 31b, 31c, 31d, 31e, and 31f so that air is blown towards the specific area being tracked (S5). In the next step S6, the in-vehicle information processing device 20 determines whether or not the termination conditions related to airflow control are met (S6). The termination conditions can be, for example, when an operation input to terminate airflow control is received, or when the body temperature of the area to which air is blown reaches a threshold corresponding to cooling or heating.

[0042] If the in-vehicle information processing device 20 determines in step S6 that the termination condition is met, it terminates the series of airflow control processes. On the other hand, if the in-vehicle information processing device 20 does not determine in step S6 that the termination condition is met, it returns to S1 and repeats the processes from there onward.

[0043] If the process proceeds to step S7 due to a "No" branch in step S2, the in-vehicle information processing device 20 determines whether the area to be air-ventilated has already been identified (S7). If the in-vehicle information processing device 20 determines in step S7 that the area to be air-ventilated has already been identified, it proceeds to step S4. On the other hand, if the in-vehicle information processing device 20 determines in step S7 that the area to be air-ventilated has not been identified, it returns to step S1 and repeats the subsequent processing.

[0044] (Effects of the embodiment) According to the vehicle 1 and in-vehicle information processing device 20 of this embodiment, an occupant of the vehicle 1 can simply make a gesture to identify a part of their body, and the airflow direction of the air conditioner 30 will be automatically controlled, allowing them to continuously receive airflow from the air conditioner 30 on the desired part of their body. Furthermore, since the airflow volume and temperature can also be set with gestures, the occupant in the driver's seat can control the air conditioning without losing attention to the road ahead. In addition, since the airflow direction, airflow volume, and temperature can be controlled with gestures, even occupants in the rear seats can operate the air conditioner 30.

[0045] <Example 1> In the above embodiment, the target area for airflow and the airflow direction were controlled solely based on gesture operations analyzed from image data. However, it is also possible to combine this with operations using voice input from the microphone 13. Such modified examples will be described below. do.

[0046] The configuration of the modified vehicle 1 is substantially the same as that of the embodiment. In the following, the same reference numerals are used for components similar to those in the embodiment, and further explanation is omitted. Figure 9 is a block diagram showing the functional configuration of the in-vehicle information processing device 210 according to this modified example. As shown in Figure 9, the in-vehicle information processing device 210 according to this modified example has the same functional configuration as the in-vehicle information processing device 20 according to the embodiment, except that it is equipped with a voice recognition unit 211.

[0047] The voice recognition unit 211 receives voice input from the occupant via the microphone 13. Specifically, regarding the control of the air conditioner 30, it receives voice input requesting airflow. However, the voice recognition unit 211 can also receive voice input for various other functions provided by the in-vehicle information processing device 210.

[0048] In this modified version, the in-vehicle information processing device 210 does not immediately blow air onto a body part of the occupant, even if the gesture detection unit 201, the airflow area identification unit 202, and the identified area tracking unit 203 have identified and tracked that body part. Specifically, when the voice recognition unit 211 receives a voice input operation from the occupant requesting airflow, the airflow direction control unit 204 transmits a control signal to the louver drive mechanism 32. In other words, if the voice recognition unit 211 does not receive a voice input operation, airflow direction control is not performed. The timing of receiving the voice input operation may be before the gesture detection unit 201 detects a gesture operation to identify a body part, or after the identified area tracking unit 203 has started tracking the identified body part.

[0049] When the airflow direction of the air conditioner 30 is controlled solely based on gesture operations, there is a risk of misdetection of actions such as swatting away insects that have entered the vehicle or scratching an itchy area as gesture operations. According to the in-vehicle information processing device 210 of this modified example, by combining voice input operations and gesture operations, such malfunctions based on misdetection can be prevented.

[0050] <Modification 2> The following describes further modifications based on Figures 10 and 11. In this modification, camera 152 is a thermographic camera. Therefore, the in-vehicle information processing device 220 in this modification can acquire the body temperature information of the occupants from thermal image data inside the vehicle. In addition, in this modification, the auxiliary storage device of memory 22 stores the body surface temperature value for turning the air conditioning ON (OFF) (first threshold) and the body surface temperature value for turning the heating ON (OFF) (second threshold).

[0051] Figure 10 is a functional block diagram showing the functional parts of the in-vehicle information processing device 220 according to this modified example. The in-vehicle information processing device 220 according to this modified example differs from the in-vehicle information processing device 20 according to the embodiment in that it includes a body temperature detection unit 221 and a threshold comparison unit 222.

[0052] The body temperature detection unit 221 acquires information on the body temperature of the occupants based on the thermal image data acquired by the camera 152. The threshold comparison unit 222 compares the body temperature of the occupants' body parts identified by the airflow area identification unit 202 with the first threshold and second threshold stored in the memory 22, and detects the difference.

[0053] Then, as a result of the comparison by the threshold comparison unit 222, if the body temperature of a specific part of the occupant is higher than the first threshold, the temperature control unit 206 sends a command signal to the air supply mechanism 33 to blow cold air. The body temperature detection unit 221 continuously acquires the body temperature information of the occupant, and the threshold comparison unit 222 also continuously compares the first threshold with the body temperature of a specific part of the occupant. The airflow control unit 205 controls the airflow according to the magnitude of the temperature difference. Specifically, when the body temperature of a specific body part is higher than the first threshold, the airflow control unit 205 sends a command signal to the air supply mechanism 33 so that the airflow increases as the difference increases and decreases as the difference decreases. When the body temperature of a specific body part of the occupant falls below the first threshold, the control of the air conditioner 30 by the airflow direction control unit 204, airflow control unit 205, and temperature control unit 206 stops.

[0054] On the other hand, if the comparison by the threshold comparison unit 222 shows that the body temperature of a specific part of the occupant is lower than the second threshold, the temperature control unit 206 sends a command signal to the air supply mechanism 33 to blow warm air. The body temperature detection unit 221 continuously acquires the body temperature information of the occupant, and the threshold comparison unit 222 also continuously compares the second threshold with the body temperature of a specific part of the occupant. At this time, the airflow control unit 205 controls the amount of airflow according to the magnitude of the difference between the second threshold and the body temperature of a specific part of the occupant. Specifically, when the body temperature of a specific part is lower than the second threshold, the airflow control unit 205 sends a command signal to the air supply mechanism 33 such that the airflow is larger when the difference is large and smaller when the difference is small. When the body temperature of a specific part of the occupant becomes equal to or greater than the second threshold, the control of the air conditioner 30 by the airflow direction control unit 204, airflow control unit 205, and temperature control unit 206 is stopped.

[0055] Furthermore, if the body temperature of a specific part of a crew member is higher than the first threshold, and as a result of continuously blowing cold air to that part of the body, the body temperature of that part falls below the second threshold, the temperature control unit 206 sends a command signal to the air supply mechanism 33 to blow warm air instead of cold air. The same applies to the reverse scenario of heating and cooling.

[0056] In this modified example, the display unit 207 displays thermal images of the occupants before and after control of the air conditioner 30 on the display 12. An example of such a display screen is shown in Figure 11. Figure 11 schematically shows an example of a screen displaying thermal images of the occupants from the chest up. In Figure 11, the darker shaded areas indicate high temperatures, and the lighter shaded areas indicate low temperatures.

[0057] Specifically, the display unit 207 displays a thermal image of the occupant on the display 12 before (or immediately after) blowing air to a specific part of the occupant. The left side of Figure 11 shows the thermal image at that time, where the temperature of the neck is highest. Then, as blowing air continues to that specific part, if the body temperature of that part falls below the first threshold, the display unit 207 displays the thermal image of the occupant at the end of the blowing on the display 12 again. The right side of Figure 11 shows the thermal image of the occupant at the end of the blowing.

[0058] According to this modified version, based on the occupant's body temperature obtained from thermal images, not only the airflow direction but also the air temperature of the air conditioner 30 can be automatically set and air can be directed to a specific area. Furthermore, since the occupant's body temperature is continuously monitored and the airflow volume and temperature are automatically adjusted according to the body temperature of the area to which air is being directed, convenience can be greatly improved. In addition, since the thermal images of the occupant before and after airflow control to a specific area are displayed on the display 12, the occupant can visually experience the effect of automatic airflow direction control.

[0059] <Other> As described above, the present invention can be understood as an in-vehicle information processing device for controlling an in-vehicle air conditioner, or as a control method for an in-vehicle air conditioner. Furthermore, the present invention can also be understood as an in-vehicle air conditioning system having an air conditioner, a camera, and a control unit for controlling these components, mounted on a vehicle.

[0060] The above examples are merely illustrative illustrations of the present invention, and the present invention is not specifically defined as described above. The present invention is not limited to its form. Various modifications are possible within the scope of its technical concept. For example, a configuration can be made by combining the configuration of Modification 1 and the configuration of Modification 2 described above.

[0061] Furthermore, in the above examples, cameras 151 and 152 were positioned near the ceiling close to the windshield 16, but the cameras can be positioned anywhere as long as they can be filmed by the occupants inside the vehicle. For example, the cameras may be positioned near the dashboard on the passenger side, on the indicator of the rearview mirror, etc. Also, the number of cameras is not limited to two; there may be one, three or more, etc.

[0062] Furthermore, in the above modified example 2, the memory 22 may store, in addition to the first threshold and the second threshold, the average value of the temperature change per predetermined time when the occupant used the air conditioner 30 in the past. The in-vehicle information processing device 220 may then change the airflow and / or temperature by comparing the current temperature change per predetermined time with the past average value.

[0063] Furthermore, in the above modified example 2, the in-vehicle information processing device 220 may, before the airflow control unit 205 and / or temperature control unit 206 transmit command signals to the air supply mechanism 33, suggest to the occupants, via the display 12 or speaker, a change in airflow or temperature. [Explanation of symbols]

[0064] 1. Vehicle 11. Steering wheel 12...Display 13. Mike 14. Control Panel 151, 152... Camera 16. Windshield 20, 210, 220... In-vehicle information processing unit 21... Processor 22...memory 23. Input Interface 24. Output Interface 25. Communication Interface 29... Connecting bus 30. Air conditioner 31, 31a, 31b, 31c, 31d, 31e, 31f... air outlets 32. Louver drive mechanism 33...Air supply mechanism

Claims

1. Based on images of the vehicle interior captured by a camera, the system detects actions taken by the vehicle's occupants to identify any part of the occupants' bodies. The aforementioned part is set as the target of airflow from the air conditioner, Track the changes in the position of the part in the aforementioned image and determine the position of the part in real space from the aforementioned image, Controlling the airflow direction of the air conditioner so as to blow air toward the location of the part in the real space, An in-vehicle information processing device equipped with a control unit that performs the following actions.

2. The control unit detects the action of the occupant placing their hand on the body part as an action to identify the body part. The in-vehicle information processing device according to claim 1.

3. The control unit, Based on the aforementioned image, detect a predetermined action performed by the occupant after placing their hand on the aforementioned body part. The air conditioner is controlled to change at least one of the strength or temperature of the air supplied to the part in accordance with the predetermined operation. The in-vehicle information processing device according to claim 2.

4. The control unit detects the action of the occupant bringing the part close to the air conditioner outlet as an action to identify the part. The in-vehicle information processing device according to claim 1.

5. The control unit detects the action of the occupant pointing the body part towards the camera as an action to identify the body part. The in-vehicle information processing device according to claim 1.

6. The control unit, When the system receives an audio input from the occupant requesting airflow, it controls the air conditioner to supply air to the aforementioned area. The in-vehicle information processing device according to claim 1.

7. The aforementioned camera includes a thermal imaging camera. The control unit, The air conditioner is controlled such that if the temperature of the area obtained from the thermal image captured by the thermographic camera is higher than a first threshold, cold air is sent to the area, or if it is lower than a second threshold, warm air is sent to the area. The in-vehicle information processing device according to claim 1.

8. The control unit, The temperature of the aforementioned part is continuously monitored, and the air conditioner is controlled to change at least one of the intensity or temperature of the air supplied to the aforementioned part according to the temperature of the aforementioned part. The in-vehicle information processing device according to claim 7.

9. This involves obtaining images of the vehicle's occupants taken with a camera, Based on the aforementioned image, detect an action that identifies any part of the occupant's body. and, The part related to the detected operation is to be the target of airflow from the air conditioner, Track the changes in the position of the part in the aforementioned image and determine the position of the part in real space from the aforementioned image, Controlling the airflow direction of the air conditioner so as to blow air toward the location of the part in the real space, A method for controlling an in-vehicle air conditioner, including [the specified feature].

10. A camera to film the occupants of the vehicle, Air conditioner and, A control unit that detects the action of identifying any part of the occupant's body based on an image captured by the camera, sets that part as the target of airflow from the air conditioner, tracks changes in the position of the part in the image, determines the position of the part in real space from the image, and controls the airflow direction of the air conditioner to blow air towards the position of the part in real space. An in-vehicle air conditioning system equipped with this feature.

Images