Determination system

Abstract

To provide a determination system that can simply and reliably identify a driver's intention for specific processing including application control.SOLUTION: A determination system includes a detection unit that detects an operation by a driver on an operation unit 202 in a vehicle 200, a determination device 10 that identifies a physical state of a part of the driver's body as a command action, and a vehicle control device 11 that controls the vehicle, wherein the vehicle control device executes processing that is at least partially different when the determination device identifies the driver's command action and the detection unit detects the driver's operation, compared with when the detection unit detects the driver's operation without the determination device identifying the driver's command action.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present embodiment relates to a determination system. [Background technology]

[0002] In recent years, vehicles have been considered to be equipped with a function to capture images of the interior of the vehicle using an on-board camera. Images captured by the on-board camera may be used to estimate whether the driver is looking away or has their eyes closed using facial image recognition technology, or may be recorded. In addition, it has been proposed to equip vehicles with microphones and use voice recognition technology to enable drivers to operate car navigation systems and other devices hands-free while driving.

[0003] The use of cameras and microphones inside the vehicle may infringe on the user's privacy. Therefore, in-car applications that use image recognition or voice recognition functions can be switched on and off. In conventional technologies, the user often allows or disallows the use of these in-car applications the first time they are used, and the setting is maintained. Alternatively, the use of in-car applications may be set to on by default. To switch an in-car application on or off, it is necessary to perform an application on or off operation deep within the In-Vehicle Infotainment (IVI) hierarchy, for example.

[0004] Meanwhile, technologies have been developed to enable users to operate in-vehicle devices without having to perform cumbersome manual operations on IVIs and the like.

[0005] For example, Patent Document 1 discloses an image recognition device that recognizes human behavior based on signals input from a range image sensor capable of three-dimensional recognition of a subject, in which a first behavior detection space and a second behavior detection space that is at least partially located on a line connecting the range image sensor and the first behavior detection space and does not overlap with the first behavior detection space are determined in advance as spaces in which human behavior is recognized, and the result of recognizing human behavior in at least one of the first behavior detection space and the second behavior detection space is reflected in the output. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2017-097608 Summary of the Invention [Problem to be solved by the invention]

[0007] However, when applications are switched on and off using gestures or facial expression recognition alone, which does not involve physical operation, there is a risk of malfunction due to the characteristics of image recognition technology, which is affected by external light.Since applications involve the privacy of drivers and passengers by using facial images and voices, and applications may also be used to prevent drivers from looking away while driving, it is undesirable for malfunctions to occur when switching on and off.

[0008] Therefore, a determination system is provided that simply and reliably recognizes the driver's intention regarding specific processing such as application control. [Means for solving the problem]

[0009] The determination system of the present disclosure includes a detection unit that detects a driver's operation of an operating unit in a vehicle, a determination device that identifies a physical state of a part of the driver's body as a command action, and a vehicle control device that controls the vehicle. When the determination device identifies the driver's command action and the detection unit detects the driver's operation, the vehicle control device executes control content that is at least partially different from control content when the determination device does not identify the driver's command action and the detection unit detects the driver's operation.

[0010] According to the determination system of the present disclosure, when a driver issues an instruction through a command action, in order to execute a process based on the instruction, it is necessary to detect both a physical operation by the driver on an operation unit and detection of the command action based on a change in the driver's facial expression, etc., thereby making it possible to reliably reduce erroneous determination of the driver's intention compared to a case where the detection is based only on an image captured by camera 20. Therefore, a determination system is provided that simply and reliably recognizes the driver's intention regarding a specific process, such as application control. Furthermore, a single operation on the operation unit can selectively instruct multiple types of processes depending on whether a command action is performed and the type of command action, thereby improving user convenience. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of a determination system according to a first embodiment. [Figure 2] Schematic diagram showing a portion of the interior of a vehicle [Figure 3] Example of a table showing the correspondence between command actions and the on / off status of the target application [Figure 4] 1 is a flowchart illustrating an example of a process for controlling on / off of a target application by a determination system according to a first embodiment. [Figure 5] 1 is a flowchart illustrating a process for determining whether driving is suitable for the determination system according to the first embodiment; [Figure 6]FIG. 10 is a block diagram illustrating an example of the configuration of a determination system according to a second embodiment. [Figure 7] 10 is an example of a table showing the correspondence between the necessity of a command action and a third action and the on / off state of a target application according to the second embodiment. [Figure 8] 10 is another example of a table showing the correspondence between the necessity of the command action and the third action and the on / off of the target application according to the second embodiment. [Figure 9] 10 is a flowchart illustrating an example of a process for controlling on / off of a target application by a determination system according to a second embodiment (part 1). [Figure 10] 10 is a flowchart illustrating an example of a process for controlling on / off of a target application by a determination system according to a second embodiment (part 2). [Figure 11] FIG. 10 is a block diagram illustrating an example of the configuration of a determination system according to a third embodiment. [Figure 12] 10A and 10B are diagrams showing examples of images captured of command actions according to the third embodiment, and a schematic diagram showing the position of the IG button in each case; [Figure 13] 10 is a flowchart illustrating an example of a process for controlling on / off of a target application by a determination system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, a determination system and a determination device according to several embodiments will be described with reference to the drawings. The same reference numerals will be used to designate substantially the same components in several embodiments, and the description thereof will be omitted.

[0013] (First embodiment) A first embodiment will be described with reference to FIGS. 1 to 5. The determination system 1 includes a determination device 10 and a vehicle control device (hereinafter, sometimes referred to as an ECU (Electronic Control Unit)) 11. The determination device 10 has a function of turning on / off applications based on a user's intention. The determination device 10 has a function of turning on / off target applications that use personal information, or of permitting / denying transmission of personal information to target applications that use personal information, based on the user's intention. In this specification, personal information includes an image of a user in a manner that allows the user to be identified by image recognition or the like, and audio including speech by a user in a manner that allows the user to be identified by voiceprint recognition or the like. The vehicle control device 11 is a device mounted on a moving object such as a vehicle 200. The ECU 11 is connected to a camera 20, a microphone 30, a display device group 40, a door open / close switch 201, and an ignition (IG) button 202 mounted on the vehicle 200 via an in-vehicle network such as a CAN (Controller Area Network) or dedicated wiring.

[0014] The determination device 10 controls the on / off setting of the target application while the vehicle 200 is stopped. That is, the determination device 10 acquires an on / off instruction for the target application from the driver who is not driving the vehicle 200.

[0015] The camera 20 has a function of capturing an image of an object within the cabin of the vehicle 200. The imaged object may be, for example, the driver's face or the driver's upper body. A floodlight (not shown) has a function of illuminating the object to be captured when the camera 20 captures an image, thereby assisting in capturing a good image. The microphone 30 captures target audio within the cabin, such as speech from the driver or passengers, for use in operating a target in-vehicle application or for recording. The display device group 40 includes, for example, meters, in-vehicle infotainment (IVI), etc.

[0016] The camera 20 may be installed anywhere in the vehicle cabin as long as the subject to be imaged is included in the field of view at a desired position. For example, the camera 20 may be installed on the steering column 203, the instrument panel center 204, the overhead console 205, etc., as shown in FIG. 2.

[0017] The microphone 30 may be installed anywhere in the vehicle cabin as long as it can capture the target voice with the desired accuracy, for example, on the steering column 203, the overhead console 205, or the like in the vehicle cabin.

[0018] The determination system 1 further includes a storage area 12. The ECU 11 is responsible for controlling the entire determination device 10 and also has the function of controlling various electronic devices mounted on the vehicle 200. The ECU 11 is connected to a door opening / closing switch 201, an IG button 202, a camera 20, a microphone 30, a display device group 40, and the like via an in-vehicle network such as a Controller Area Network (CAN). The storage area 12 can be configured, for example, by a read-only memory (ROM), a random access memory (RAM), or a flash memory provided in the ECU 11. The storage area 12 may also be provided in an external server device (not shown) connected via the Internet.

[0019] The vehicle 200 is equipped with a door opening / closing switch 201 and an IG button 202. The door opening / closing switch 201 detects whether a door of the vehicle 200 is opened or closed. When the ECU power supply unit 111 is in an OFF state and a door of the vehicle 200 changes from a closed state to an open state, the ECU power supply unit 111 is turned ON. The IG button 202 accepts an operation from the driver to start or stop the engine. The ECU 11 switches the ignition of the vehicle 200 between ON and OFF.

[0020] The ECU 11 includes an ECU power supply unit 111, a CAN signal acquisition unit 112, a camera control unit 113, and a floodlight control unit 114. The ECU power supply unit 111 serves as a driving power source for the determination device 10. The CAN signal acquisition unit 112 is connected to a door open / close switch 201, an IG button 202, a camera 20, a microphone 30, a display device group 40, and the like via an in-vehicle network such as a CAN. The camera control unit 113 controls the operation of a camera system including the camera 20 and a floodlight (not shown). The camera control unit 113 may detect the constantly changing luminance distribution of the subject to be captured (in this case, the driver's face area) and control the imaging conditions of the camera system so that the face can be captured with a desirable degree of shading. For example, the camera control unit 113 may control the exposure time, gain, and light intensity of the floodlight of the camera 20.

[0021] In this embodiment, the camera 20, a floodlight (not shown), and the microphone 30 are connected to the ECU 11 through an in-vehicle network such as a CAN, but this is not limiting. For example, in other embodiments, the camera 20, the floodlight, and the microphone 30 may be connected to the ECU 11 by dedicated wiring, respectively.

[0022] The determination system 1 of this embodiment performs a first control when the first detection unit detects a user's operation on the first operation unit, and performs a second control different from the first control when the first detection unit detects the user's operation on the first operation unit and, at the same time, the second detection unit detects a specific physical state of a part of the user's body. The content of the second control performed by the determination system 1 may include some of the same processing as the first control, or may be entirely different from the processing of the first control. In this embodiment, the processing of the second control includes some of the same processing as the first control.

[0023] In this case, the second control includes a process different from the process of the first control. Here, the first control is a process that the ECU 11 normally executes on the vehicle 200 in response to an operation of the first operating unit by the user. The second control is a control that performs a different process in addition to the process that the ECU 11 normally executes in response to an operation of the first operating unit by the user.

[0024] The first operation unit is an operation unit provided in the vehicle 200 that is operated by the driver while the vehicle is stopped. In this embodiment, the first operation unit is the IG button 202, and the first detection unit is the CAN signal acquisition unit 112. The second detection unit is the determination device 10, which includes an image acquisition unit 13, an image recognition unit 15, and an intention interpretation unit 16 (described later). In this embodiment, the first control is a control for turning on the ignition. The second control is a control for turning on the ignition and switching the on / off setting of a target application. Note that in this embodiment, the second control is a process for switching the on / off setting of a target application in addition to the process of the first control, but is not limited thereto. For example, in other embodiments, the second control may be a process for switching the on / off setting of a driving assistance function of the vehicle 200, such as a collision avoidance assistance function or a preceding vehicle following function, or may be a process for switching the on / off setting of a function for tracking the driving route of the vehicle 200 in addition to the process of the first control.

[0025] The determination system 1 includes an image acquisition unit 13, a voice acquisition unit 14, an image recognition unit 15, an intention interpretation unit 16, and an ON / OFF control unit 17. The image acquisition unit 13, the voice acquisition unit 14, the image recognition unit 15, the intention interpretation unit 16, and the ON / OFF control unit 17 are realized in software by the ECU 11 executing a program stored in the memory area 12. Note that any one or all of the image acquisition unit 13, the voice acquisition unit 14, the image recognition unit 15, the intention interpretation unit 16, and the ON / OFF control unit 17 may be configured by hardware, or may be realized by a combination of software and hardware. The image recognition unit 15, the intention interpretation unit 16, and the ON / OFF control unit 17 constitute the determination device 10.

[0026] The image acquisition unit 13 has a function of acquiring images captured by the camera 20. The audio acquisition unit 14 has a function of acquiring audio picked up by the microphone 30. The image recognition unit 15 acquires images captured by the camera 20 via the image acquisition unit 13 and recognizes the state of the captured subject (in this case, the driver's face, hands, etc.) appearing in each image. The intention interpretation unit 16 determines the user's command action regarding the on / off of a target application from the physical state of the driver's body parts appearing in a predetermined number of images recognized by the image recognition unit 15 and converts the determined command action into an on / off signal for the target application. The on / off control unit 17 sets the target application on / off based on the on / off signal converted by the intention interpretation unit 16. The command action refers to a change over time in the physical state of the driver's body parts associated with the on / off of a specific target application. The driver's body parts include, for example, the driver's face and the driver's hands.

[0027] The intention interpretation unit 16 stores, in a ring buffer, information on the physical state of parts of the driver's body recognized by the image acquisition unit 13 from a predetermined number of images, based on the most recent image acquired by the image acquisition unit 13. When the IG button 202 is pressed, the intention interpretation unit 16 retrieves the physical state information stored in the ring buffer as input information and determines the user's command action. The predetermined number of images used by the intention interpretation unit 16 can be set arbitrarily, for example, to plus or minus 20% of the number of images captured in a predetermined time. The predetermined time can be, for example, the average time it takes the driver to operate the operating unit, in this case the IG button 202. For example, if the average time it takes the driver to operate the IG button is one second and the camera 20 captures 30 images per second, the predetermined number of images may be set within a range of 24 to 36.

[0028] In this specification, turning off the target application includes switching the target application from an on state to an off state, maintaining the off state, or setting the target application to not allow the target application to transmit personal information used by the target application. Turning on the target application includes switching the target application from an off state to an on state, maintaining the on state, or setting the target application to allow the target application to transmit personal information used by the target application. Personal information includes, for example, images captured by the camera 20 and audio picked up by the microphone 30. Unless otherwise specified in this specification, images refer to images captured by the camera 20, and audio refers to audio picked up by the microphone 30.

[0029] FIG. 3 shows an example of a chart of on / off setting information for target applications associated with command actions. The on / off setting information is information on the on / off setting of each target application. The memory area 12 stores a chart, such as the example shown in FIG. 3, that associates command actions with on / off setting information for target applications. The intention interpretation unit 16 refers to the chart stored in the memory area 12 and converts the information for the identified command action into an on / off signal for the target application. The command action and the target application can be set so that they are easy for the driver to intuitively perceive. For example, if the target application uses images captured by the camera 20, i.e., visual information, the command action may be associated with a command action related to the driver's eyes. Furthermore, if the target application uses audio captured by the microphone 30, i.e., the driver's speech, the command action may be associated with a command action related to the driver's mouth. Furthermore, the command action for turning on the target application may use a physical state of the user's body, such as opening or highlighting a part of their body. The command action for turning off the target application may use a physical state of the driver's body, such as closing or hiding a part of their body. Furthermore, a command action of the driver hiding his / her entire face with his / her hands may be linked to setting information for turning off all of the target applications. Note that in this specification, an application that uses an image captured by the camera 20 may be referred to as an image-using application or an image-using app, and an application that uses sound picked up by the microphone 30 may be referred to as an audio-using application or an audio-using app.

[0030] For example, consider an image captured when the driver intentionally presses the IG button 202 with their eyes closed. In this case, the image recognition unit 15 recognizes that the driver's eyes are closed in each of the multiple images. The intention interpretation unit 16 chronologically collects multiple image recognition results from the image recognition unit 15 for images captured over a predetermined period of time and identifies that the driver has taken a command action of closing their eyes. The intention interpretation unit 16 converts the recognition results of the multiple images into an electrical signal that turns off an image-using application associated with the command action. Since the CAN signal acquisition unit 112 acquired the operation of the IG button 202 when the image for which the command action was identified was captured, the on / off control unit 17 outputs a signal that turns off the image-using application.

[0031] Consider, for example, images captured when a driver intentionally closes and opens his or her eyes repeatedly, blinking, and then presses the IG button 202. In this case, the image recognition unit 15 recognizes that the driver has his or her eyes closed or open for each of the multiple images. The intention interpretation unit 16 chronologically collects multiple image recognition results from the image recognition unit 15 for images captured over a predetermined period and identifies that the driver has performed a command action of blinking his or her eyes. The intention interpretation unit 16 converts the recognition results of the multiple images into an electrical signal that turns on an image-using application associated with the command action. Since the CAN signal acquisition unit 112 has acquired the operation of the IG button 202 when capturing an image for which a command action has been identified, the on / off control unit 17 outputs a signal that turns on the image-using application.

[0032] The on / off setting of the target application can be determined based on a default setting previously determined for each target application or an on / off setting set at the previous time of riding, unless the determination system 1 performs processing. In this embodiment, if no command action is detected, the on / off control unit 17 sets the on / off setting according to the default setting.

[0033] As shown in FIG. 1, the ECU 11 includes a driving suitability determination unit 50. The driving suitability determination unit 50 is an example of a target application. The driving suitability determination unit 50 uses an image captured by the camera 20 as personal information. The driving suitability determination unit 50 has a function of recognizing an image of a subject captured by the camera system, in this case the driver's face, and detecting an unsuitable state for driving, such as the driver looking away or falling asleep while driving, or an incapacitated state for driving, such as a coma, and alerting the driver.

[0034] The driving suitability determination unit 50 includes a state estimation unit 51, an alarm determination unit 52, and an alarm output processing unit 53. The state estimation unit 51 calculates the driver's state using the driver's facial physical state calculated by the image recognition unit 15 as input information. In this specification, the facial physical state refers to the physical state of the face, including the position of the face, the direction of the face, the degree of eye opening and closing, the direction of gaze, etc. The driver state includes at least "normal" and "unsuitable." "Normal" refers to a state in which the driver's face is facing forward. "Unsuitable" refers to, for example, a state in which the driver's eyes are closed for more than two seconds, the driver is dozing, or the driver's face is out of the field of view of the camera 20 for more than 10 seconds, i.e., the driver's posture is disrupted. The alarm determination unit 52 determines whether to output an alarm based on the driver's state calculated by the state estimation unit 51. For example, if the driver's state is "normal," the alarm determination unit 52 determines that the driver is in a suitable state for driving and therefore no alarm is necessary. If the driver's condition is "unsuitable," the warning determination unit 52 determines that the driver is not suitable for driving and therefore a warning is required. Note that the state estimation unit 51 and the image recognition unit 15 may be configured in the same manner.

[0035] The alarm output processing unit 53 sends an instruction to output an alarm to the alarm output means based on the determination result of the alarm determination unit 52. The alarm output means can be realized by various means such as optical means, auditory means, tactile means, olfactory means, etc. For example, the alarm output means may be realized by the display device group 40 including meters and IVI.

[0036] (Flowchart of on / off control) The process when the determination system 1 controls the on / off of an application will be described with reference to Figure 4. In this case, it is assumed that the target application is set to on by default at the start of Figure 4. In the flowchart of Figure 4, the process by each component is described as the process by the determination system 1.

[0037] In step S101, the determination system 1 determines whether the door opening / closing switch 201 has detected that the door is open. If the door opening / closing switch 201 has not detected that the door is open (NO in step S101), the determination system 1 repeats the processing of step S201. If the door opening / closing switch 201 has detected that the door is open (YES in step S101), the determination system 1 proceeds to the processing of step S102.

[0038] In step S102, the determination system 1 turns on the ECU power supply unit 111. In step S103, the determination system 1 starts capturing images by driving the camera 20 using the camera control unit 113. In step S104, the determination system 1 acquires images captured by the camera 20. In step S105, the determination system 1 recognizes the physical state of the driver's body for each of the acquired images.

[0039] In step S106, the determination system 1 determines whether the IG button 202 has been operated and the CAN signal acquisition unit 112 has received a command to transition to the IG-ON state. If the command to transition to the IG-ON state has not been received (NO in step S106), the determination system 1 returns the process to step S104. If the command to transition to the IG-ON state has been received (YES in step S106), the determination system 1 proceeds to step S107. In step S107, the determination system 1 turns on the ignition.

[0040] In step S108, the judgment system 1 extracts the recognition results of the physical state of the driver's body for the multiple images stored in the ring buffer as input information, and determines whether the driver has performed a command action and the type of command action by referring to the chart stored in the memory area 12.

[0041] In step S109, the determination system 1 determines whether a command behavior has been identified in step S108. If a command behavior has not been identified (NO in step S109), the determination system 1 ends the process (END). If a command behavior has been identified (YES in step S109), the determination system 1 proceeds to step S110.

[0042] In step S110, the determination system 1 refers to the information stored in the memory area 12 and identifies an on / off instruction for the target application associated with the command behavior determined in step S106. If the instruction is to turn on the target application (APPLICATION ON in step S110), the determination system 1 ends the process (END). If the instruction is to turn off the application (APPLICATION OFF in step S110), the determination system 1 proceeds to step S111.

[0043] In step S111, the determination system 1 turns off the target application or stops sending personal information to the target application. In this embodiment, the determination system 1 stops sending personal information, such as image information captured by the camera 20 or audio information acquired by the microphone, to the target application. Thereafter, the determination system 1 ends the processing (END). In this way, the determination system 1 sets the target application to on or off.

[0044] 4, for simplicity, there is only one target application, but this is not limiting. In other embodiments, one behavior command may be associated with on / off commands for multiple target applications.

[0045] (Flowchart of the driving suitability determination section) The processing of the driving suitability determination unit 50 will be described with reference to the flowchart shown in Fig. 5. At the start of Fig. 5, the ECU 11 is supplied with driving power from the ECU power supply unit 111, and the driving suitability determination unit 50 as a target application is set to on by the on / off control unit 17.

[0046] In step S201, the driving suitability determination unit 50 acquires an image from the camera 20. The camera 20 captures images multiple times within a predetermined period, for example, approximately 30 times per second. In step S202, the driving suitability determination unit 50 determines whether an image signal has been received. If an image signal has not been received (NO in step S202), the driving suitability determination unit 50 proceeds to step S203. In step S203, the driving suitability determination unit 50 outputs a signal to an output device, in this case the display device group 40, notifying that there is no image.

[0047] If an image signal has been received (YES in step S202), the driving suitability determination unit 50 proceeds to step S204. In step S204, the driving suitability determination unit 50 estimates the driver's state based on the acquired image. In step S205, the driving suitability determination unit determines whether a warning is necessary based on the estimated driver's state. If a warning is not necessary (NO in step S205), the driving suitability determination unit 50 returns the process to step S201. If a warning is necessary (YES in step S205), the driving suitability determination unit 50 proceeds to step S206. In step S206, the driving suitability determination unit 50 outputs a warning signal to an output device, in this case the display device group 40. In this way, the processing by the driving suitability determination unit is executed (END).

[0048] Here, it is conceivable to grasp the driver's instructions based solely on capturing and identifying the driver's command actions, and then control, for example, the on / off of a target application. However, control based solely on camera images is affected by external lighting conditions and other factors, and there is a risk of erroneous judgments of around 5%.

[0049] In contrast, according to the present embodiment described above, the determination system 1 includes a CAN signal acquisition unit 112 as a detection unit, a determination device 10, and a vehicle control device (ECU) 11. The CAN signal acquisition unit 112 detects a driver's operation of an IG button 202 as an operation unit in the vehicle 200. The determination device 10 identifies a physical state of a part of the driver's body as a command action. The ECU 11 controls the vehicle 200. When the determination device 10 identifies the driver's command action and the CAN signal acquisition unit 112 detects the driver's operation, the ECU 11 executes processing that is at least partially different from processing executed when the determination device 10 does not identify the driver's command action and the CAN signal acquisition unit 112 detects the driver's operation.

[0050] According to this, when the driver issues an instruction by a command action, in order to execute processing based on the instruction, it is necessary to detect both the driver's physical operation on the operation unit and the command action based on a change in the driver's facial expression, etc., thereby making it possible to reliably suppress erroneous determination of the driver's intention compared to when it is based only on an image captured by the camera 20. Furthermore, since a single operation on the operation unit can selectively instruct whether or not a command action is performed and a plurality of types of processing depending on the type of command action, user convenience is improved.

[0051] When the determination device 10 identifies the driver's command action and the CAN signal acquisition unit 112 as a detection unit detects the driver's operation, the ECU 11 performs additional processing in addition to the processing performed when the determination device 10 does not identify the driver's command action and the CAN signal acquisition unit 112 detects the driver's operation.

[0052] According to this, since a process based on a command action is added to the normal process, i.e., turning on the ignition, which is performed in response to an operation on an operation unit, for example, the IG button 202, the driver can easily instruct the ECU 11 to perform multiple control operations with a single operation, thereby improving user convenience.

[0053] According to the determination system 1 of this embodiment, the operation unit is a start operation unit of the vehicle 200, for example, the IG button 202. The determination device 10 has an image acquisition unit 13 that acquires images from a camera 20 that captures images of the interior of the vehicle 200, an image recognition unit 15 that detects the physical state of a part of the driver's body from the image acquired by the image acquisition unit 13, and an intention interpretation unit 16 that determines the driver's command action based on the change over time in the physical state of the part of the driver's body detected by the image recognition unit 15.

[0054] According to this, when the start operating unit is turned on, the vehicle 200 is stopped, i.e., not yet running, and the driver is not operating the vehicle, so the operation of the operating unit is executed reliably and safely. Therefore, it is possible to execute control that more reliably and simply reflects the driver's intentions while ensuring the safety of the driver and the running of the vehicle 200.

[0055] For example, if vehicle 200 is an electric vehicle, the operation unit may be a push start button, instead of IG button 202, that is operated by the driver when starting the electric vehicle.

[0056] Here, conventionally, turning on / off an application used in the vehicle 200 often requires a user to operate and set it at a deep level in the IVI.

[0057] In response to this, the determination system 1 includes an on / off control unit 17 that controls the on / off setting of the target application. When the determination device 10 identifies the driver's command action and detects the driver's operation of the IG button 202 as a start operation unit, the on / off control unit 17 controls the on / off setting of the target application.

[0058] Before the vehicle 200 starts to travel, the driver always operates the start operating unit. Therefore, according to this embodiment, the driver can simply and reliably issue a command for a specific process by simply performing a command action when performing an action that the driver always performs. Furthermore, the driver can simply and reliably set the on / off settings of target applications used in the vehicle. This improves convenience and safety for the driver.

[0059] The determination system 1 includes a memory area 12 that stores on / off setting information of a target application in association with a command action. The intention interpretation unit 16 refers to the on / off setting information of the target application associated with the command action stored in the memory area 12, and converts information on changes over time in the physical state of a part of the driver's body into an on / off signal for the target application.

[0060] According to this, the on / off signal for the target application is generated by referring to the information stored in memory area 12, so that by updating the information in memory area 12, the type of target application and command actions can be added or changed as appropriate.

[0061] Although the determination device 10 of this embodiment is configured with the ECU 11 on one circuit board, it may be configured with a combination of multiple circuit boards. Furthermore, the determination system 1 of this embodiment may be configured to include multiple physically separated components. Furthermore, the determination system 1 may be configured with multiple in-vehicle devices or external devices brought in by the driver from outside the vehicle. For example, one or more of the image recognition unit 15, intention interpretation unit 16, and on / off control unit 17 may be realized by an external device such as a smartphone or tablet device owned by the driver or a passenger.

[0062] Furthermore, in this embodiment, when the on / off setting information of a target application linked to a command action is “off,” the determination device 10 turns off the target application or stops sending personal information to the target application. However, this is not limited to this. In other embodiments, in such cases, the determination device 10 may send alternative information to the target application instead of personal information. The alternative information is information that does not identify an individual and includes an alternative image or an alternative sound. For example, the determination device 10 may send a black screen, such as a full-black screen, as an alternative image to the image-using application instead of an image captured by the camera 20. The determination device 10 may also send a 1 kHz sine wave audio signal as an alternative sound instead of the sound captured by the microphone 30. Even if the image-using application receives an alternative image such as a black screen, it cannot detect the driver's face and will not, for example, output a warning about inattentive driving. Even if the audio-using application receives an alternative sound such as a sine wave sound, it cannot detect the driver's voiceprint. Even if the image-using application or the audio-using application transmits an alternative image or an alternative sound to an external communication device, it still cannot identify an individual. Even in this way, the determination device 10 can protect the privacy of the driver in accordance with his or her intention.

[0063] (Second embodiment) A second embodiment will be described with reference to FIGS. 6 to 10 . In this embodiment, when at least some of the target applications are turned on or off, the determination system 1 requires the third detection unit to simultaneously detect a third behavior of the driver in addition to detecting the driver's operation of the first operation unit and the driver's command behavior. For some target applications, the driver's turning on or off may affect the safe driving of the vehicle 200. In this case, it is undesirable for a determination that differs from the user's intention when turning on or off the target application. Therefore, in this embodiment, by requiring the third detection unit to detect the driver's third behavior when turning on or off at least some of the target applications, the driver's intention can be reliably reflected. A target application that requires the third detection unit to detect the third behavior when turning on or off the target application is referred to as a “specific target application.”

[0064] For example, one example of a specific target application is an image-based application related to improving vehicle safety, such as an application for preventing inattentive driving. Since turning off such a target application tends to reduce vehicle safety, it is desirable to turn it off only if the driver is certain that he or she intends to turn it off. Another example of a specific target application is a voice-based application, such as a chat dialogue application that uses AI. Turning on such a target application may impair the driver's attention to driving. Therefore, it is desirable to turn on a specific target application that may reduce the driver's attention to driving only if the driver understands this and is certain that he or she intends to turn it on.

[0065] 6, vehicle 200 includes buckle sensor 206 and brake pedal sensor 207. Buckle sensor 206 has a function of detecting that a seat belt (not shown) of a driver's seat is inserted into a buckle. Brake pedal sensor 207 has a function of detecting that a brake pedal (not shown) is depressed by the driver.

[0066] As shown in FIG. 6 , the determination system 1 further includes a third behavior acquisition unit 18 and a third behavior determination unit 19. The third behavior acquisition unit 18 is the third detection unit and has a function of acquiring a driver's behavior with respect to the second operation unit. The third behavior determination unit 19 has a function of determining whether or not a target application can be permitted to be turned on or off depending on whether or not the driver's third behavior acquired by the third behavior acquisition unit 18 is present. That is, if the signal converted by the intention interpretation unit 16 includes a signal to turn on or off a specific target application, the third behavior determination unit 19 determines whether or not the third behavior acquisition unit 18 has detected the third behavior simultaneously with the detection of the command behavior. If the third behavior acquisition unit 18 detects the third behavior simultaneously with the detection of the command behavior, the on / off control unit 17 permits the specific target application to be turned on or off. If the third behavior acquisition unit 18 does not detect the third behavior simultaneously with the detection of the command behavior, the on / off control unit 17 does not permit the specific target application to be turned on or off.

[0067] For a specific target application that requires detection of a third behavior for on / off setting, if the third behavior acquisition unit 18 does not acquire the third behavior, the specific target application is set to on or off according to the default setting or the setting at the time of the previous ride. In this embodiment, if the third behavior acquisition unit 18 does not acquire the third behavior, the on / off control unit 17 sets the specific target application to on / off according to the default setting. Furthermore, for a target application that does not require detection of a third behavior for on / off setting, the on / off control unit 17 sets the target application to on / off based on the processing results of the image recognition unit 15 and the intention interpretation unit 16, regardless of whether the third behavior acquisition unit 18 acquires the third behavior.

[0068] The third action is an action by the driver with respect to a second operation unit provided in the vehicle 200 other than the first operation unit. The second operation unit is an operation unit other than the first operation unit, and may be, for example, a seat belt buckle on the driver's seat or a brake pedal. The third action acquisition unit 18 may acquire from the buckle sensor 206 a signal that the seat belt has been inserted into the seat belt buckle on the driver's seat, or may acquire from the brake pedal sensor 207 a signal that the brake pedal has been depressed. In this embodiment, the second operation unit is the buckle sensor 206, and the third action acquisition unit 18 acquires from the buckle sensor 206 a signal that the seat belt has been inserted.

[0069] Specific target applications that require the detection of a third behavior when turned on or off can be arbitrarily set. In the example shown in FIG. 7, the detection of a third behavior is required when an image-using application is turned off, and the detection of a third behavior is not required when an image-using application is turned on. Furthermore, the detection of a third behavior is required when a voice-using application is turned on, and the detection of a third behavior is not required when a voice-using application is turned off. Note that if a command behavior is linked to the on / off settings of multiple target applications, if the detection of a third behavior is required to set one of the target applications, the detection of the third behavior is required to execute the on / off setting based on the command behavior, even if the detection of the third behavior is not required for the settings of the remaining target applications. In the example shown in FIG. 8, the detection of a third behavior is required when turning on or off both an image-using application and a voice-using application. Note that if there are multiple image-using applications, the need for the detection of a third behavior can be set for each of the on / off settings. Similarly, if there are multiple voice-using applications, the need for the detection of a third behavior can be set for each of the on / off settings.

[0070] As shown in FIGS. 7 and 8 , the memory area 12 stores a chart linking each command action with whether or not detection of a third action is required and the on / off setting of a target application. When the on / off control unit 17 acquires information about a command action determined by the intention interpretation unit 16, it calls up the chart and acquires information about whether or not detection of the third action is required and the on / off setting of the associated target application. If the command action determined by the intention interpretation unit 16 is related to the on / off setting of a specific target application, i.e., if the on / off setting of the target application linked to the command action requires the detection of the third action, the third action determination unit 19 determines whether or not the third action has been detected by the third action acquisition unit 18. If the target application linked to the command action is a specific target application and the third action determination unit determines that the third action has been detected, the on / off control unit 17 authorizes the on / off setting of the specific target application according to the chart stored in the memory area 12. That is, the on / off control unit 17 allows the specific target application to be set to on or off according to the user's intention indicated by the command action.

[0071] 9 and 10, the process performed by the determination system 1 to control the on / off of a target application will be described, particularly the parts that differ from the flowchart shown in Fig. 4. In the examples shown in Fig. 9 and 10, the default setting of the image-using application or audio-using application as the target application may be either on or off at the start.

[0072] If a command behavior is identified in step S108 (YES in step S109), the determination system 1 proceeds to step S121. In step S121, the determination system 1 refers to the chart stored in the memory area 12 and determines whether the on / off setting associated with the command behavior identified in step S108 requires the detection of a third behavior. That is, the determination system 1 determines whether the command behavior instructs the on / off setting of a specific target application whose on / off setting requires the detection of a third behavior. If the on / off setting of the target application associated with the command behavior identified in step S108 does not require the detection of the third behavior (NO in step S121), the determination system 1 proceeds to step S123 of the flowchart shown in FIG. 10. If the on / off setting of the target application associated with the command behavior identified in step S108 requires the detection of the third behavior (YES in step S121 of FIG. 9), the determination system 1 proceeds to step S122.

[0073] In step S122, the determination system 1 determines whether the third behavior acquisition unit 18 has detected the third behavior. If the third behavior has not been detected (NO in step S122), the determination system 1 terminates the processing (END). In this case, the third behavior required for the on / off setting of the specific target application has not been detected, so the determination system 1 does not change the on / off setting of the specific target application. In this case, the default on / off setting for the specific target application is maintained as is.

[0074] When the third behavior acquisition unit 18 detects the third behavior (YES in step S122), the determination system 1 advances the processing to step S123 in the flowchart shown in FIG.

[0075] In step S123 of Figure 10, the determination system 1 determines whether the on / off signal linked to the command behavior identified in step S108 is an on signal related to a voice-based application, an off signal, or a signal not related to a voice-based application.

[0076] If the on / off signal is an on signal for the voice-using application (ON signal in step S123), the determination system 1 proceeds to step S124. In step S124, the determination system 1 turns on the voice-using application and permits audio transmission to the voice-using application. This starts audio transmission to the voice-using application. If the voice-using application was originally set to on and audio transmission to the voice-using application was permitted, the voice application remains on and audio transmission to the voice-using application continues. The determination system 1 then proceeds to step S126.

[0077] If the on / off signal is an off signal for the voice-using application (OFF signal in step S123), the determination system 1 proceeds to step S125. In step S125, the determination system 1 turns off the voice-using application or stops sending voice to the voice-using application. In this embodiment, the determination system 1 disallows sending voice to the voice-using application. This stops sending voice to the voice-using application. If the voice-using application was originally off or sending voice to the voice-using application was disallowed, the voice-using application continues to be off, or no voice is sent to the voice-using application. The determination system 1 then proceeds to step S126.

[0078] If the on / off signal is not a signal related to a voice-using application (no signal in step S123), the determination system 1 does not change the on / off setting for the voice-using application. In this case, the default on / off setting is applied to the voice-using application. The determination system 1 then proceeds to step S126.

[0079] In step S126, the determination system 1 determines whether the on / off signal associated with the command action identified in step S108 is an on signal or an off signal for an image-using application, or whether there is no signal for an image-using application.

[0080] If the on / off signal is an on signal for the image-using application (ON signal in step S126), the determination system 1 proceeds to step S127. In step S127, the determination system 1 turns on the image-using application and permits the transmission of images to the image-using application. This starts the transmission of images to the image-using application. If the audio-using application was originally set to on and the transmission of images to the image-using application was permitted, the determination system 1 maintains the on setting of the audio-using application and continues the transmission of images to the image-using application. Thereafter, the determination system 1 ends the processing (End in FIG. 9).

[0081] If the on / off signal is an off signal for the image-using application (OFF signal in step S126), the determination system 1 proceeds to step S128. In step S128, the determination system 1 turns off the image-using application or does not permit image transmission to the image-using application. In this embodiment, the determination system 1 does not permit image transmission to the image-using application. This stops image transmission to the image-using application. If the image-using application is set to off by default or image transmission to the image-using application is not permitted, the determination system 1 continues to maintain the image-using application set to off or does not permit image transmission to the image-using application. Thereafter, the determination system 1 ends the processing (End in FIG. 9).

[0082] If the on / off signal is not a signal related to an image-using application (no signal in step S126), the determination system 1 ends the process (end of FIG. 9). In this case, the default on / off setting is applied to the image-using application.

[0083] According to the present embodiment described above, the determination system 1 includes a third behavior determination unit 19 that determines whether or not a third behavior, which is a driver behavior other than an operation on the first operation unit, has occurred. At least some of the target applications are specific target applications that require the third behavior determination unit to determine that the third behavior has occurred in either one or both of the on and off settings. The on / off control unit 17 performs on / off setting of the target application when the third behavior determination unit 19 determines that the third behavior has occurred.

[0084] According to this, by using the determination system 1 to set stricter conditions for turning on or off target applications that require special attention for privacy or safety reasons, the on / off setting can be changed reliably to reflect the driver's intention. Therefore, erroneous setting of any application can be further suppressed, and convenience and safety for the driver can be improved.

[0085] (Third embodiment) The third embodiment will be described with reference to Fig. 11 to Fig. 13. As shown in Fig. 11, the determination system 1 of this embodiment includes an individual determination unit 70. The individual determination unit 70 has a function of determining whether the command action identified by the intention interpretation unit 16 is an action of the driver himself / herself. That is, the individual determination unit 70 determines whether the part of the body executing the physical state recognized by the image recognition unit 15 is a part of the body of the driver himself / herself. If the individual determination unit 70 determines that the command action is not an action of the driver himself / herself, the on / off control unit 17 does not execute the on / off setting.

[0086] For example, when the command action is an action using the driver's hands, the identity determination unit 70 determines whether the hands belong to the driver. Command actions using the driver's hands include, for example, hiding the eyes with the hands, hiding the mouth with the hands, and further hiding the entire face with the hands, as shown in Fig. 12 .

[0087] In this case, the identity determination unit 70 can determine whether the hand used to make the command action is the driver's own, based on the relative positional relationship between the operating unit and the driver. That is, if the operating unit, for example, the IG button 202, is located to the right of the driver, it is natural for the driver to operate the IG button 202 with his / her right hand, and therefore the left hand is used to make the command action. On the other hand, if the operating unit, for example, the IG button 202, is located to the left of the driver, it is natural for the driver to operate the IG button 202 with his / her left hand, and therefore the right hand is used to make the command action. In FIG. 12, the symbol 202R denotes the IG button 202 located on the right hand side of the driver when the driver is sitting in the driver's seat, and the symbol 202L denotes the IG button 202 located on the left hand side of the driver when the driver is sitting in the driver's seat.

[0088] For example, Figure 12(a) shows an example of an image captured when camera 20, a narrow-angle camera, is placed on steering column 203. When IG button 202R is placed on the right side as seen from the driver, the driver will usually use their left hand to make a command action to cover their mouth. The person determination unit 70 determines whether the hand covering the mouth is the left hand, and if it is the left hand, determines that the command action is made by the person.

[0089] FIG. 12(b) shows an example of an image captured when camera 20, which is a wide-angle camera, is placed on overhead console 205. When IG button 202L is placed on the left side as viewed from the driver, the driver will usually use his right hand to make a command action to cover his mouth. The person determination unit 70 determines whether the hand covering the mouth is the right hand, and if it is the right hand, determines that the command action is made by the driver. Furthermore, when camera 20 is a wide-angle camera, it may perform image recognition of the connection between the right hand and the driver's torso, and determine whether the right hand belongs to the driver's body, i.e., whether it does not belong to the body of a third party other than the driver.

[0090] 12(c) shows an example of an image captured when camera 20, which is a narrow-angle camera, is placed in instrument panel center 204. When IG button 202L is placed on the left side as seen from the driver, the driver will usually use his right hand to make a command action to cover his eyes. Personality determination unit 70 determines whether the hand covering the eyes is the right hand, and if it is the right hand, determines that the command action is made by the driver.

[0091] FIG. 12(d) shows an example of an image captured when camera 20, which is a wide-angle camera, is placed in instrument panel center 204. When IG button 202L is placed on the left side as seen from the driver, the driver usually uses his right hand to command the driver to cover his eyes. The person determination unit 70 determines whether the hand covering the eyes is the right hand, and if it is the right hand, determines that the command action is made by the driver. Furthermore, when camera 20 is a wide-angle camera, it may determine whether the right hand belongs to the driver, i.e., whether it is the right hand of a third party other than the driver.

[0092] With reference to the flowchart shown in Fig. 13, the process performed by the determination system 1 to control the on / off of a target application will be described, particularly the parts that differ from the flowchart shown in Fig. 4. In the example shown in Fig. 13, it is assumed that the image-using application or audio-using application serving as the target application is set to on by default at the start.

[0093] If a command action is identified in step S108 (YES in step S109), the determination system 1 proceeds to step S301. In step S301, the determination system 1 determines whether the identified command action is the driver's own action. If the determination system 1 determines that a command action is identified but is not the driver's own action (not the driver in step S301), the determination system 1 terminates the processing (END). In these cases, the target application remains on according to the default setting. If the determination system 1 determines that the command action is the driver's own action (the driver is the driver in step S301), the determination system 1 proceeds to step S110. As a result, the on / off setting of the target application is set based on the driver's intention.

[0094] According to the present embodiment described above, the determination system 1 includes an identity determination unit 70 that determines whether or not the body part executing the physical state recognized by the image recognition unit 15 is a part of the driver's body. When the identity determination unit 70 determines that the body part is a part of the driver's body, the on / off control unit 17 determines whether or not the target application is on or off.

[0095] This makes it possible to prevent a third party from accidentally performing an action similar to a command action, such as covering the driver's mouth or eyes, at the same time that the driver operates the operation unit, and mistakenly determining that the action is a command action by the driver and turning on / off the target application. This makes it possible to more accurately control the on / off of the target application, reflecting the driver's own intentions.

[0096] Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments or structures. The present disclosure also encompasses various modifications and equivalent modifications. In addition, various combinations and forms, including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure.

[0097] The controller and the method described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and the method described herein may be implemented by a special-purpose computer configured with a processor comprising one or more dedicated hardware logic circuits. Alternatively, the controller and the method described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transitory tangible storage medium as instructions executed by a computer.

[0098] In addition to the inventions set forth in the claims, the present disclosure further includes the following inventions.

[0099] [1] a detection unit that detects an operation of a driver on an operation unit (202) in a vehicle (200); a determination device (10) for identifying a physical state of a part of the driver's body as a command action; a vehicle control device (11) that controls the vehicle, When the determination device identifies the command action of the driver and the detection unit detects an operation of the driver, the vehicle control device executes a process that is at least partially different from a process executed when the determination device does not identify the command action of the driver and the detection unit detects an operation of the driver. Judging system.

[0100] [2] When the determination device identifies the command action of the driver and the detection unit detects the operation of the driver, the vehicle control device performs an additional process in addition to the process performed when the determination device does not identify the command action of the driver and the detection unit detects the operation of the driver. The determination system described in [1].

[0101] [3] the operation unit is a start operation unit of the vehicle, The determination device includes an image acquisition unit (13) that acquires an image from a camera (20) that captures an image of the interior of the vehicle, an image recognition unit (14) that detects a physical state of a part of the driver's body from the image acquired by the image acquisition unit, and an intention interpretation unit (15) that determines the command action of the driver based on a change over time in the physical state of the part of the driver's body detected by the image recognition unit. The determination system according to [1] or [2].

[0102] [4] An on / off control unit (17) for controlling the on / off setting of the target application; the on / off control unit performs on / off setting of the target application when detecting an operation of the driver on the start operation unit in a case where the determination device identifies the command action. A determination system according to any one of [1] to [3].

[0103] [5] a storage area for storing the on / off setting information of the target application and the command action in association with each other; the intention interpretation unit refers to on / off setting information of the target application linked to the command action stored in the storage area and converts information on a change over time in the physical state of a part of the driver's body into an on / off signal of the target application; A determination system according to any one of [1] to [4].

[0104] [6] a third behavior determination unit (19) that determines whether or not a third behavior, which is a behavior of the driver other than an operation on the operation unit, is present; At least some of the target applications are specific target applications that require the third behavior determination unit to determine that the third behavior has occurred in either one or both of an on setting and an off setting, the on / off control unit performs on / off setting of the target application when the third behavior determination unit determines that the third behavior has occurred. [1] to [5]. A determination system according to any one of [1] to [5].

[0105] [7] an identity determination unit (70) that determines whether or not the part of the body that executes the physical state recognized by the image recognition unit is a part of the body of the driver himself; the on / off control unit performs on / off setting of the target application when the person determination unit determines that the target application is a part of the driver's body. A determination system according to any one of [1] to [6]. [Explanation of symbols]

[0106] 1...Determination system, 10...Determination device, 11...Vehicle control unit (ECU), 112...CAN signal acquisition unit (detection unit), 12...Memory area, 13...Image acquisition unit, 15...Image recognition unit, 16...Intention interpretation unit, 17...On / off control unit, 19...Third behavior determination unit, 20...Camera, 70...Personal identification unit, 200...Vehicle, 202...IG button (operation unit)

Claims

1. a detection unit (112) that detects an operation of a driver on an operation unit (202) in a vehicle (200); a determining device (10) for identifying a physical state of a part of the driver's body as a command behavior; a vehicle control device (11) that controls the vehicle, When the determination device identifies the command action of the driver and the detection unit detects an operation of the driver, the vehicle control device executes a process that is at least partially different from a process executed when the determination device does not identify the command action of the driver and the detection unit detects an operation of the driver. Judging system.

2. When the determination device identifies the command action of the driver and the detection unit detects the operation of the driver, the vehicle control device performs an additional process in addition to the process performed when the determination device does not identify the command action of the driver and the detection unit detects the operation of the driver. The determination system according to claim 1 .

3. the operation unit is a start operation unit of the vehicle, The determination device includes an image acquisition unit (13) that acquires an image from a camera (20) that captures an image of the interior of the vehicle, an image recognition unit (15) that detects a physical state of a part of the driver's body from the image acquired by the image acquisition unit, and an intention interpretation unit (16) that determines the command action of the driver based on a change over time in the physical state of the part of the driver's body detected by the image recognition unit. The determination system according to claim 1 or 2.

4. An on / off control unit (17) that controls on / off settings of a target application; the on / off control unit performs on / off setting of the target application when detecting an operation of the driver on the start operation unit in a case where the determination device identifies the command action. The determination system according to claim 3 .

5. a storage area (12) for storing the on / off setting information of the target application and the command action in association with each other; the intention interpretation unit refers to on / off setting information of the target application linked to the command action stored in the storage area and converts information of a change over time in the physical state of a part of the driver's body into an on / off signal of the target application; The determination system according to claim 4.

6. a third behavior determination unit (19) that determines whether or not a third behavior, which is a behavior of the driver other than an operation on the operation unit, is present; At least some of the target applications are specific target applications that require the third behavior determination unit to determine that the third behavior has occurred in either one or both of an on-setting and an off-setting, the on / off control unit performs on / off setting of the specific target application when the third behavior determination unit determines that the third behavior has occurred. The determination system according to claim 4 .

7. an identity determination unit (70) that determines whether or not the part of the body that executes the physical state recognized by the image recognition unit is a part of the body of the driver himself; the on / off control unit performs on / off setting of the target application when the person determination unit determines that the target application is a part of the driver's body. The determination system according to claim 4 .

Images