Smoking determination device and smoking determination method

Abstract

This smoking determination device comprises: a smoker determination unit (11) that determines whether a subject is a smoker on the basis of smoker determination information; and a smoking determination unit (12, 12a) that, if the smoker determination unit (11) determines that the subject is a smoker, performs smoking determination as to whether the subject is currently smoking on the basis of a captured image in which at least the face of the subject is captured, and that, if the smoker determination unit (11) determines that the subject is a non-smoker, does not perform smoking determination.

Description

Smoking determination device and smoking determination method

[0001] The present disclosure relates to a smoking determination device and a smoking determination method.

[0002] 2. Description of the Related Art Conventionally, there is known a technique for determining whether a person is smoking based on a captured image of the person's face (for example, see Patent Document 1).

[0003] International Publication No. 2013 / 157466

[0004] When determining whether a person is smoking from a captured image, for example, there is a possibility that a behavior similar to a smoking behavior may be mistakenly detected as a smoking behavior, or an object similar to a cigarette may be mistakenly detected as a cigarette. Therefore, conventional technology has had the problem of possibly determining that a non-smoker who is clearly not smoking is smoking.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a smoking determination device that prevents non-smokers from being mistakenly determined to be smoking.

[0006] The smoking determination device of the present disclosure includes a smoker determination unit that determines whether a subject to be determined as smoking is a smoker based on smoker determination information for determining whether the subject is a smoker, and a smoking determination unit that, if the smoker determination unit determines that the subject is a smoker, makes a smoking determination as to whether the subject is smoking based on an image in which at least the face of the subject is captured, and does not make a smoking determination if the smoker determination unit determines that the subject is a non-smoker.

[0007] According to the present disclosure, it is possible to prevent a non-smoker from being mistakenly determined to be a smoker.

[0008] 1 is a diagram showing an example of the configuration of a smoking determination system including a smoking determination device according to embodiment 1. FIG. 1 is a diagram showing an example of the configuration of a smoking determination system in embodiment 1, where the smoker determination information acquisition device is a camera. FIG. 2 is a flowchart explaining the operation of the smoking determination device according to embodiment 1. FIG. 3 is a flowchart explaining the operation of a smoker determination unit in the smoking determination device in embodiment 1, where the smoker determination information is a captured image. FIG. 4 is a diagram showing an example of the configuration of a smoking determination system in embodiment 1, where the smoker determination information acquisition device is a smoker setting device. FIG. 5 is a flowchart explaining the operation of a smoker determination unit in the smoking determination device in embodiment 1, where the smoker determination information is smoker setting information. FIG. 6 is a diagram showing an example of the configuration of a smoking determination system in embodiment 1, where the smoker determination information acquisition device is a personal authentication information acquisition device. FIG. 7 is a flowchart explaining the operation of a smoker determination unit in the smoking determination device in embodiment 1, where the smoker determination information is personal authentication information. FIG. 8 is a diagram showing an example of the configuration of a smoking determination system in embodiment 1, where the smoker determination information acquisition device is a nicotine dependence determination device. 1 is a flowchart illustrating the operation of a smoker determination unit in a smoking determination device when the smoker determination information is nicotine dependence information in embodiment 1. FIG. 2 is a diagram illustrating an example of the configuration of a smoking determination system when the smoker determination information acquisition device is a temperature measurement device in embodiment 1. FIG. 3 is a flowchart illustrating the operation of a smoker determination unit in a smoking determination device when the smoker determination information is heat source information in embodiment 1. FIG. 4 is a diagram illustrating an example of the configuration of a smoking determination system when the smoker determination information acquisition device is a breathing detection device in embodiment 1. FIG. 5 is a flowchart illustrating the operation of a smoker determination unit in a smoking determination device when the smoker determination information is breathing information in embodiment 1. FIG. 6 is a diagram illustrating an example of the configuration of a smoking determination system including a smoking determination device having a function of determining whether a driver is wearing a mask and configured not to perform smoking determination if it is determined that the driver is wearing a mask in embodiment 1.17A and 17B are flowcharts illustrating the operation of a smoking determination device according to Embodiment 1, which has a function of determining whether a driver is wearing a mask and is configured not to perform a smoking determination if it is determined that the driver is wearing a mask.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Embodiment 1. FIG. 1 is a diagram showing an example configuration of a smoking determination system 100 including a smoking determination device 1 according to embodiment 1. The smoking determination device 1 according to embodiment 1 determines whether a person (hereinafter referred to as "subject"), whose smoking status is to be determined, is smoking. In the following embodiment 1, as an example, the smoking determination device 1 is mounted on a vehicle (not shown) and determines whether the driver of the vehicle is smoking. That is, in the following embodiment 1, the subject is the vehicle driver. Note that in embodiment 1, "smoking status" refers to a state in which the person holds a cigarette in their mouth.

[0010] The smoking determination device 1 is connected to a camera 2, a smoker determination information acquisition device 3, and an alarm device 4. The smoking determination device 1, the camera 2, the smoker determination information acquisition device 3, and the alarm device 4 constitute a smoking determination system 100.

[0011] The camera 2 is mounted on the vehicle. For example, the smoking determination device 1 and the camera 2 may constitute a so-called DMS (Driver Monitoring System). The camera 2 is installed, for example, in the center of the vehicle's instrument panel, on the meter panel, on the steering column, on an A-pillar, or near the center console. The camera 2 is installed so as to be able to capture images of at least the faces of the occupants. The camera 2 is a visible light camera or an infrared camera. Note that, in FIG. 1 , only one camera 2 is installed in the vehicle, but this is merely an example. Multiple cameras 2 may be installed in the vehicle. The camera 2 outputs the captured image to the smoking determination device 1.

[0012] The smoker determination information acquisition device 3 is provided in the vehicle or outside the vehicle, and acquires information (hereinafter referred to as "smoker determination information") that the smoking determination device 1 uses to determine whether the driver is a smoker. Details of the smoker determination information acquisition device 3 will be described later with specific examples.

[0013] The alarm device 4 is, for example, an audio output device installed in a vehicle. The alarm device 4 outputs, for example, an alarm sound or a warning voice. Note that the alarm device 4 is not limited to an audio output device, and may be, for example, a display device that displays an alarm message or an alarm icon, or a vibration generating device that generates vibrations. The alarm device 4 is installed in the vehicle or outside the vehicle. For example, the alarm device 4 is installed in the center of the vehicle's instrument panel, meter panel, steering column, A-pillar, or around the center console. The alarm device 4 may be installed in, for example, a navigation device (not shown). Furthermore, the alarm device 4 may be installed in, for example, a PC (personal computer, not shown) installed in a management center (not shown), or in a mobile terminal (not shown) carried by an observer. Furthermore, while FIG. 1 illustrates only one alarm device 4 being connected to the smoking determination device 1, this is merely an example. Multiple alarm devices 4 may be connected to the smoking determination device 1.

[0014] The smoking determination device 1 includes a smoker determination unit 11, a smoking determination unit 12, and an alarm control unit 13. The smoker determination unit 11 acquires smoker determination information from the smoker determination information acquisition device 3 and determines whether the driver is a smoker based on the acquired smoker determination information. If the smoker determination unit 11 determines that the driver is a smoker, the smoking determination unit 12 determines whether the driver is smoking based on the captured image output from the camera 2 (hereinafter referred to as "smoking determination"). If the smoker determination unit 11 determines that the driver is not a smoker, i.e., that the driver is a non-smoker, the smoking determination unit 12 does not perform the smoking determination. If the smoking determination unit 12 determines that the driver is smoking as a result of performing the smoking determination, the alarm control unit 13 outputs information for outputting an alarm (hereinafter referred to as "alarm output control information") to the alarm device 4. The alarm device 4 outputs, for example, an alarm sound or a warning voice, displays an alarm message or a warning icon, or generates vibrations in accordance with the alarm output control information.

[0015] <Smoker Determination Information is Captured Images> The smoker determination information acquisition device 3 may be any device capable of acquiring smoker determination information. For example, the smoker determination information acquisition device 3 is a camera. In this case, the smoker determination information is a captured image captured by the camera. Hereinafter, a detailed configuration example of the smoking determination device 1 according to the first embodiment will be described, assuming that the smoker determination information acquisition device 3 is a camera and the smoker determination information is a captured image. FIG. 2 is a diagram showing a configuration example of the smoking determination system 100 according to the first embodiment, in which the smoker determination information acquisition device 3 is a camera 3a. The camera 3a is installed so as to be able to capture at least the face of the occupant, and captures a captured image in which at least the face of the occupant is captured. The camera 3a is a visible light camera or an infrared camera. Note that, for convenience, FIG. 2 illustrates the camera 2 and the camera 3a separately, but the camera 2 and the camera 3a may be the same camera.

[0016] In this case, in the smoking determination device 1, the smoker determination unit 11 acquires captured images from the camera 3a as smoker determination information. Then, the smoker determination unit 11 determines whether the driver is a smoker based on the captured images acquired from the camera 3a. Specifically, the smoker determination unit 11 acquires a time series of captured images output from the camera 3a during a predetermined period (hereinafter referred to as the "smoking determination period"). For example, the smoker determination unit 11 stores the captured images output from the camera 3a in a buffer of the smoker determination unit 11 or a storage unit (not shown) in time series. The smoker determination unit 11 may acquire captured images for the smoking determination period from the stored time series of captured images. The captured images stored by the smoker determination unit 11 are deleted at a predetermined timing. For example, the smoker determination unit 11 deletes captured images stored prior to the smoking determination period.

[0017] When the smoker determination unit 11 acquires the time-series captured images output from the camera 3a during the smoking determination period, it determines whether the driver's mouth is no longer detected at a predetermined period (hereinafter referred to as the "mouth detection period") based on the acquired time-series captured images. The smoker determination unit 11 may use a known image recognition technique to determine whether the driver's mouth is detectable. If the smoker determination unit 11 determines that the driver's mouth is no longer detected at the mouth detection period, it calculates the time during which the driver's mouth was no longer detected (hereinafter referred to as the "mouth non-detection time"). Then, the smoker determination unit 11 compares the calculated mouth non-detection time with a predetermined threshold (hereinafter referred to as the "mouth non-detection time determination threshold"). If the calculated mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold, the smoker determination unit 11 determines that the driver is a smoker. If the calculated mouth non-detection time is less than the mouth non-detection time determination threshold, the smoker determination unit 11 determines that the driver is a non-smoker.

[0018] To give a specific example, suppose the smoking determination period is 10 hours and the mouth detection cycle is 15 seconds. Furthermore, suppose the calculated mouth non-detection time, i.e., the total time during which a mouth is not detected at 15-second intervals, is 3 hours. Furthermore, suppose the mouth non-detection time determination threshold is 2 hours. In this case, the mouth non-detection time of 3 hours exceeds the mouth non-detection time determination threshold of 2 hours. Therefore, the smoker determination unit 11 determines that the driver is a smoker.

[0019] If a mouth is not detected during a mouth detection cycle and the total time during which the mouth is not detected exceeds the threshold for determining mouth non-detection time, it is presumed that the driver has placed a cigarette in their mouth, i.e., smoked a cigarette. For example, when a mouth is not detected in a captured image, in other words, when the mouth is hidden, the driver's actions may include not only placing a cigarette in their mouth but also eating. However, the action of bringing food to their mouth when eating is not expected to occur as periodically and for as long as the action of placing a cigarette in their mouth to smoke. The smoker determination unit 11 determines whether the driver is placing a cigarette in their mouth by periodically determining whether the mouth is not detected in the captured image for a certain period of time, thereby eliminating the possibility that the driver is placing a cigarette in their mouth. Note that the mouth detection cycle is set to, for example, a period generally expected as the period during which a person places a cigarette in their mouth to smoke one. The threshold for determining mouth non-detection time is set as appropriate by an administrator or the like.

[0020] The smoker determination unit 11 outputs the determination result (hereinafter referred to as the “smoker determination result”) of whether the driver is determined to be a smoker or a non-smoker to the smoking determination unit 12 .

[0021] As described above, when the smoker determination unit 11 determines that the driver is a smoker, the smoking determination unit 12 makes a smoking determination based on the captured image output from the camera 2. When the smoker determination unit 11 determines that the driver is not a smoker, that is, that the driver is a non-smoker, the smoking determination unit 12 does not make the smoking determination. Note that the smoking determination unit 12 can determine whether the smoker determination unit 11 has determined that the driver is a smoker or a non-smoker from the smoker determination result.

[0022] In detail, when the smoker determination unit 11 determines that the driver is a smoker, the smoking determination unit 12 determines whether the driver is smoking based on the captured image and a trained model (hereinafter referred to as the "machine learning model"). In the first embodiment, the machine learning model used by the smoking determination unit 12 to determine whether the driver is smoking is also referred to as the "first machine learning model." The first machine learning model is a trained model that receives a captured image in which at least a person's face is captured as input and outputs information indicating whether the person is smoking. The first machine learning model is generated in advance and stored in a location that can be referenced by the smoking determination unit 12. The smoking determination unit 12 inputs the captured image output from the camera 2 into the first machine learning model and obtains information indicating whether the driver is smoking. When the smoking determination unit 12 obtains information indicating that the driver is smoking, the smoking determination unit 12 determines that the driver is smoking, and when the smoking determination unit 12 obtains information indicating that the driver is not smoking, the smoking determination unit 12 determines that the driver is not smoking.

[0023] If the smoking determination unit 12 determines that the driver is smoking, it outputs information to the alarm control unit 13 that the driver is smoking (hereinafter referred to as ``smoking information'').

[0024] As described above, if the smoking determination unit 12 determines that the driver is smoking as a result of making the smoking determination, in other words, if smoking information is output from the smoking determination unit 12, the alarm control unit 13 outputs alarm output control information to the alarm device 4.

[0025] For example, if the alarm device 4 is an audio output device, the alarm output control information is information for causing the audio output device to output an alarm sound, a warning message warning against smoking, or a warning message urging caution when driving. For example, if the alarm device 4 is a display device, the alarm output control information is information for causing the display device to display a warning message warning against smoking or a warning message urging caution when driving. For example, the alarm output control information may be information for causing the display device to display an icon, such as a cigarette icon. For example, if the alarm device 4 is a vibration generating device, the alarm output control information is information for causing the vibration generating device to generate vibrations warning against smoking.

[0026] The following describes the operation of the smoking determination apparatus 1 according to embodiment 1. Here, the operation of the smoking determination apparatus 1 will be described assuming that the smoker determination information acquisition device 3 is a camera 3a as shown in FIG.

[0027] Fig. 3 is a flowchart for explaining the operation of the smoking determination device 1 according to embodiment 1. The smoking determination device 1 performs the operation shown in the flowchart of Fig. 3 each time an image is output from the camera 2, for example, after the vehicle's power is turned on. More specifically, for example, a control unit (not shown) of the smoking determination device 1 detects that an image has been output from the camera 2 and outputs operation instructions to the smoker determination unit 11, the smoking determination unit 12, and the alarm control unit 13 of the smoking determination device 1. For example, when the vehicle's power is turned off, the smoking determination device 1 ends the operation shown in the flowchart of Fig. 3.

[0028] The smoker determination unit 11 acquires the smoker determination information from the smoker determination information acquisition device 3 and determines whether the driver is a smoker based on the acquired smoker determination information (step ST10). The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0029] The smoking determination unit 12 determines whether the smoker determination unit 11 determined that the driver is a smoker in step ST10 (step ST20). If the smoker determination unit 11 determines that the driver is a smoker in step ST20 (if "YES" in step ST20), the smoking determination unit 12 performs a smoking determination based on the captured image output from the camera 2 (step ST30). If the smoking determination unit 12 determines that the driver is smoking as a result of the smoking determination, it outputs smoking occurrence information to the alarm control unit 13.

[0030] If the smoking determination unit 12 determines in step ST30 that the driver is smoking, in other words, if smoking occurrence information is output from the smoking determination unit 12, the warning control unit 13 outputs warning output control information to the warning device 4 (step ST40). Note that if smoking occurrence information is not output from the smoking determination unit 12, the warning control unit 13 does not output the warning output control information.

[0031] If, in step ST20, the smoker determination unit 11 determines that the driver is a non-smoker (if "NO" in step ST20), the operation of the smoking determination device 1 skips the processing of steps ST30 and ST40 and terminates the processing shown in the flowchart of Figure 3.

[0032] Fig. 4 is a flowchart for explaining an example of the details of the process of step ST10 in Fig. 3. That is, Fig. 4 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 when the smoker determination information is captured images in embodiment 1. The smoker determination unit 11 acquires captured images from the camera 3a as the smoker determination information. More specifically, the smoker determination unit 11 acquires the captured images in time series output from the camera 3a during the smoking determination period (step ST101a).

[0033] When the smoker determination unit 11 acquires the time-series captured images output from the camera 3a during the smoking determination period, it determines whether the driver's mouth is no longer detected in the mouth detection period based on the acquired time-series captured images, and if it determines that the driver's mouth is no longer detected in the mouth detection period, it calculates the mouth non-detection time (step ST102a).

[0034] The smoker determination unit 11 compares the calculated mouth non-detection time with a mouth non-detection time determination threshold and determines whether the mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold (step ST103a). If the mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold ("YES" in step ST103a), the smoker determination unit 11 determines that the driver is a smoker (step ST104a). The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12. On the other hand, if the mouth non-detection time is less than the mouth non-detection time determination threshold ("NO" in step ST103a), the smoker determination unit 11 determines that the driver is a non-smoker (step ST105a). The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

[0035] In this way, the smoking determination device 1 determines whether the driver is a smoker or not based on the smoker determination information, and if it determines that the driver is a smoker, it makes a smoking determination based on the captured image to determine whether the driver is smoking or not, and if it determines that the driver is a non-smoker, it does not make a smoking determination.

[0036] In more detail, for example, if the smoker determination information acquisition device 3 is a camera 3a and the smoker determination information is a captured image, the smoking determination device 1 calculates, based on the captured image, if the driver's mouth is not detected at the mouth detection cycle during the smoking determination period, the mouth non-detection time during which the driver's mouth was not detected during the smoking determination period, and if the mouth non-detection time is equal to or greater than the mouth non-detection time determination threshold, determines that the driver is a smoker. If the mouth non-detection time is not equal to or greater than the mouth non-detection time determination threshold, the smoking determination device 1 determines that the driver is a non-smoker. Then, if the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. Note that if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

[0037] This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.

[0038] If the smoking determination device 1 did not determine whether a driver is a smoker or a non-smoker, but instead performed a smoking determination on non-smokers, there is a possibility that it would erroneously determine that a non-smoker who clearly does not smoke is a smoker. This is because, when determining whether a person is smoking from a captured image, for example, a motion similar to a smoking motion may be erroneously detected as a smoking motion, or an object similar to a cigarette may be erroneously detected as a cigarette. For example, cigarettes and stick-shaped snacks have similar shapes. Therefore, the motion of holding a stick-shaped snack in one's mouth may be erroneously detected as holding a cigarette in one's mouth. As a result, a false alarm warning of smoking may be output based on the erroneous determination that the person is smoking. Such a false alarm is particularly annoying for non-smokers who clearly do not smoke. In contrast, the smoking determination device 1, as described above, determines whether a driver is a smoker or a non-smoker, and performs a smoking determination only if it determines that the driver is a smoker. This prevents the smoking determination device 1 from erroneously determining that a non-smoker is a smoker. For non-smokers who never smoke, the function of determining whether or not a person is smoking is itself an unnecessary function. The smoking determination device 1 disables the function of determining whether or not a person is smoking, which is an unnecessary function for non-smokers, in other words, does not determine whether or not a person is smoking, thereby reducing the inconvenience that may be caused to non-smokers as described above.

[0039] In the above description, the smoker determination information acquisition device 3 is, for example, a camera 3a. However, the smoker determination information acquisition device 3 may be any device capable of acquiring smoker determination information, and the smoking determination device 1 can determine whether the driver is a smoker based on the smoker determination information obtained from various smoker determination information acquisition devices 3.

[0040] <Smoker Determination Information is Smoker Setting Information> For example, the smoker determination information acquisition device 3 may be a smoker setting device that allows the driver to set that he or she is a smoker. In this case, the smoker determination information is information set in the smoker setting device indicating whether the vehicle driver is a smoker (hereinafter referred to as "smoker setting information"). Details of the smoking determination device 1 when the smoker determination information acquisition device 3 is a smoker setting device and the smoker determination information is smoker setting information are described below. Figure 5 is a diagram showing an example of the configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is a smoker setting device 3b. Note that the example configuration of the smoking determination device 1 is the same as the example configuration described in Figures 1 and 2, and therefore the same reference numerals are used.

[0041] The smoker setting device 3b is mounted on, for example, a vehicle and has a setting button. The smoker setting device 3b may be provided in a navigation device located in the center of an instrument panel or on a meter panel, etc. For example, when the driver gets into the vehicle, he or she operates the setting button to input setting information for setting whether the driver is a smoker or a non-smoker. The smoker setting device 3b accepts the setting information input by the driver and generates smoker setting information. After generating the smoker setting information, the smoker setting device 3b outputs the generated smoker setting information to the smoking determination device 1.

[0042] In the smoking determination device 1, the smoker determination unit 11 acquires smoker setting information from the smoker setting device 3b as information for smoker determination. Then, the smoker determination unit 11 determines whether the driver is a smoker based on the smoker setting information acquired from the smoker setting device 3b. In detail, if the driver is set as a smoker in the smoker setting information, the smoker determination unit 11 determines that the driver is a smoker. If the driver is set as a non-smoker in the smoker setting information, the smoker determination unit 11 determines that the driver is a non-smoker. The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0043] The details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, and therefore will not be explained again.

[0044] Even when the smoker determination information is smoker setting information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is a captured image. When the smoker determination information is smoker setting information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG. 3 . However, the details of the processing of step ST10 in FIG. 3 by the smoker determination unit 11 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is smoker setting information, as described above. FIG. 6 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 when the smoker determination information is smoker setting information in embodiment 1. When the smoker determination information is smoker setting information, the smoking determination device 1 performs the processing in step ST10 in FIG. 3 , specifically as described in the flowchart of FIG. 6 .

[0045] The smoker determining unit 11 acquires smoker setting information as smoker determination information from the smoker setting device 3b (step ST101b).

[0046] The smoker determination unit 11 determines whether or not the driver is set as a smoker in the smoker setting information (step ST102b).

[0047] If it is determined that the driver is set as a smoker ("YES" in step ST102b), the smoker determination unit 11 determines that the driver is a smoker (step ST103b). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12. On the other hand, if it is determined that the driver is set as a non-smoker ("NO" in step ST102b), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104b). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0048] Thus, for example, if the smoker determination information acquisition device 3 is a smoker setting device 3b and the smoker determination information is smoker setting information, the smoking determination device 1 determines that the driver is a smoker if the driver is set as a smoker in the smoker setting information. If the smoking determination device 1 determines that the driver is a smoker, it performs a smoking determination based on the captured image to determine whether the driver is smoking. Note that if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

[0049] This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.

[0050] <Smoker Determination Information is Personal Authentication Information> Furthermore, for example, the smoker determination information acquisition device 3 may be a personal authentication information acquisition device that acquires personal authentication information for personal authentication of a driver. In this case, the smoker determination information is personal authentication information acquired by the personal authentication information acquisition device. Below, details of the smoking determination device 1 when the smoker determination information acquisition device 3 is a personal authentication information acquisition device and the smoker determination information is personal authentication information will be described. FIG. 7 is a diagram showing an example configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is a personal authentication information acquisition device 3c. Note that the example configuration of the smoking determination device 1 is similar to the example configurations described in FIGS. 1, 2, and 5, and therefore the same reference numerals are used.

[0051] The personal authentication information acquisition device 3c may be, for example, a camera 2, a reading device that reads a card key such as a contactless IC card, or a biometric information acquisition device that reads biometric information. The personal authentication information acquisition device 3c may be any device that can acquire personal authentication information for performing any known personal authentication. For example, when the driver gets into the vehicle, the personal authentication information acquisition device 3c reads the personal authentication information. After acquiring the personal authentication information, the personal authentication information acquisition device 3c outputs the acquired personal authentication information to the smoking determination device 1.

[0052] In the smoking determination device 1, the smoker determination unit 11 acquires personal authentication information as smoker determination information from the personal authentication information acquisition device 3 c. The smoker determination unit 11 then performs personal authentication of the driver using a known personal authentication method based on the personal authentication information acquired from the personal authentication information acquisition device 3 c. After performing personal authentication, the smoker determination unit 11 compares information that can identify the authenticated driver (e.g., an ID or a facial image) with information that can identify people registered in a smoker registration list (e.g., an ID or a facial image), and determines whether the driver is a smoker or a non-smoker.

[0053] The smoker registration list is a list of registered smokers, and lists information that can identify smokers in the smoker registration list. The smoker registration list is generated in advance by an administrator or the like, and is held by the smoker determination unit 11 or stored in a location that the smoker determination unit 11 can refer to.

[0054] The smoker determination unit 11 determines that the driver is a smoker if the personally authenticated driver is registered in the smoker registration list, more specifically, if information that can identify the personally authenticated driver is registered in the smoker registration list. The smoker determination unit 11 determines that the driver is a non-smoker if the personally authenticated driver is not registered in the smoker registration list, more specifically, if information that can identify the personally authenticated driver is not registered in the smoker registration list. The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0055] The details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, and therefore will not be explained again.

[0056] Even when the smoker determination information is personal authentication information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is a captured image or smoker setting information. When the smoker determination information is personal authentication information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG. 3 . However, the details of the processing by the smoker determination unit 11 in step ST10 of FIG. 3 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is a captured image or smoker setting information, as described above. FIG. 8 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 when the smoker determination information is personal authentication information in embodiment 1. When the smoker determination information is personal authentication information, the smoking determination device 1 performs the processing in step ST10 of FIG. 3 , specifically as described in the flowchart of FIG. 8 .

[0057] The smoker determination unit 11 acquires personal authentication information as smoker determination information from the personal authentication information acquisition device 3c (step ST101c).The smoker determination unit 11 performs personal authentication of the driver based on the personal authentication information acquired from the personal authentication information acquisition device 3c using a known personal authentication method (step ST102c).

[0058] After performing personal authentication, the smoker determination unit 11 compares information that can identify the authenticated driver with information that can identify people registered in the smoker registration list. Specifically, the smoker determination unit 11 determines whether the authenticated driver is registered in the smoker registration list in step ST102c (step ST103c). If the smoker determination unit 11 determines in step ST103c that the authenticated driver is registered in the smoker registration list (YES in step ST103c), the smoker determination unit 11 determines that the driver is a smoker (step ST104c). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12. If the smoker determination unit 11 determines in step ST103c that the authenticated driver is not registered in the smoker registration list (NO in step ST103c), the smoker determination unit 11 determines that the driver is a non-smoker (step ST105c). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12 .

[0059] Thus, for example, if the smoker determination information acquisition device 3 is a personal authentication information acquisition device 3c and the smoker determination information is personal authentication information, the smoking determination device 1 determines that the driver is a smoker if the driver is registered in the smoker registration list based on the personal authentication information. If the smoking determination device 1 determines that the driver is a smoker, it then performs a smoking determination based on the captured image to determine whether the driver is smoking. Note that if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

[0060] This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.

[0061] <Information for smoker determination is nicotine dependence information> Furthermore, for example, the information acquisition device 3 for smoker determination may be a nicotine dependence determination device that determines the driver's nicotine dependence. In this case, the information for smoker determination is nicotine dependence information indicating the driver's nicotine dependence. Below, details of the smoking determination device 1 when the information acquisition device 3 for smoker determination is a nicotine dependence determination device and the information for smoker determination is nicotine dependence information will be described. Figure 9 is a diagram showing an example configuration of the smoking determination system 100 in the first embodiment when the information acquisition device 3 for smoker determination is a nicotine dependence determination device 3d. Note that the example configuration of the smoking determination device 1 is the same as the example configurations described in Figures 1, 2, 5, and 7, and therefore the same reference numerals are used.

[0062] The nicotine dependence assessment device 3d acquires the results of a nicotine dependence test, such as the Fagerstrom Tobacco Dependence Test, administered to subjects including a driver, and assesses the subject's nicotine dependence. The nicotine dependence test is, for example, a test in which subjects answer questions and assess the subject's nicotine dependence based on the total score. The nicotine dependence assessment device 3d assesses the nicotine dependence such that the higher the score on the nicotine dependence test, the higher the nicotine dependence. Here, the nicotine dependence assessment device 3d is assumed to be mounted on a vehicle. For example, when the driver gets into the vehicle and turns on the vehicle, the nicotine dependence assessment device 3d displays the nicotine dependence test on a display device (not shown). The display device is, for example, a touch panel display provided in a navigation device. The driver inputs answers to the questions in the nicotine dependence test by touching the display device. The nicotine dependence assessment device 3d acquires the results of the nicotine dependence test administered by the driver and assesses the driver's nicotine dependence. The nicotine dependence determination device 3 d then outputs nicotine dependence information indicating the determined nicotine dependence to the smoking determination device 1 .

[0063] For example, the nicotine dependence determination device 3d may have the driver take a nicotine dependence test only if the driver has never taken a nicotine dependence test in the past. In this case, for example, the nicotine dependence determination device 3d has a personal authentication function, and stores information that can identify the driver who took the nicotine dependence test (the driver's personal authentication result) and associates the determined nicotine dependence of the driver (hereinafter referred to as "test result information") in a memory unit (not shown). The memory unit may be included in the nicotine dependence determination device 3d, or may be provided in a location external to the nicotine dependence determination device 3d that can be referenced by the nicotine dependence determination device 3d. When the nicotine dependence determination device 3d performs personal authentication of the driver when the driver gets into the vehicle, it refers to the memory unit and determines whether test result information including the personal authentication result is stored. If the test result information is stored, it is determined that the driver has taken a nicotine dependence test in the past, and the nicotine dependence set in the test result information is set as the driver's nicotine dependence. In this case, the nicotine dependence determination device 3d does not cause the display device to display the nicotine dependence test. If the test result information is not stored, the nicotine dependence determination device 3d determines that the driver has never taken a nicotine dependence test, and causes the display device to display the nicotine dependence test and have the driver take the nicotine dependence test. In this way, if the driver has taken a nicotine dependence test in the past, the nicotine dependence determination device 3d uses the test result to determine the nicotine dependence and does not cause the driver to take the nicotine dependence test, so that the nicotine dependence determination device 3d can save the driver the hassle of taking the nicotine dependence test multiple times.

[0064] In the smoking determination device 1, the smoker determination unit 11 acquires nicotine dependence information from the nicotine dependence determination device 3d as information for smoker determination. The smoker determination unit 11 then determines whether the driver is a smoker or a non-smoker based on the nicotine dependence information acquired from the nicotine dependence determination device 3d. The smoker determination unit 11 determines that the driver is a smoker if the driver's nicotine dependence is estimated to be high based on the nicotine dependence information. On the other hand, the smoker determination unit 11 determines that the driver is a smoker if the driver's nicotine dependence is estimated to be low based on the nicotine dependence information. For example, the smoker determination unit 11 estimates that the driver's nicotine dependence is high if the driver's nicotine dependence indicated by the nicotine dependence information is equal to or higher than a predetermined threshold (hereinafter referred to as the "dependence determination threshold"), and estimates that the driver's nicotine dependence is low if the driver's nicotine dependence is below the dependence determination threshold. The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0065] The details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, and therefore will not be explained again.

[0066] Even when the smoker determination information is nicotine dependence information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is a captured image, smoker setting information, or personal authentication information. When the smoker determination information is nicotine dependence information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG. 3 . However, the details of the processing by the smoker determination unit 11 in step ST10 of FIG. 3 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is a captured image, smoker setting information, or personal authentication information, as described above. FIG. 10 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 when the smoker determination information is nicotine dependence information in Embodiment 1. When the smoker determination information is nicotine dependence information, the smoking determination device 1 performs the processing in step ST10 of FIG. 3 , specifically, the processing described in the flowchart of FIG. 10 .

[0067] The smoker determination unit 11 acquires nicotine dependence information from the nicotine dependence determination device 3d as information for smoker determination (step ST101d). Based on the nicotine dependence information, the smoker determination unit 11 estimates whether the driver's nicotine dependence is high or low (step ST102d). If the smoker determination unit 11 estimates that the driver's nicotine dependence is high in step ST102d (YES in step ST102d), the smoker determination unit 11 determines that the driver is a smoker (step ST103d). The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12. If the smoker determination unit 11 estimates that the driver's nicotine dependence is low in step ST102d (NO in step ST102d), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104d). The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

[0068] Thus, for example, if the information acquisition device 3 for smoker determination is a nicotine dependence determination device 3d and the information for smoker determination is nicotine dependence information, the smoking determination device 1 determines that the driver is a smoker if it estimates that the driver has a high level of nicotine dependence based on the nicotine dependence information. If the smoking determination device 1 determines that the driver is a smoker, it then performs a smoking determination based on the captured image to determine whether the driver is smoking. Note that if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

[0069] This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.

[0070] <Smoker Determination Information is Heat Source Information> Furthermore, for example, the smoker determination information acquisition device 3 may be a temperature measurement device that measures the temperature around the driver's face. In this case, the smoker determination information is information related to the temperature around the driver's face measured by the temperature measurement device (hereinafter referred to as "heat source information"). Details of the smoking determination device 1 when the smoker determination information acquisition device 3 is a temperature measurement device and the smoker determination information is heat source information are described below. FIG. 11 is a diagram showing an example configuration of the smoking determination system 100 in the first embodiment when the smoker determination information acquisition device 3 is a temperature measurement device 3e. The example configuration of the smoking determination device 1 is similar to the example configurations described in FIGS. 1, 2, 5, 7, and 9, and therefore the same reference numerals are used.

[0071] The temperature measuring device 3e may be, for example, a heat source camera or a temperature sensor. The temperature measuring device 3e may be any device capable of acquiring heat source information. Here, as an example, the temperature measuring device 3e is assumed to be a heat source camera. A heat source camera is a so-called thermograph that generates a heat source image according to the amount of infrared rays emitted from an object. The temperature measuring device 3e outputs the heat source information, more specifically, the heat source image, to the smoking determination device 1.

[0072] In the smoking determination device 1, the smoker determination unit 11 acquires heat source information from the temperature measuring device 3e as information for smoker determination. The smoker determination unit 11 then determines whether the driver is a smoker based on the heat source information acquired from the temperature measuring device 3e. Specifically, the smoker determination unit 11 acquires time-series heat source information output from the temperature measuring device 3e during the smoking determination period. For example, the smoker determination unit 11 chronologically stores the heat source information output from the temperature measuring device 3e in a buffer of the smoker determination unit 11 or in a storage unit (not shown). The smoker determination unit 11 may acquire heat source information for the smoking determination period from the stored time-series heat source information. The heat source information stored by the smoker determination unit 11 is deleted at a preset timing. For example, the smoker determination unit 11 deletes heat source information stored prior to the smoking determination period, going back from the present.

[0073] When the smoker determination unit 11 acquires the time-series heat source information output from the temperature measurement device 3e during the smoking determination period, it determines, based on the acquired time-series heat source information, whether a high-temperature area above a preset threshold (hereinafter referred to as the "temperature determination threshold") has been detected around the driver's face for a preset time (hereinafter referred to as the "high-temperature determination time"). The smoker determination unit 11, for example, acquires captured images from the camera 2 and performs known image recognition processing on the acquired captured images and heat source images to identify which area on the heat source image corresponds to the area around the driver's mouth. Because the installation position and field of view of the camera 2 and the installation position and field of view of the heat source camera are known in advance, the smoker determination unit 11 can associate areas on the captured images with areas on the heat source images. Note that the arrow from the camera 2 to the smoker determination unit 11 is omitted in FIG. 11 .

[0074] The smoker determination unit 11 determines that the driver is a smoker when it determines that a high-temperature area equal to or greater than the temperature determination threshold has been detected around the driver's face for a period equal to or greater than the high-temperature determination time. On the other hand, the smoker determination unit 11 determines that the driver is a non-smoker when there is no high-temperature area equal to or greater than the temperature determination threshold around the driver's face, or when there is a high-temperature area equal to or greater than the temperature determination threshold around the driver's face but the high-temperature area has not been detected for a period equal to or greater than the high-temperature determination time.

[0075] Note that the method by which the smoker determination unit 11 determines whether a driver is a smoker or a non-smoker based on the time-series heat source information is not limited to the above-described method. The smoker determination unit 11 may determine whether a driver is a smoker or a non-smoker based on the time-series heat source information and in accordance with other conditions. For example, the smoker determination unit 11 may determine whether the appearance frequency of high-temperature areas around the driver's face is equal to or greater than a predetermined threshold (hereinafter referred to as the "high-temperature appearance frequency determination threshold") based on the acquired time-series heat source information. In this case, the smoker determination unit 11 determines that the driver is a smoker if the appearance frequency of high-temperature areas around the driver's face is equal to or greater than the high-temperature appearance frequency determination threshold. The smoker determination unit 11 determines that the driver is a non-smoker if the appearance frequency of high-temperature areas around the driver's face is less than the high-temperature appearance frequency determination threshold. The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0076] The details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, and therefore will not be explained again.

[0077] Even when the smoker determination information is heat source information, the basic operation of the smoking determination device 1 is the same as when the smoker determination information is captured images, smoker setting information, personal authentication information, or nicotine dependence information. When the smoker determination information is heat source information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG. 3 . However, the details of the processing by the smoker determination unit 11 in step ST10 of FIG. 3 differ from the details of the processing by the smoker determination unit 11 when the smoker determination information is captured images, smoker setting information, personal authentication information, or nicotine dependence information, as described above. FIG. 12 is a flowchart for explaining the operation of the smoker determination unit 11 in the smoking determination device 1 when the smoker determination information is heat source information in the first embodiment. When the smoker determination information acquisition device 3 is a temperature measurement device 3e, the smoking determination device 1 performs the processing in step ST10 of FIG. 3 , specifically, the processing described in the flowchart of FIG. 12 .

[0078] The smoker determination unit 11 acquires heat source information from the temperature measuring device 3e as information for smoker determination. More specifically, the smoker determination unit 11 acquires time-series heat source information output from the temperature measuring device 3e during the smoking determination period (step ST101e). Based on the time-series heat source information, the smoker determination unit 11 determines whether a high-temperature area equal to or greater than the temperature determination threshold has been detected around the driver's face for a period equal to or greater than the high-temperature determination time, or whether the frequency of occurrence of high-temperature areas around the driver's face is equal to or greater than the high-temperature occurrence frequency determination threshold (step ST102e).

[0079] In step ST102e, if it is determined that a high-temperature area equal to or higher than the temperature determination threshold has been detected around the driver's face for the high-temperature determination time or longer, or if it is determined that the appearance frequency of high-temperature areas around the driver's face is equal to or higher than the high-temperature appearance frequency determination threshold ("YES" in step ST102e), the smoker determination unit 11 determines that the driver is a smoker (step ST103e). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0080] In step ST102e, if it is determined that there is no high temperature area around the driver's face that is equal to or higher than the temperature determination threshold, if it is determined that even if there is a high temperature area around the driver's face that is equal to or higher than the temperature determination threshold, the high temperature area has not been detected for at least the high temperature determination time, or if it is determined that the frequency of appearance of high temperature areas around the driver's face is less than the high temperature appearance frequency determination threshold ("NO" in step ST102e), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104e). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0081] Thus, for example, if the smoker determination information acquisition device 3 is a temperature measurement device 3e and the smoker determination information is heat source information, the smoking determination device 1 determines that the driver is a smoker if, based on the time-series heat source information, it detects a high-temperature area around the driver's face that is equal to or greater than the temperature determination threshold for a high-temperature determination time or longer, or if the frequency of occurrence of high-temperature areas around the driver's face is equal to or greater than the high-temperature occurrence frequency determination threshold. If the smoking determination device 1 determines that the driver is a smoker, it then performs a smoking determination based on the captured image to determine whether the driver is smoking. Note that if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

[0082] This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.

[0083] <Information for Smoker Determination is Respiratory Information> Furthermore, for example, the information acquisition device 3 for smoker determination may be a breathing detection device that detects the breathing of the driver. In this case, the information for smoker determination is information related to the breathing of the vehicle driver (hereinafter referred to as "respiratory information") set by the breathing detection device. Below, details of the smoking determination device 1 when the information acquisition device 3 for smoker determination is a breathing detection device and the information for smoker determination is respiratory information will be described. FIG. 13 is a diagram showing an example configuration of the smoking determination system 100 in the first embodiment when the information acquisition device 3 for smoker determination is a breathing detection device 3f. Note that the example configuration of the smoking determination device 1 is similar to the example configurations described in FIGS. 1, 2, 5, 7, 9, and 11, and therefore the same reference numerals are used.

[0084] The breathing detection device 3f is, for example, a millimeter wave sensor. The millimeter wave sensor transmits radio waves, such as millimeter waves, into the vehicle interior and receives the reflected waves from an object. The millimeter wave sensor can detect the presence, position, speed, etc. of an object associated with the reflected waves based on the time from when the radio waves are transmitted to when the reflected waves are received, thereby detecting movement that can be estimated as a person's breathing. Note that this is merely an example, and the breathing detection device 3f may be, for example, a camera 2 or a breathing sensor attached to clothing. The breathing detection device 3f may be any device that can detect the driver's breathing. The breathing detection device 3f outputs detected breathing information of the driver to the smoking determination device 1. The breathing information includes, for example, the waveform of the driver's breathing.

[0085] In the smoking determination device 1, the smoker determination unit 11 acquires breathing information from the breathing detection device 3f as information for smoker determination. The smoker determination unit 11 then determines whether the driver is a smoker based on the breathing information acquired from the breathing detection device 3f. Specifically, the smoker determination unit 11 acquires time-series breathing information output from the breathing detection device 3f during a smoking determination period. For example, the smoker determination unit 11 chronologically stores the breathing information output from the breathing detection device 3f in a buffer of the smoker determination unit 11 or in a storage unit (not shown). The smoker determination unit 11 may acquire breathing information for the smoking determination period from the stored time-series breathing information. The breathing information stored by the smoker determination unit 11 is deleted at a preset timing. For example, the smoker determination unit 11 deletes breathing information stored prior to the smoking determination period, going back from the present.

[0086] When the smoker determination unit 11 acquires time-series breathing information output from the breathing detection device 3f during the smoking determination period, it detects breathing that is estimated to be breathing associated with smoking behavior (hereinafter referred to as "smoking breathing") based on the acquired time-series breathing information. Generally, a person's breathing during smoking behavior, in other words, the behavior of smoking a cigarette, consists of a single inhalation phase and a single exhalation phase. Breathing during smoking behavior is estimated to have a duration and amplitude that is a predetermined times longer or larger than normal breathing. The smoker determination unit 11 can detect the driver's smoking breath by determining the timing of the driver's inhalation and exhalation from the breathing information.

[0087] The smoker determination unit 11 determines that the driver is a smoker if the number of smoking breaths during the smoking determination period is equal to or greater than a preset number (hereinafter referred to as the "smoking breath determination threshold"). On the other hand, the smoker determination unit 11 determines that the driver is a non-smoker if the number of smoking breaths during the smoking determination period is less than the smoking breath determination threshold. The smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0088] The details of the smoking determination unit 12 and the alarm control unit 13 are the same as those already explained, and therefore will not be explained again.

[0089] Even when the information for smoker determination is respiratory information, the basic operation of the smoking determination device 1 is the same as when the information for smoker determination is captured images, smoker setting information, personal authentication information, nicotine dependence information, or heat source information. When the information for smoker determination is respiratory information, the basic operation of the smoking determination device 1 is the same as the operation of the smoking determination device 1 described using the flowchart of FIG. 3 . However, the details of the processing by the smoker determination unit 11 in step ST10 of FIG. 3 differ from the details of the processing by the smoker determination unit 11 when the information for smoker determination is captured images, smoker setting information, personal authentication information, nicotine dependence information, or heat source information, as described above. FIG. 14 is a flowchart illustrating the operation of the smoker determination unit 11 in the smoking determination device 1 when the information for smoker determination is respiratory information in embodiment 1. When the information for smoker determination is respiratory information, the smoking determination device 1 performs the processing in step ST10 of FIG. 3 , specifically, the processing described in the flowchart of FIG. 14 .

[0090] The smoker determination unit 11 acquires breathing information from the breathing detection device 3f as information for determining whether the smoker is present. More specifically, the smoker determination unit 11 acquires time-series breathing information output from the breathing detection device 3f during the smoking determination period (step ST101f). Based on the time-series breathing information, the smoker determination unit 11 determines whether the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold (step ST102f).

[0091] If it is determined in step ST102f that the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold (YES in step ST102f), the smoker determination unit 11 determines that the driver is a smoker (step ST103f).The smoker determination unit 11 then outputs the smoker determination result to the smoking determination unit 12.

[0092] If it is determined in step ST102f that the number of smoking breaths during the smoking determination period is less than the smoking breath determination threshold ("NO" in step ST102f), the smoker determination unit 11 determines that the driver is a non-smoker (step ST104e). Then, the smoker determination unit 11 outputs the smoker determination result to the smoking determination unit 12.

[0093] Thus, for example, if the information acquisition device 3 for smoker determination is a breathing detection device 3f and the information for smoker determination is breathing information, the smoking determination device 1 determines that the driver is a smoker based on the time-series breathing information if the number of smoking breaths during the smoking determination period is equal to or greater than the smoking breath determination threshold. If the smoking determination device 1 determines that the driver is a smoker, it then performs a smoking determination based on the captured image to determine whether the driver is smoking. Note that if the smoking determination device 1 determines that the driver is a non-smoker, it does not perform a smoking determination.

[0094] This allows the smoking determination device 1 to prevent erroneous determination that a non-smoker is smoking.

[0095] Furthermore, in the above-described first embodiment, the possibility that the driver is wearing a mask was not taken into consideration. However, it cannot be said with certainty that the driver does not wear a mask. In the first embodiment, "mask" refers to a nonwoven fabric, gauze, or other protective device that covers the nose or mouth. If the driver is wearing a mask, it is not assumed that the driver is smoking. In other words, if the driver is wearing a mask, the function of determining whether or not the driver is smoking is itself an unnecessary function, just as in the case where the driver is a non-smoker. Therefore, in the above-described first embodiment, the smoking determination device 1 may be configured to have a function of determining whether or not the driver is wearing a mask, and not to perform a smoking determination if it is determined that the driver is wearing a mask.

[0096] FIG. 15 is a diagram showing an example of the configuration of a smoking determination system 100a according to the first embodiment, including a smoking determination device 1a having a function for determining whether a driver is wearing a mask and configured to not perform a smoking determination if it is determined that the driver is wearing a mask. In FIG. 15, components similar to those in the smoking determination system 100 described with reference to FIG. 1 are denoted by the same reference numerals, and redundant description will be omitted. The smoking determination system 100a shown in FIG. 15 differs from the smoking determination system 100 described with reference to FIG. 1 in that the smoking determination device 1a includes a mask wearing determination unit 14. Furthermore, the specific operation of the smoking determination unit 12a in the smoking determination device 1a differs from the specific operation of the smoking determination unit 12 in the smoking determination device 1 shown in FIG. 1. In the smoking determination system 100a shown in FIG. 15, the smoker determination information acquisition device 3 may be a camera 3a, a smoker setting device 3b, a personal authentication information acquisition device 3c, a nicotine dependence determination device 3d, a temperature measurement device 3e, or a respiration detection device 3f. In other words, for the smoking determination device 1a shown in Figure 15, the information acquired for determining a smoker may be a captured image, smoker setting information, personal authentication information, nicotine dependence information, heat source information, or breathing information.

[0097] The mask wearing determiner 14 determines whether the driver is wearing a mask based on the captured image output from the camera 2. The mask wearing determiner 14 determines whether the driver is wearing a mask based on, for example, the captured image acquired from the camera 2 and a machine learning model. In the following description, the machine learning model used by the mask wearing determiner 14 to determine whether the driver is wearing a mask is also referred to as a "second machine learning model." The second machine learning model has been trained to input a captured image of a person's face and output information indicating whether the person is wearing a mask (hereinafter referred to as "mask wearing information"). The mask wearing information includes, for example, information indicating whether the person is wearing a mask. Note that this is merely an example, and the mask wearing information may include, for example, information indicating the color, pattern, or type of mask (nonwoven fabric, gauze, etc.) in addition to the information indicating whether the person is wearing a mask. The mask wearing information only needs to include information indicating whether the person is wearing a mask. In other words, the second machine learning model is required to be a model that has been trained to take as input at least an image of a person's face and output mask wearing information including information indicating whether the person is wearing a mask.

[0098] When the captured image is input to the second machine learning model and mask wearing information indicating that the driver is wearing a mask is obtained, the mask wearing determination unit 14 determines that the driver is wearing a mask. On the other hand, when the captured image is input to the second machine learning model and mask wearing information indicating that the driver is not wearing a mask is obtained, the mask wearing determination unit 14 determines that the driver is not wearing a mask.

[0099] The above-described method is merely an example, and the mask wearing determiner 14 may determine whether the driver is wearing a mask using other methods based on the captured image. For example, the mask wearing determiner 14 may determine whether the driver is wearing a mask based on whether or not a feature point of the nose can be extracted based on the captured image. If the feature point of the nose cannot be extracted, the mask wearing determiner 14 determines that the driver is wearing a mask. The mask wearing determiner 14 may extract feature points that represent parts of the driver's face (specifically, the nose) in the captured image using a known technique such as a method using Haar-Like features or pattern matching.

[0100] When the mask wearing determination unit 14 determines whether the driver is wearing a mask or not, it outputs the result (hereinafter referred to as the "mask determination result") to the smoking determination unit 12a. The mask determination result includes information indicating whether the driver is wearing a mask or not.

[0101] If the mask wearing determination unit 14 determines that the driver is wearing a mask, the smoking determination unit 12a does not make a smoking determination regardless of whether the smoker determination unit 11 has determined that the driver is a smoker. The smoking determination unit 12a makes a smoking determination only if the mask wearing determination unit 14 determines that the driver is not wearing a mask. Note that the smoking determination unit 12a can determine, from the mask determination result output from the mask wearing determination unit 14, whether the mask wearing determination unit 14 has determined that the driver is wearing a mask or not wearing a mask.

[0102] Fig. 16 is a flowchart illustrating the operation of the smoking determination device 1a in the first embodiment, which has a function of determining whether the driver is wearing a mask and is configured not to perform a smoking determination if it is determined that the driver is wearing a mask. The specific details of the processes of steps ST10, ST20, ST30, and ST40 in Fig. 16 are the same as the specific details of the processes of steps ST10, ST20, ST30, and ST40 in Fig. 1, respectively, and therefore will not be described again. The details of the process of step ST10 in Fig. 16 are the same as those shown and described in the flowcharts of Figs. 4, 6, 8, 10, 12, or 14.

[0103] The mask wearing determiner 14 determines whether the driver is wearing a mask based on the captured image output from the camera 2 (step ST25). The mask wearing determiner 14 outputs the mask determination result to the smoking determiner 12a.

[0104] The smoking determination unit 12a determines whether the mask wearing determination unit 14 has determined in step ST25 that the driver is wearing a mask (step ST27). If the smoking determination unit 12a determines that the mask wearing determination unit 14 has determined that the driver is wearing a mask ("YES" in step ST27), the smoking determination unit 12a proceeds to the processing of step ST30 and performs smoking determination. If the smoking determination unit 12a determines that the mask wearing determination unit 14 has determined that the driver is not wearing a mask ("NO" in step ST27), the operation of the smoking determination device 1a skips the processing of steps ST30 and ST40 and ends the processing shown in the flowchart of FIG. 16.

[0105] 16, the smoking determination device 1a performs the process of step ST25 after the process of step ST10, but this is merely an example. For example, the smoking determination device 1a may perform the process of step ST10 and the process of step ST25 in parallel. Alternatively, the smoking determination device 1a may perform the processes in the order of step ST25, step ST27, step ST10, step ST20, step ST30, and step ST40. That is, if the smoking determination device 1a determines, as a result of the process of step ST25, that the driver is not wearing a mask ("NO" in step ST27), it may perform the process of step ST10, and if it determines that the driver is a smoker ("YES" in step ST20), it may perform the process of step ST30.

[0106] In this way, for example, the smoking determination device 1a may be configured to determine whether or not the driver is wearing a mask based on the captured image, and if it determines that the driver is wearing a mask, not to make a smoking determination regardless of whether or not it determines that the driver is a smoker.

[0107] This allows the smoking determination device 1 to prevent unnecessary smoking determinations when the driver is wearing a mask. As a result, the smoking determination device 1 can prevent erroneous warnings about smoking from being issued to a driver who is wearing a mask and is not expected to be smoking, thereby reducing the annoyance of false warnings for the driver.

[0108] 17A and 17B are diagrams showing an example of the hardware configuration of the smoking determination device 1 according to embodiment 1. In embodiment 1, the functions of the smoker determination unit 11, the smoking determination units 12 and 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) are realized by a processing circuit 1001. That is, the smoking determination device 1 includes the processing circuit 1001 for determining whether a subject is a smoker or a non-smoker based on the smoker determination information, and for controlling the device to perform a smoking determination based on the captured image acquired from the camera 2 only if the subject is a smoker or only if the subject is not wearing a mask and is a smoker. The processing circuit 1001 may be dedicated hardware as shown in FIG. 17A, or a processor 1004 that executes a program stored in memory as shown in FIG. 17B.

[0109] If the processing circuit 1001 is dedicated hardware, the processing circuit 1001 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0110] When the processing circuit is the processor 1004, the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1005. The processor 1004 reads and executes the program stored in the memory 1005 to perform the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown). In other words, the smoking determination device 1 includes the memory 1005 for storing a program that, when executed by the processor 1004, results in the execution of steps ST10 to ST40 of FIG. 3 or steps ST10 to ST40 of FIG. 16. In addition, the program stored in memory 1005 can also be said to cause the computer to execute the processing procedures or methods of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit not shown. Here, the memory 1005 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

[0111] Note that the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and the control unit (not shown) may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the function of the alarm control unit 13 may be implemented by a processing circuit 1001 as dedicated hardware, while the functions of the smoker determination unit 11, the smoking determination unit 12, 12a, the mask wearing determination unit 14, and the control unit (not shown) may be implemented by a processor 1004 reading and executing programs stored in a memory 1005. The smoking determination device 1 also includes an input interface device 1002 and an output interface device 1003 that communicate via wired or wireless communication with devices such as the camera 2, the smoker determination information acquisition device 3, or the alarm device 4.

[0112] In the first embodiment described above, the "smoking state" is assumed to be a state in which a cigarette is held in the mouth. In other words, a subject is considered to be smoking if the subject actually has a cigarette in his or her mouth. However, this is merely an example, and the "smoking state" may also include, for example, a state in which the subject does not have a cigarette in his or her mouth but has a cigarette in his or her hand.

[0113] Furthermore, in the first embodiment described above, the smoking determination device 1, 1a is provided with the alarm control unit 13, but this is merely an example, and the smoking determination device 1, 1a is not required to have the alarm control unit 13. The function of the alarm control unit 13 may be provided by another device external to the smoking determination device 1, 1a and connected to the smoking determination device 1, 1a. If the smoking determination device 1 is configured without the alarm control unit 13, step ST40 is omitted in the operation of the smoking determination device 1 described using the flowchart of FIG. 3 and the operation of the smoking determination device 1a described using the flowchart of FIG. 16.

[0114] In the first embodiment described above, the smoking determination device 1, 1a is an in-vehicle device mounted on a vehicle, and the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) are provided in the in-vehicle device. However, this is not limiting. For example, a system may be configured with the in-vehicle device and the server, with some of the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) being mounted on the in-vehicle device of the vehicle, and the others being provided in a server connected to the in-vehicle device via a network. Furthermore, for example, the smoker determination unit 11, the smoking determination unit 12, 12a, the alarm control unit 13, the mask wearing determination unit 14, and a control unit (not shown) all being provided in the server.

[0115] Furthermore, in the above-described first embodiment, the subject is the driver of the vehicle, but this is merely an example. The subject may be a vehicle occupant other than the driver. Furthermore, the subject is not limited to a vehicle occupant. For example, the subject may be an occupant of a moving body other than a vehicle, or a person present in a room or the like.

[0116] Furthermore, when the faces of multiple people are captured in the image captured by camera 2, smoking determination device 1 can determine whether each of the multiple people is a smoker, and perform smoking determination only for those subjects determined to be smokers. When the faces of multiple people are captured in the image captured by camera 2, smoking determination device 1a can determine whether each of the multiple people is a smoker and whether they are wearing a mask, and perform smoking determination only for those subjects who are not wearing a mask and are determined to be smokers.

[0117] As described above, according to the first embodiment, the smoking determination device 1, 1a is configured to include the smoker determination unit 11 that determines whether a subject, whose smoking status is to be determined, is a smoker based on smoker determination information for determining whether the subject is a smoker, and the smoking determination unit 12 that, if the smoker determination unit 11 determines that the subject is a smoker, makes a smoking determination as to whether the subject is a smoker based on an image in which at least the face of the subject is captured, and does not make a smoking determination if the smoker determination unit 11 determines that the subject is a non-smoker. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is a smoker.

[0118] In detail, for example, the information for determining whether a smoker is a captured image, and in the smoking determination device 1, 1a, the smoker determination unit 11 calculates, based on the captured image, the mouth non-detection time during which the subject's mouth was not detected during the mouth detection cycle during the smoking determination period, and determines that the subject is a smoker if the mouth non-detection time is equal to or greater than a mouth non-detection time determination threshold. If the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 determines whether the subject is smoking based on the captured image in which at least the subject's face is captured. If the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is smoking.

[0119] Furthermore, for example, the smoker determination information is smoker setting information indicating whether the subject is a smoker or a non-smoker, generated based on the subject's input of instruction information prompting the subject to register whether the subject is a smoker or a non-smoker. In the smoking determination device 1, 1a, the smoker determination unit 11 determines that the subject is a smoker if the subject is set as a smoker in the smoker setting information. If the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 determines whether the subject is smoking based on a captured image that captures at least the subject's face. If the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is smoking.

[0120] Furthermore, for example, the smoker determination information is personal authentication information, and in the smoking determination device 1, 1a, the smoker determination unit 11 performs personal authentication of the subject based on the personal authentication information, and if the authenticated subject is registered in a smoker registration list in which smokers are registered, the smoker determination unit 11 determines that the subject is a smoker. If the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 determines whether the subject is smoking based on a captured image in which at least the subject's face is captured. If the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is smoking.

[0121] Furthermore, for example, the information for determining whether a smoker is a smoker is nicotine dependence information indicating the subject's level of nicotine dependence, and in the smoking determination device 1, 1a, the smoker determination unit 11 determines that the subject is a smoker if the subject is estimated to have a high level of nicotine dependence based on the nicotine dependence information. If the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 determines whether the subject is smoking based on a captured image that captures at least the subject's face, and does not determine that the subject is a non-smoker if the smoker determination unit 11 determines that the subject is a non-smoker. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is a smoker.

[0122] Furthermore, for example, the smoker determination information is heat source information regarding the temperature around the subject's face. In the smoking determination device 1, 1a, the smoker determination unit 11 determines that the subject is a smoker based on the time-series heat source information if it detects a high-temperature area around the subject's face that is equal to or greater than the temperature determination threshold for a period of time equal to or greater than the high-temperature determination time, or if the frequency of high-temperature areas around the subject's face is equal to or greater than the high-temperature occurrence frequency determination threshold. If the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 determines whether the subject is smoking based on a captured image that captures at least the subject's face. If the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is smoking.

[0123] Furthermore, for example, the information for determining whether a person is a smoker is breathing information of the subject, and in the smoking determination device 1, 1a, the smoker determination unit 11 detects smoking breaths estimated to be associated with smoking behavior based on the time-series breathing information, and determines that the subject is a smoker if the number of smoking breaths during the smoking determination period is equal to or greater than a smoking breath determination threshold. If the smoker determination unit 11 determines that the subject is a smoker, the smoking determination unit 12 determines whether the subject is smoking based on captured images that capture at least the subject's face. If the smoker determination unit 11 determines that the subject is a non-smoker, the smoking determination unit 12 does not perform a smoking determination. Therefore, the smoking determination device 1, 1a can prevent erroneously determining that a non-smoker is smoking.

[0124] Furthermore, according to the first embodiment, the smoking determination device 1a includes, in addition to the smoker determination unit 11 and the smoking determination unit 12, a mask wearing determination unit 14 that determines whether the subject is wearing a mask based on the captured image, and if the mask wearing determination unit 14 determines that the subject is wearing a mask, the smoking determination unit 12a does not perform a smoking determination regardless of whether the smoker determination unit 11 has determined that the subject is a smoker. This prevents the smoking determination device 1a from erroneously determining that a non-smoker is smoking and from making unnecessary smoking determinations when the subject is wearing a mask.

[0125] Furthermore, according to the first embodiment, the smoking determination device 1, 1a can be configured to include an alarm control unit 13 that outputs alarm output control information for outputting an alarm when the smoking determination unit 12 determines that the subject is smoking. This allows the smoking determination device 1, 1a to notify the subject or a person other than the subject that the subject is smoking.

[0126] In addition, in the present disclosure, any of the components of the embodiments may be modified or omitted.

[0127] The smoking determination device of the present disclosure can prevent erroneous determination that a non-smoker is smoking.

[0128] 1, 1a smoking determination device, 11 smoker determination unit, 12, 12a smoking determination unit, 13 alarm control unit, 14 mask wearing determination unit, 2, 3a camera, 3 smoker determination information acquisition device, 3b smoker setting device, 3c personal authentication information acquisition device, 3d nicotine dependence determination device, 3e temperature measurement device, 3f breathing detection device, 1001 processing circuit, 1002 input interface device, 1003 output interface device, 1004 processor, 1005 memory.

Claims

1. A smoker determination unit that determines whether or not the subject person who is the subject of determination as to whether or not he / she is in a smoking state is a smoker based on smoker determination information; A smoking determination unit that, when the smoker determination unit determines that the subject person is a smoker, performs a smoking determination as to whether or not the subject person is in a smoking state based on a captured image in which at least the face of the subject person is captured, and does not perform the smoking determination when the smoker determination unit determines that the subject person is a non-smoker A smoking determination device comprising:

2. The smoker determination information is the captured image, Based on the captured image, the smoker determination unit calculates a mouth non-detection time during which the mouth of the subject person has not been detected at a mouth detection cycle during a smoking determination period, and when the mouth non-detection time is equal to or greater than a mouth non-detection time determination threshold, determines that the subject person is a smoker The smoking determination device according to claim 1, characterized in that

3. The smoker determination information is smoker setting information indicating whether or not the subject person is a smoker, which is generated based on an input of setting information for setting whether the subject person is a smoker or a non-smoker by the subject person, Based on the smoker setting information, the smoker determination unit determines that the subject person is a smoker when the subject person is set as a smoker in the smoker setting information The smoking determination device according to claim 1, characterized in that

4. The smoker determination information is personal authentication information, Based on the personal authentication information, the smoker determination unit performs personal authentication of the subject person, and when the subject person who has been personally authenticated is registered in a smoker registration list in which smokers are registered, determines that the subject person is a smoker The smoking determination device according to claim 1, characterized in that

5. The smoker determination information is nicotine dependence information indicating the nicotine dependence degree of the subject person, Based on the nicotine dependence information, the smoker determination unit determines that the subject person is a smoker when it is estimated that the nicotine dependence degree of the subject person is high The smoking determination device according to claim 1, characterized in that

6. The smoker determination information is heat source information regarding the temperature around the face of the subject person, When the smoker determination unit detects a high-temperature region equal to or higher than the temperature determination threshold for a time equal to or longer than the high-temperature determination time around the face of the subject based on the time-series heat source information, or when the appearance frequency of the high-temperature region around the face of the subject is equal to or higher than the high-temperature appearance frequency determination threshold, the smoker determination unit determines that the subject is a smoker. The smoking determination device according to claim 1, characterized in that.

7. The smoker determination information is the breathing information of the subject, The smoker determination unit detects a smoking breath that is estimated to be a breath associated with a smoking action based on the time-series breathing information, and when the number of smoking breaths during the smoking determination period is equal to or more than the smoking breath determination threshold, the smoker determination unit determines that the subject is a smoker. The smoking determination device according to claim 1, characterized in that.

8. A mask wearing determination unit that determines whether or not the subject is wearing a mask based on the captured image is provided, When the mask wearing determination unit determines that the subject is wearing the mask, the smoking determination unit does not perform the smoking determination regardless of whether or not the smoker determination unit determines that the subject is a smoker. The smoking determination device according to any one of claims 1 to 7, characterized in that.

9. An alarm control unit that outputs alarm output control information for outputting an alarm when the smoker determination unit determines that the subject is in a smoking state. The smoking determination device according to any one of claims 1 to 7, comprising.

10. The subject is a driver of a vehicle. The smoking determination device according to any one of claims 1 to 7, characterized in that.

11. The smoker determination unit performs the smoking determination based on a machine learning model that takes as input the captured image in which at least the face of a person is captured and outputs information indicating whether or not the person is in a smoking state, and the captured image in which at least the face of the subject is captured. The smoking determination device according to any one of claims 1 to 7, characterized in that.

12. A step of determining whether or not the subject is a smoker based on smoker determination information for determining whether or not the subject to be determined whether or not is in a smoking state is a smoker, by the smoker determination unit. When the smoking determination unit determines that the subject is a smoker based on the determination by the smoker determination unit that the subject is a smoker, the smoking determination unit performs a smoking determination as to whether or not the subject is in a smoking state based on an imaging image in which at least the face of the subject is imaged. When the smoker determination unit determines that the subject is a non-smoker, the smoking determination is not performed. A smoking determination method comprising the above.